CN110738061A - Ancient poetry generation method, device and equipment and storage medium - Google Patents

Ancient poetry generation method, device and equipment and storage medium Download PDF

Info

Publication number
CN110738061A
CN110738061A CN201910987761.3A CN201910987761A CN110738061A CN 110738061 A CN110738061 A CN 110738061A CN 201910987761 A CN201910987761 A CN 201910987761A CN 110738061 A CN110738061 A CN 110738061A
Authority
CN
China
Prior art keywords
ancient
target
chinese
keywords
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910987761.3A
Other languages
Chinese (zh)
Other versions
CN110738061B (en
Inventor
刘艳鹏
张引
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sohu Internet Information Service Co Ltd
Original Assignee
Beijing Sohu Internet Information Service Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sohu Internet Information Service Co Ltd filed Critical Beijing Sohu Internet Information Service Co Ltd
Priority to CN201910987761.3A priority Critical patent/CN110738061B/en
Priority claimed from CN201910987761.3A external-priority patent/CN110738061B/en
Publication of CN110738061A publication Critical patent/CN110738061A/en
Application granted granted Critical
Publication of CN110738061B publication Critical patent/CN110738061B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

According to the ancient poetry generating method, the ancient poetry generating device, the ancient poetry generating equipment and the ancient poetry generating storage medium, keyword extraction is carried out on multi-modal information, standardized processing of the multi-modal information is achieved, more modal information can be received, more -extensive practical application scenes can be supported, user intentions can be fully reflected, a data base is provided for processing of the multi-modal information through probability statistical results of the modal information, the ancient poetry mapping knowledge graph is used for solving the mapping problem between modern Chinese and ancient Chinese, finally, the obtained ancient poetry keywords are determined to be ancient poetry subject terms, target ancient poetry corresponding to the ancient poetry subject terms is generated through a pre-trained ancient poetry generating model, accordingly, the multi-modal information is fully fused in the ancient poetry generating process, user intentions are more fully reflected, and the ancient poetry generating quality is effectively improved.

Description

Ancient poetry generation method, device and equipment and storage medium
Technical Field
The invention relates to the technical field of natural language processing, in particular to a method, a device, equipment and a storage medium for generating ancient poems.
Background
Natural Language Generation (NLG) technology is becoming more and more popular in industrial applications, and with the rapid development of Natural Language Processing (NLP) technology, generating coherent natural language utterances from a given topic (structured data) becomes an emerging topic of hot , and it becomes possible to make a computer perform the creation of ancient poems using machine learning.
In addition , in practical application, for the generation of ancient poems, information given by users is usually multi-modal, for example, the users may give various types of information such as texts, images, videos, audio and the like at the same time, and how to improve the ancient poem generation quality by fusing the multi-modal information does not have a good solution at present.
Therefore, there is an urgent need for practical and effective ancient poetry generating schemes to fuse multi-modal information to improve the quality of the ancient poetry generating results.
Disclosure of Invention
In view of this, the invention provides ancient poetry generating methods and devices, so as to solve the technical problems that the ancient poetry generating quality is poor based on single modal information, and the quality of the ancient poetry generating result is difficult to improve by fusing multi-modal information at present.
In order to achieve the purpose, the invention provides the following technical scheme:
ancient poetry generating method, comprising:
acquiring multi-modal information input by a user, wherein the multi-modal information comprises at least two of images, videos, texts and audios;
extracting keywords from each modal information in the multi-modal information respectively to obtain a modern Chinese keyword corresponding to each modal information respectively as a target modern Chinese keyword;
acquiring the occurrence probability of each modal information according to historical modal information statistical data;
acquiring ancient Chinese keywords with the highest relevance with the target modern Chinese keywords according to a pre-constructed ancient poetry mapping knowledge graph, and taking the ancient Chinese keywords as the target ancient Chinese keywords;
determining the weight of the target ancient Chinese keywords according to the correlation degree between the target ancient Chinese keywords and the target modern Chinese keywords and the occurrence probability of each modal information;
determining a preset number of ancient Chinese keywords with the maximum weight in the target ancient Chinese keywords as ancient poem topic words;
and generating a target ancient poetry corresponding to the ancient poetry theme by utilizing a pre-trained ancient poetry generating model.
Preferably, the extracting keywords from each modal information in the multi-modal information to obtain the modern chinese keywords corresponding to each modal information respectively, and the extracting keywords as the target modern chinese keywords include:
when the multi-mode information comprises a text, identifying keywords in the text by using a named entity identification technology to obtain modern Chinese keywords corresponding to the text, and using the modern Chinese keywords as target modern Chinese keywords;
when the multi-mode information comprises audio, firstly, converting the voice in the audio into a voice recognition text by using a voice recognition technology, and then, recognizing key words in the voice recognition text by using a named entity recognition technology to obtain modern Chinese key words corresponding to the audio as target modern Chinese key words;
when the multi-mode information comprises an image, firstly identifying the content information of the image by using an image identification technology, and then determining a modern Chinese keyword corresponding to the image as a target modern Chinese keyword according to the content information of the image;
when the multi-mode information comprises a video, identifying a key frame image in the video by using a video understanding technology, identifying content information of the key frame image by using an image identification technology, and determining a modern Chinese keyword corresponding to the video as a target modern Chinese keyword according to the content information of the key frame image.
Preferably, the obtaining of the occurrence probability of each modal information according to the statistical data of the historical modal information includes:
the probability of occurrence P (B) of the target mode information Bi is calculated by the following formulai):
P(Bi)=COUNT(Bi)/COUNT(B),i∈[1,n],B=∑Bi
Wherein, COUNT (B)i) Representing target modal information B in the multi-modal informationiNumber of occurrences in historical modality information; count (B) represents the total number of occurrences of all modality information B in the history modality information in the multimodal information.
Preferably, the representation form of the modal information in the multi-modal information includes single-modal information and multi-modal combined information.
Preferably, the determining the weight of the target ancient chinese keyword according to the degree of correlation between the target ancient chinese keyword and the target modern chinese keyword and the occurrence probability of each modal information includes:
determining the probability corresponding to the target ancient Chinese keywords according to the corresponding relation between the modal information and the target modern Chinese keywords, the incidence relation between the target modern Chinese keywords and the target ancient Chinese keywords and the occurrence probability of the modal information;
and taking the product of the corresponding relevancy and the probability of the target ancient Chinese key words as the weight of the target ancient Chinese key words.
Preferably, the determining the preset number of ancient Chinese keywords with the maximum weight in the target ancient Chinese keywords as the ancient poetry topic words comprises:
when the number of the target ancient Chinese keywords is larger than or equal to a preset number, taking the ancient Chinese keywords with the maximum weight in the ancient Chinese keywords as ancient poem subject terms;
and when the number of the target ancient Chinese keywords is less than the preset number, expanding the target ancient Chinese keywords by using ancient Chinese Word vectors trained in advance by means of Topical Word Embedding technology to obtain the target ancient Chinese keywords with the preset number, and using the target ancient Chinese keywords as ancient poem subject terms.
Preferably, the generating of the target ancient poetry corresponding to the subject term of the ancient poetry by using the pre-trained ancient poetry generating model comprises:
acquiring a pre-trained ancient poetry generating model;
and inputting the subject terms of the ancient poetry into the ancient poetry generating model, and acquiring the target ancient poetry output by the ancient poetry generating model.
ancient poetry generating device, comprising:
the system comprises an input information acquisition unit, a processing unit and a display unit, wherein the input information acquisition unit is used for acquiring multi-modal information input by a user, and the multi-modal information comprises at least two of images, videos, texts and audios;
the modern vocabulary extraction unit is used for respectively extracting keywords from each modal information in the multi-modal information to obtain a modern Chinese keyword corresponding to each modal information as a target modern Chinese keyword;
the modal probability obtaining unit is used for obtaining the occurrence probability of each modal information according to the statistical data of the historical modal information;
the ancient Chinese vocabulary matching unit is used for acquiring an ancient Chinese keyword with the highest relevance with the target modern Chinese keyword as the target ancient Chinese keyword according to a pre-constructed ancient poetry mapping knowledge graph;
the ancient Chinese weight determining unit is used for determining the weight of the target ancient Chinese key words according to the correlation degree between the target ancient Chinese key words and the target modern Chinese key words and the occurrence probability of each modal information;
a poetry theme determining unit, configured to determine a preset number of ancient Chinese keywords with the largest weight in the target ancient Chinese keywords as ancient poetry theme words;
and the target poetry generating unit is used for generating target ancient poetry corresponding to the subject term of the ancient poetry by utilizing an ancient poetry generating model trained in advance.
ancient poetry generating equipment, comprising a processor and a memory;
the memory is used for storing programs;
the processor is used for executing the program stored in the memory so as to realize the ancient poetry generating method.
kinds of computer-readable storage media having stored therein a program that, when executed by a processor, implements the ancient poetry generating method described above.
According to the ancient poetry generating method, the ancient poetry generating device, the ancient poetry generating equipment and the storage medium, the multi-modal information is subjected to keyword extraction, standardized processing of the multi-modal information is achieved, more modal information can be received, more -extensive practical application scenes are supported, the user intention is fully reflected, then a data base is provided for processing of the multi-modal information through probability statistical results of the modal information, the mapping problem between modern Chinese and ancient Chinese is solved by using an ancient poetry mapping knowledge map, finally, the obtained ancient poetry keywords with the preset number are determined as ancient poetry theme words, and target ancient poetry words corresponding to the ancient poetry theme words are generated by using a pre-trained ancient poetry generating model, so that the multi-modal information is fully fused in the ancient poetry generating process, the user intention is more fully embodied, and the ancient poetry generating quality is effectively improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is kinds of flow charts of the method for generating ancient poems according to the embodiment of the present invention;
FIG. 2 is another flowcharts of the method for generating ancient poetry according to the embodiment of the present invention;
FIG. 3 is an exemplary diagram of an ancient poetry mapping knowledge-graph provided by an embodiment of the present invention;
FIG. 4 is a schematic processing flow diagram of an ancient poetry generating system provided by an embodiment of the invention;
fig. 5 is a schematic structural diagram of an ancient poetry generating device provided by the embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only partial embodiments of of the present invention, rather than all embodiments.
The invention provides a technical scheme for generating ancient poetry based on multi-modal information, which can support more modal information as user input information, mainly comprises the normalized processing, the fusion processing and the mapping processing from modern Chinese to ancient Chinese of the multi-modal information, and also provides a subject word expansion scheme which can ensure the conformity of a subject when the subject words are insufficient.
Referring to fig. 1, fig. 1 is a flow chart of methods for generating ancient poems according to an embodiment of the present invention.
As shown in fig. 1, the method for generating ancient poetry of the present embodiment may include:
s101: multi-modal information input by a user is obtained, wherein the multi-modal information comprises at least two of images, videos, texts and audios.
The multi-modal information inputted by the user is an initial material for generating the ancient poetry, and the expression form of the modal information in the multi-modal information may include single-modal information and multi-modal combined information, wherein the separately existing images, videos, texts, audios and the like are all single-modal information, and when different single-modal information are fused with each other at , the multi-modal combined information is a multi-modal combined information, for example, text and image information, text and short video information, audio and image information and the like.
S102: and respectively extracting keywords from each modal information in the multi-modal information to obtain a modern Chinese keyword corresponding to each modal information as a target modern Chinese keyword.
Since the modal information has diversity, it is necessary to normalize the modal information by extracting the modern chinese keywords corresponding to the modal information and using the modern chinese keywords as the normalization result of the modal information.
When performing normalization processing, different modality information needs to be processed, and the modality information mainly related in an actual scene includes images, videos, texts, audios and the like, and mainly includes the following four processing modes a1 to a 4:
a1, when the multi-mode information comprises a text, using a named entity recognition technology to recognize the keywords in the text, and obtaining the modern Chinese keywords corresponding to the text as the target modern Chinese keywords.
For example, for the white language text, named entity recognition is performed to obtain entity keyword information in the white language text and add the entity keyword information to the topic keyword set.
A2, when the multi-mode information comprises audio, firstly, converting the voice in the audio into a voice recognition text by using a voice recognition technology, and then, recognizing the keywords in the voice recognition text by using a named entity recognition technology to obtain the modern Chinese keywords corresponding to the audio as the target modern Chinese keywords.
For example, for audio, the deep learning model may be used to convert the speech in the audio into text, and the subsequent processing is similar to the processing of the white language text, and corresponding entity keyword information is obtained and added to the topic keyword set.
A3, when the multi-mode information includes an image, firstly using an image recognition technology to recognize the content information of the image, and then determining the modern Chinese keywords corresponding to the image according to the content information of the image, wherein the modern Chinese keywords are used as the target modern Chinese keywords.
The content information of the image may include information such as an object and a scene.
For example, for an image, image object and scene recognition is performed, and corresponding modern Chinese keywords and confidence levels are obtained and added to the topic keyword set.
A4, when the multi-mode information comprises a video, firstly identifying a key frame image in the video by using a video understanding technology, then identifying content information of the key frame image by using an image identification technology, and then determining a modern Chinese keyword corresponding to the video according to the content information of the key frame image to be used as a target modern Chinese keyword (or performing content understanding by using other time sequence-based video understanding technologies).
For example, for a video, firstly, key frame recognition is performed, conversion from a dynamic video to a static image is realized, a key frame image and a frame frequency are obtained, then, according to an image processing method, corresponding modern Chinese keywords and confidence coefficients are recognized and added into a topic keyword set.
S103: and acquiring the occurrence probability of each modal information according to the statistical data of the historical modal information.
The probability of the occurrence of each modal information in the historical modal information can be obtained according to the law of large numbers.
S104: and acquiring the ancient Chinese keywords with the highest relevance with the target modern Chinese keywords as the target ancient Chinese keywords according to a pre-constructed ancient poetry mapping knowledge graph.
The method has the advantages that the significance difference gap exists between the ancient Chinese language and the modern Chinese language, and in order to solve the Mapping problem between the ancient Chinese language and the modern Chinese language, the invention constructs a ancient Poetry Mapping Knowledge map (PMKG) in advance, and the Mapping relation between the ancient Chinese keywords and the modern Chinese keywords is provided.
With respect to the ancient poetry mapping knowledge graph, reference may be made to the specific example shown in fig. 3, wherein circles with shading represent modern chinese words and circles with no shading represent ancient chinese words, and the degree of correlation between modern chinese words and ancient chinese words may be calculated using TF-IDF (term frequency-inverse text frequency) technique, it should be noted that fig. 3 shows only examples of the ancient poetry mapping knowledge graph of the present invention, and the ancient poetry mapping knowledge graph provided by the present invention may also cover more vocabulary data and is not limited to the content shown in fig. 3.
According to the ancient poetry mapping knowledge graph established in advance, the ancient Chinese keywords most relevant to the target modern Chinese keywords can be sequentially obtained and used as the target ancient Chinese keywords.
S105: and determining the weight of the target ancient Chinese keywords according to the correlation between the target ancient Chinese keywords and the target modern Chinese keywords and the occurrence probability of each modal information.
In the example, the product of the relevance and the probability corresponding to the target ancient Chinese keyword can be used as the weight of the target ancient Chinese keyword, wherein the probability corresponding to the ancient Chinese keyword is, in effect, the probability of occurrence of the corresponding modal information.
In addition, ancient chinese keywords and their weights may be added to the keyword dictionary.
S106: and determining the preset number of ancient Chinese keywords with the maximum weight in the target ancient Chinese keywords as ancient poem theme words.
The preset number refers to the number of ancient chinese keywords required for generating ancient poems.
S107: and generating a target ancient poetry corresponding to the ancient poetry theme by utilizing a pre-trained ancient poetry generating model.
The ancient poetry generating model can be a Seq2Seq model obtained by training according to ancient poetry theme word samples.
The ancient poetry generating method provided by the embodiment has the advantages that the multi-modal information is subjected to keyword extraction, the multi-modal information is subjected to standardized processing, more modal information can be received, more extensive practical application scenes are supported, the user intention is fully reflected, then a data base is provided for the multi-modal information processing through the probability statistical result of each modal information, the mapping problem between modern Chinese and ancient Chinese is solved by using an ancient poetry mapping knowledge map, finally, the obtained ancient poetry keywords with the preset number are determined as the ancient poetry subject terms, and the target ancient poetry corresponding to the ancient poetry subject terms is generated by using the pre-trained ancient poetry generating model, so that the multi-modal information is fully fused in the ancient poetry generating process, the user intention is more fully embodied, and the ancient poetry generating quality is effectively improved.
Referring to fig. 2, fig. 2 is another flowcharts of the method for generating ancient poetry according to the embodiment of the present invention.
As shown in fig. 2, the method for generating ancient poems according to the embodiment includes:
s201: multimodal information input by a user is obtained.
The multimodal information includes at least two of images, video, text, and audio.
S202: and extracting the target modern Chinese keywords.
And respectively extracting keywords from each modal information in the multi-modal information to obtain a modern Chinese keyword corresponding to each modal information as a target modern Chinese keyword.
Steps S201 to S202 are similar to steps S101 to S102 in the foregoing embodiment , and are not repeated herein.
S203: and calculating the occurrence probability of each modal information.
Specifically, according to the law of large numbers, the probability P (B) of occurrence of the target mode information Bi can be calculated using the following formulai):
P(Bi)=COUNT(Bi)/COUNT(B),i∈[1,n],B=∑Bi
Wherein, COUNT (B)i) Representing target modal information B in the multi-modal informationiNumber of occurrences in historical modality informationCounting; count (B) represents the total number of occurrences of all modality information B in the history modality information in the multimodal information. The target modality information BiIs any modality information in the all modality information B.
S204: and acquiring the target ancient Chinese keywords.
And acquiring the ancient Chinese keywords with the highest relevance with the target modern Chinese keywords as the target ancient Chinese keywords according to a pre-constructed ancient poetry mapping knowledge graph.
Step S204 corresponds to step S104 of the previous embodiment and step , which is not described herein again.
S205: and acquiring the weight of the target ancient Chinese keywords.
Taking the product of the corresponding relevancy and the probability of the target ancient Chinese key words as the weight of the target ancient Chinese key words;
the relevancy corresponding to the target ancient Chinese key words is the relevancy between the target ancient Chinese key words and the target modern Chinese key words; and the probability corresponding to the target ancient Chinese key words is the occurrence probability of the modal information corresponding to the target ancient Chinese key words.
Specifically, before step S205, the method may further include: and determining the probability corresponding to the target ancient Chinese keywords according to the corresponding relation between the modal information and the target modern Chinese keywords, the incidence relation between the target modern Chinese keywords and the target ancient Chinese keywords and the occurrence probability of the modal information. That is, according to the correspondence and the association relationship, the correspondence between the target ancient chinese language keyword and the modal information can be found, and the occurrence probability of the modal information is used as the probability corresponding to the corresponding target ancient chinese language keyword.
S206: judging whether the COUNT (D) is greater than or equal to K, if yes, executing S207; if not, go to step S208.
And judging whether the COUNT (D) is more than or equal to K, namely judging whether the number of the target ancient Chinese keywords is more than or equal to the preset number.
Wherein, count (D) represents the number of the target ancient Chinese keywords, D represents the target ancient Chinese keywords, and K is a preset number. The K value can be set manually or determined automatically according to historical experience, and only the K value is required to meet the vocabulary requirement for generating ancient poems.
S207: and removing redundancy of the target ancient Chinese keywords to obtain the subject terms of the ancient poems.
And taking the ancient Chinese keywords with the maximum weight in the ancient Chinese keywords as ancient poem subject terms.
That is to say, when the number of the target ancient Chinese keywords is greater than or equal to the preset number, the ancient Chinese keywords with the maximum weight in the ancient Chinese keywords are used as the ancient poetry theme words.
In the example, the target ancient Chinese keywords may be sorted according to their weights, and if count (D) is greater than or equal to K, Top-K ancient Chinese keywords in D are selected as ancient poetry topic words, and Top-K represents K Top-ranked words.
Through the step S207, when the ancient poetry theme words are obtained, the redundant target ancient Chinese keywords with lower weight can be effectively filtered, and the redundancy problem of the target ancient Chinese keywords is further solved.
S208: and expanding the target ancient Chinese keywords to obtain ancient poetry theme words.
And expanding the target ancient Chinese keywords by using ancient Chinese Word vectors trained in advance through a topic Word Embedding technology to obtain a preset number of target ancient Chinese keywords as ancient poem topic words.
The topic Word Embedding technology is used for learning Word vectors based on words and topics, so that the generated Word vectors are closer to Word vectors of topics, and the technology can ensure consistency of expanded keywords on the topics.
That is, when the number of the target ancient Chinese keywords is less than the preset number, the target ancient Chinese keywords are expanded by using ancient Chinese Word vectors trained in advance by means of Topical Word Embedding technology to obtain the preset number of target ancient Chinese keywords as ancient poem subject terms.
In the example, the nearest neighbors (count (d) -K) of the obtained target ancient chinese keywords can be found from the ancient chinese Word vectors trained by local Word Embedding, wherein cosine similarity is used as a dimension, so as to expand the target ancient chinese keywords, ensure the theme consistence of the ancient poetry theme words, and further effectively improve the ancient poetry generation quality.
S209: and generating target ancient poems.
And generating a target ancient poetry corresponding to the ancient poetry theme by utilizing a pre-trained ancient poetry generating model.
In the example, the step S209 may specifically include obtaining a pre-trained ancient poetry generating model, inputting the subject terms of the ancient poetry into the ancient poetry generating model, and obtaining target ancient poetry output by the ancient poetry generating model.
Specifically, a Seq2Seq ancient poetry generating model can be trained according to an ancient poetry theme word sample, and then the ancient poetry theme words obtained here are used as subject code input and decoded to generate target ancient poetry.
The method for generating the ancient poetry provided by the embodiment calculates the occurrence probability of each modal information according to a law of large numbers, provides a sufficient data base for processing multi-modal information, respectively executes corresponding redundancy removal processing or expansion processing according to the number of the obtained target ancient Chinese keywords, thereby solving the redundancy problem and the missing problem of the ancient poetry theme words, filters the target ancient Chinese keywords with low weight during the redundancy removal processing, ensures the accuracy of the ancient poetry theme words, and expands the target ancient Chinese keywords by adopting the ancient Chinese word vectors during the expansion processing, thereby ensuring that the expanded ancient Chinese keywords and the target ancient Chinese keywords have higher theme consistency, and effectively improving the quality of the generated ancient poetry.
Referring to fig. 4, fig. 4 is a schematic processing flow diagram of an ancient poetry generating system according to an embodiment of the present invention.
The basic flow of the ancient poetry generating process mainly comprises the following steps: "normalization- > fusion- > mapping- > extension".
As shown in fig. 4, first, multi-modal information is obtained, which includes images, short video, white text and voice.
Secondly, carrying out user intention recognition and standardization processing on the multi-modal information by using an information processing module, wherein the user intention recognition and standardization processing comprises image recognition on an image, image recognition is carried out after key frame recognition is carried out on a short video, named entity recognition is carried out on a white text, and named entity recognition is carried out after text transcription is carried out on voice; by means of the method, the normalized modern Chinese keywords can be obtained and output, and the intention of the user can be reflected accordingly.
Then, the information processing module is used for selecting and organizing the content of the normalized output result, wherein the content comprises keyword fusion and mapping, namely, obtaining the modern Chinese keywords output after normalization, and carrying out ancient Chinese keyword mapping according to the modern Chinese keywords to obtain the ancient Chinese keywords required by generating the ancient poetry, and the content further comprises subject -caused planning, namely, selecting or expanding the ancient Chinese keywords obtained by mapping to obtain the subject words of the ancient poetry.
And finally, generating an ancient poetry corresponding to the ancient poetry theme by using the ancient poetry generating model.
Wherein, when ancient poetry generation model training, can gather ancient poetry data set in advance to carry out the preprocessing process that text washd, text screening, participle, rhythm were drawed to ancient poetry data set, obtain ancient poetry subject matter word sample, then utilize ancient poetry subject matter word sample to train and obtain ancient poetry generation model.
The processing flow of the ancient poetry generating system provided by the embodiment provides a more comprehensive and complete processing flow of ancient poetry generation from a system level, realizes the standardized processing of multi-mode information by extracting keywords from the multi-mode information, can accept more modal information, supports more -wide practical application scenes, provides a data base for the processing of the multi-mode information through the probability statistical result of each modal information, solves the mapping problem between modern Chinese and ancient Chinese through an ancient poetry mapping knowledge map, finally determines a preset number of ancient poetry keywords as ancient poetry theme words, and generates target ancient poetry corresponding to the ancient poetry theme words by utilizing a pre-trained ancient poetry generating model, so that the multi-mode information is fully fused in the ancient poetry generating process, and the quality of the generated ancient poetry is effectively improved.
The embodiment of the invention also provides an ancient poetry generating device, wherein the ancient poetry generating device is used for implementing the ancient poetry generating method provided by the embodiment of the invention, the technical content of the ancient poetry generating device described below can be correspondingly referred to with the technical content of the ancient poetry generating method described above, and the same or similar content is not repeated.
Referring to fig. 5, fig. 5 is a schematic view of structures of an ancient poetry generating device according to an embodiment of the present invention.
As shown in fig. 5, the apparatus includes: the poetry theme recognition method comprises an input information acquisition unit 100, a modern vocabulary extraction unit 200, a modal probability acquisition unit 300, an ancient vocabulary matching unit 400, an ancient weight determination unit 500, a poetry theme determination unit 600 and a target poetry generation unit 700.
The input information acquiring unit 100 is configured to acquire multi-modal information input by a user, where the multi-modal information includes at least two of an image, a video, a text, and an audio.
The expression form of the modal information in the multimodal information comprises single modal information and multimodal combined information.
And a modern vocabulary extracting unit 200, configured to perform keyword extraction on each modal information in the multimodal information, to obtain a modern chinese keyword corresponding to each modal information, as a target modern chinese keyword.
A modal probability obtaining unit 300, configured to obtain occurrence probabilities of the modal information according to the historical modal information statistical data.
And the ancient language vocabulary matching unit 400 is used for acquiring the ancient Chinese keywords with the highest relevance with the target modern Chinese keywords as the target ancient Chinese keywords according to a pre-constructed ancient poetry mapping knowledge graph.
A ancient language weight determining unit 500, configured to determine a weight of the target ancient chinese keyword according to a correlation between the target ancient chinese keyword and the target modern chinese keyword and an occurrence probability of each modal information.
A poetry topic determination unit 600, configured to determine a preset number of ancient Chinese keywords with the largest weight in the target ancient Chinese keywords as ancient poetry topic words.
And the target poetry generating unit 700 is used for generating target ancient poetry corresponding to the subject term of the ancient poetry by utilizing an ancient poetry generating model trained in advance.
In the example, the modern vocabulary extraction unit 200 is specifically configured to:
when the multi-mode information comprises a text, identifying keywords in the text by using a named entity identification technology to obtain modern Chinese keywords corresponding to the text, and using the modern Chinese keywords as target modern Chinese keywords;
when the multi-mode information comprises audio, firstly, converting the voice in the audio into a voice recognition text by using a voice recognition technology, and then, recognizing key words in the voice recognition text by using a named entity recognition technology to obtain modern Chinese key words corresponding to the audio as target modern Chinese key words;
when the multi-mode information comprises an image, firstly identifying the content information of the image by using an image identification technology, and then determining a modern Chinese keyword corresponding to the image as a target modern Chinese keyword according to the content information of the image; the content information of the image may include information such as an object and a scene.
When the multi-mode information comprises a video, a video understanding technology is used for identifying a key frame image in the video, an image identification technology is used for identifying content information of the key frame image, and then a modern Chinese keyword corresponding to the video is determined according to the content information of the key frame image and is used as a target modern Chinese keyword (or other video understanding technologies based on time sequence for content understanding).
In the example, the modality probability obtaining unit 300 is specifically configured to:
the obtaining of the occurrence probability of each modal information according to the statistical data of the historical modal information includes:
the probability of occurrence P (B) of the target mode information Bi is calculated by the following formulai):
P(Bi)=COUNT(Bi)/COUNT(B),i∈[1,n],B=∑Bi
Wherein, COUNT (B)i) Representing target modal information B in the multi-modal informationiNumber of occurrences in historical modality information; count (B) represents the total number of occurrences of all modality information B in the history modality information in the multimodal information.
In the example, the ancient language weight determination unit 500 is specifically configured to:
determining the probability corresponding to the target ancient Chinese keywords according to the corresponding relation between the modal information and the target modern Chinese keywords, the incidence relation between the target modern Chinese keywords and the target ancient Chinese keywords and the occurrence probability of the modal information;
and taking the product of the corresponding relevancy and the probability of the target ancient Chinese key words as the weight of the target ancient Chinese key words.
In the example, the verse theme determination unit 600 is specifically configured to:
when the number of the target ancient Chinese keywords is larger than or equal to a preset number, taking the ancient Chinese keywords with the maximum weight in the ancient Chinese keywords as ancient poem subject terms;
and when the number of the target ancient Chinese keywords is less than the preset number, expanding the target ancient Chinese keywords by using ancient Chinese Word vectors trained in advance by means of Topical Word Embedding technology to obtain the target ancient Chinese keywords with the preset number, and using the target ancient Chinese keywords as ancient poem subject terms.
In the example, the target poetry generating unit 700 is specifically configured to:
the method for generating the target ancient poetry corresponding to the ancient poetry theme words by utilizing the pre-trained ancient poetry generating model comprises the following steps:
acquiring a pre-trained ancient poetry generating model;
and inputting the subject terms of the ancient poetry into the ancient poetry generating model, and acquiring the target ancient poetry output by the ancient poetry generating model.
The ancient poetry generating device provided by the embodiment realizes the standardized processing of multi-modal information by extracting keywords from the multi-modal information, can further receive more modal information, supports more extensive practical application scenes, and fully reflects the intention of a user, provides a data base for the processing of the multi-modal information through the probability statistical result of each modal information, solves the mapping problem between modern Chinese and ancient Chinese by using an ancient poetry mapping knowledge map, finally determines the obtained preset number of ancient poetry keywords as the ancient poetry subject terms, and generates the target ancient poetry corresponding to the ancient poetry subject terms by using a pre-trained ancient poetry generating model, thereby fully fusing the multi-modal information in the process of generating the ancient poetry, more comprehensively reflecting the intention of the user, and effectively improving the generating quality of the ancient poetry.
And moreover, during the redundancy removal processing, the target ancient Chinese keywords with low weight are filtered, the accuracy of the ancient poetry theme words is ensured, and the target ancient Chinese keywords are expanded by adopting the ancient Chinese word vectors during the expansion processing, so that the expanded ancient Chinese keywords and the target ancient Chinese keywords have higher theme consistency, and the quality of the generated ancient poetry words is effectively improved.
The ancient poetry generating device provided by the embodiment of the invention comprises a processor and a memory, wherein the input information acquisition unit 100, the modern vocabulary extraction unit 200, the modal probability acquisition unit 300, the ancient vocabulary matching unit 400, the ancient language weight determination unit 500, the poetry theme determination unit 600, the target poetry generating unit 700 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls corresponding program units from the memory, wherein or more kernels can be set, and the ancient poetry generating method is realized by adjusting kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), including at least memory chips.
An embodiment of the present invention provides computer-readable storage media having stored therein a program that, when executed by a processor, implements the steps of the aforementioned method of generating ancient poems.
The embodiment of the invention provides processors, wherein the processors are used for running a program, and the program runs to execute the steps of the ancient poetry generating method.
The embodiment of the invention provides ancient poetry generating equipment, which comprises a processor, a memory and a program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the ancient poetry generating method.
The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides computer program products adapted to execute a program for initializing the steps of the method for generating ancient poetry described previously when executed on a data processing device.
Finally, it should also be noted that, in this document, relational terms such as , , and the like are only used to distinguish entities or operations from another entities or operations without requiring or implying any such actual relationship or order between such entities or operations, and furthermore, the terms "comprise," "include," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises the series elements does not include only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.
Based on the understanding that the present disclosure may take the form of a software product, which may be stored on a storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing computer devices (which may be personal computers, servers, or network devices, etc.) to perform the methods described in various embodiments or portions of embodiments of the present disclosure.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present application are described herein using specific examples, which are provided only to help understand the method and the core idea of the present application, and the application scope and the detailed implementation of the method may be changed by persons skilled in the art according to the idea of the present application.

Claims (10)

1, ancient poetry generating method, characterized by comprising:
acquiring multi-modal information input by a user, wherein the multi-modal information comprises at least two of images, videos, texts and audios;
extracting keywords from each modal information in the multi-modal information respectively to obtain a modern Chinese keyword corresponding to each modal information respectively as a target modern Chinese keyword;
acquiring the occurrence probability of each modal information according to historical modal information statistical data;
acquiring ancient Chinese keywords with the highest relevance with the target modern Chinese keywords according to a pre-constructed ancient poetry mapping knowledge graph, and taking the ancient Chinese keywords as the target ancient Chinese keywords;
determining the weight of the target ancient Chinese keywords according to the correlation degree between the target ancient Chinese keywords and the target modern Chinese keywords and the occurrence probability of each modal information;
determining a preset number of ancient Chinese keywords with the maximum weight in the target ancient Chinese keywords as ancient poem topic words;
and generating a target ancient poetry corresponding to the ancient poetry theme by utilizing a pre-trained ancient poetry generating model.
2. The method of claim 1, wherein the performing keyword extraction on each modal information in the multi-modal information respectively to obtain a modern chinese keyword corresponding to each modal information respectively, and the performing the target modern chinese keyword comprises:
when the multi-mode information comprises a text, identifying keywords in the text by using a named entity identification technology to obtain modern Chinese keywords corresponding to the text, and using the modern Chinese keywords as target modern Chinese keywords;
when the multi-mode information comprises audio, firstly, converting the voice in the audio into a voice recognition text by using a voice recognition technology, and then, recognizing key words in the voice recognition text by using a named entity recognition technology to obtain modern Chinese key words corresponding to the audio as target modern Chinese key words;
when the multi-mode information comprises an image, firstly identifying the content information of the image by using an image identification technology, and then determining a modern Chinese keyword corresponding to the image as a target modern Chinese keyword according to the content information of the image;
when the multi-mode information comprises a video, identifying a key frame image in the video by using a video understanding technology, identifying content information of the key frame image by using an image identification technology, and determining a modern Chinese keyword corresponding to the video as a target modern Chinese keyword according to the content information of the key frame image.
3. The method according to claim 1, wherein the obtaining the occurrence probability of each modality information according to the statistical data of the historical modality information comprises:
the probability of occurrence P (B) of the target mode information Bi is calculated by the following formulai):
P(Bi)=COUNT(Bi)/COUNT(B),i∈[1,n],B=∑Bi
Wherein, COUNT (B)i) Representing target modal information B in the multi-modal informationiNumber of occurrences in historical modality information; count (B) represents the total number of occurrences of all modality information B in the history modality information in the multimodal information.
4. The method of claim 1, wherein the representation of modal information in the multimodal information comprises a combination of single modal information and multimodal information.
5. The method of claim 1, wherein said determining a weight of the target ancient Chinese keyword based on a degree of correlation between the target ancient Chinese keyword and the target modern Chinese keyword and a probability of occurrence of the modal information comprises:
determining the probability corresponding to the target ancient Chinese keywords according to the corresponding relation between the modal information and the target modern Chinese keywords, the incidence relation between the target modern Chinese keywords and the target ancient Chinese keywords and the occurrence probability of the modal information;
and taking the product of the corresponding relevancy and the probability of the target ancient Chinese key words as the weight of the target ancient Chinese key words.
6. The method of claim 1, wherein the determining a preset number of the targeted ancient chinese keywords having a greatest weight as ancient poetry topic words comprises:
when the number of the target ancient Chinese keywords is larger than or equal to a preset number, taking the ancient Chinese keywords with the maximum weight in the ancient Chinese keywords as ancient poem subject terms;
and when the number of the target ancient Chinese keywords is less than the preset number, expanding the target ancient Chinese keywords by using ancient Chinese Word vectors trained in advance by means of Topical Word Embedding technology to obtain the target ancient Chinese keywords with the preset number, and using the target ancient Chinese keywords as ancient poem subject terms.
7. The method of claim 1, wherein the generating of the target ancient poetry corresponding to the ancient poetry subject matter using a pre-trained ancient poetry generating model comprises:
acquiring a pre-trained ancient poetry generating model;
and inputting the subject terms of the ancient poetry into the ancient poetry generating model, and acquiring the target ancient poetry output by the ancient poetry generating model.
8, kind of ancient poetry generating device, its characterized in that includes:
the system comprises an input information acquisition unit, a processing unit and a display unit, wherein the input information acquisition unit is used for acquiring multi-modal information input by a user, and the multi-modal information comprises at least two of images, videos, texts and audios;
the modern vocabulary extraction unit is used for respectively extracting keywords from each modal information in the multi-modal information to obtain a modern Chinese keyword corresponding to each modal information as a target modern Chinese keyword;
the modal probability obtaining unit is used for obtaining the occurrence probability of each modal information according to the statistical data of the historical modal information;
the ancient Chinese vocabulary matching unit is used for acquiring an ancient Chinese keyword with the highest relevance with the target modern Chinese keyword as the target ancient Chinese keyword according to a pre-constructed ancient poetry mapping knowledge graph;
the ancient Chinese weight determining unit is used for determining the weight of the target ancient Chinese key words according to the correlation degree between the target ancient Chinese key words and the target modern Chinese key words and the occurrence probability of each modal information;
a poetry theme determining unit, configured to determine a preset number of ancient Chinese keywords with the largest weight in the target ancient Chinese keywords as ancient poetry theme words;
and the target poetry generating unit is used for generating target ancient poetry corresponding to the subject term of the ancient poetry by utilizing an ancient poetry generating model trained in advance.
9, ancient poetry generating equipment, which is characterized by comprising a processor and a memory;
the memory is used for storing programs;
the processor is used for executing the program stored in the memory to realize the ancient poetry generating method as claimed in any of claims 1-7.
10, kinds of computer-readable storage media, characterized in that the computer-readable storage media have stored therein a program which, when executed by a processor, realizes the ancient poetry generating method according to any of claims 1 to 7.
CN201910987761.3A 2019-10-17 Ancient poetry generating method, device, equipment and storage medium Active CN110738061B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910987761.3A CN110738061B (en) 2019-10-17 Ancient poetry generating method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910987761.3A CN110738061B (en) 2019-10-17 Ancient poetry generating method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN110738061A true CN110738061A (en) 2020-01-31
CN110738061B CN110738061B (en) 2024-05-28

Family

ID=

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111259215A (en) * 2020-02-14 2020-06-09 北京百度网讯科技有限公司 Multi-modal-based topic classification method, device, equipment and storage medium
CN112101040A (en) * 2020-08-20 2020-12-18 淮阴工学院 Ancient poetry semantic retrieval method based on knowledge graph
CN112560622A (en) * 2020-12-08 2021-03-26 中国联合网络通信集团有限公司 Virtual object motion control method and device and electronic equipment

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110239099A1 (en) * 2010-03-23 2011-09-29 Disney Enterprises, Inc. System and method for video poetry using text based related media
CN102402593A (en) * 2010-11-05 2012-04-04 微软公司 Multi-modal approach to search query input
US20160124933A1 (en) * 2014-10-30 2016-05-05 International Business Machines Corporation Generation apparatus, generation method, and program
CN106227714A (en) * 2016-07-14 2016-12-14 北京百度网讯科技有限公司 A kind of method and apparatus obtaining the key word generating poem based on artificial intelligence
CN106569995A (en) * 2016-09-26 2017-04-19 天津大学 Method for automatically generating Chinese poetry based on corpus and metrical rule
CN106933807A (en) * 2017-03-20 2017-07-07 北京光年无限科技有限公司 Memorandum event-prompting method and system
CN107480132A (en) * 2017-07-25 2017-12-15 浙江工业大学 A kind of classic poetry generation method of image content-based
CN107832292A (en) * 2017-11-02 2018-03-23 合肥工业大学 A kind of conversion method based on the image of neural network model to Chinese ancient poetry
CN108897734A (en) * 2018-06-13 2018-11-27 康键信息技术(深圳)有限公司 User's portrait generation method, device, computer equipment and storage medium
CN109086270A (en) * 2018-07-24 2018-12-25 重庆大学 System and method of composing poem automatically based on classic poetry corpus vectorization
CN109145102A (en) * 2018-09-06 2019-01-04 杭州安恒信息技术股份有限公司 Intelligent answer method and its knowledge mapping system constituting method, device, equipment
CN109543007A (en) * 2018-10-16 2019-03-29 深圳壹账通智能科技有限公司 Put question to data creation method, device, computer equipment and storage medium
CN109766013A (en) * 2018-12-28 2019-05-17 北京金山安全软件有限公司 Poetry sentence input recommendation method and device and electronic equipment
CN109840287A (en) * 2019-01-31 2019-06-04 中科人工智能创新技术研究院(青岛)有限公司 A kind of cross-module state information retrieval method neural network based and device
CN110134968A (en) * 2019-05-22 2019-08-16 网易(杭州)网络有限公司 Poem generation method, device, equipment and storage medium based on deep learning
CN110147442A (en) * 2019-04-15 2019-08-20 深圳智能思创科技有限公司 A kind of text snippet generation system and method for length-controllable
WO2019174186A1 (en) * 2018-03-15 2019-09-19 平安科技(深圳)有限公司 Automatic poem generation method and apparatus, and computer device and storage medium

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110239099A1 (en) * 2010-03-23 2011-09-29 Disney Enterprises, Inc. System and method for video poetry using text based related media
CN102402593A (en) * 2010-11-05 2012-04-04 微软公司 Multi-modal approach to search query input
US20160124933A1 (en) * 2014-10-30 2016-05-05 International Business Machines Corporation Generation apparatus, generation method, and program
CN106227714A (en) * 2016-07-14 2016-12-14 北京百度网讯科技有限公司 A kind of method and apparatus obtaining the key word generating poem based on artificial intelligence
CN106569995A (en) * 2016-09-26 2017-04-19 天津大学 Method for automatically generating Chinese poetry based on corpus and metrical rule
CN106933807A (en) * 2017-03-20 2017-07-07 北京光年无限科技有限公司 Memorandum event-prompting method and system
CN107480132A (en) * 2017-07-25 2017-12-15 浙江工业大学 A kind of classic poetry generation method of image content-based
CN107832292A (en) * 2017-11-02 2018-03-23 合肥工业大学 A kind of conversion method based on the image of neural network model to Chinese ancient poetry
WO2019174186A1 (en) * 2018-03-15 2019-09-19 平安科技(深圳)有限公司 Automatic poem generation method and apparatus, and computer device and storage medium
CN108897734A (en) * 2018-06-13 2018-11-27 康键信息技术(深圳)有限公司 User's portrait generation method, device, computer equipment and storage medium
CN109086270A (en) * 2018-07-24 2018-12-25 重庆大学 System and method of composing poem automatically based on classic poetry corpus vectorization
CN109145102A (en) * 2018-09-06 2019-01-04 杭州安恒信息技术股份有限公司 Intelligent answer method and its knowledge mapping system constituting method, device, equipment
CN109543007A (en) * 2018-10-16 2019-03-29 深圳壹账通智能科技有限公司 Put question to data creation method, device, computer equipment and storage medium
CN109766013A (en) * 2018-12-28 2019-05-17 北京金山安全软件有限公司 Poetry sentence input recommendation method and device and electronic equipment
CN109840287A (en) * 2019-01-31 2019-06-04 中科人工智能创新技术研究院(青岛)有限公司 A kind of cross-module state information retrieval method neural network based and device
CN110147442A (en) * 2019-04-15 2019-08-20 深圳智能思创科技有限公司 A kind of text snippet generation system and method for length-controllable
CN110134968A (en) * 2019-05-22 2019-08-16 网易(杭州)网络有限公司 Poem generation method, device, equipment and storage medium based on deep learning

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111259215A (en) * 2020-02-14 2020-06-09 北京百度网讯科技有限公司 Multi-modal-based topic classification method, device, equipment and storage medium
CN111259215B (en) * 2020-02-14 2023-06-27 北京百度网讯科技有限公司 Multi-mode-based topic classification method, device, equipment and storage medium
CN112101040A (en) * 2020-08-20 2020-12-18 淮阴工学院 Ancient poetry semantic retrieval method based on knowledge graph
CN112101040B (en) * 2020-08-20 2024-03-29 淮阴工学院 Ancient poetry semantic retrieval method based on knowledge graph
CN112560622A (en) * 2020-12-08 2021-03-26 中国联合网络通信集团有限公司 Virtual object motion control method and device and electronic equipment
CN112560622B (en) * 2020-12-08 2023-07-21 中国联合网络通信集团有限公司 Virtual object action control method and device and electronic equipment

Similar Documents

Publication Publication Date Title
CN106328147B (en) Speech recognition method and device
CN109840321B (en) Text recommendation method and device and electronic equipment
CN110444198B (en) Retrieval method, retrieval device, computer equipment and storage medium
CN110148400B (en) Pronunciation type recognition method, model training method, device and equipment
CN104598644B (en) Favorite label mining method and device
WO2017127296A1 (en) Analyzing textual data
CN108334489B (en) Text core word recognition method and device
CN111445898B (en) Language identification method and device, electronic equipment and storage medium
CN104063417A (en) Picture Drawing Support Apparatus, Method And Program
CN110222168B (en) Data processing method and related device
CN112233680A (en) Speaker role identification method and device, electronic equipment and storage medium
CN108710653B (en) On-demand method, device and system for reading book
CN112489688A (en) Neural network-based emotion recognition method, device and medium
CN112581327A (en) Knowledge graph-based law recommendation method and device and electronic equipment
CN111859950A (en) Method for automatically generating lecture notes
CN111126084A (en) Data processing method and device, electronic equipment and storage medium
CN112151021A (en) Language model training method, speech recognition device and electronic equipment
CN109344388B (en) Method and device for identifying spam comments and computer-readable storage medium
CN110738061A (en) Ancient poetry generation method, device and equipment and storage medium
CN114974310A (en) Emotion recognition method and device based on artificial intelligence, computer equipment and medium
CN114528851A (en) Reply statement determination method and device, electronic equipment and storage medium
CN114780757A (en) Short media label extraction method and device, computer equipment and storage medium
CN110738061B (en) Ancient poetry generating method, device, equipment and storage medium
CN111801673A (en) Application program introduction method, mobile terminal and server
CN108630192B (en) non-Chinese speech recognition method, system and construction method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant