CN107977472B - Method for automatically generating house property news articles - Google Patents

Abstract

The invention relates to a method for automatically generating a house property news article, which comprises the following steps: step (1): preparing a data set, collecting three parts including article titles, abstracts and texts, and summarizing the three parts to form sentences; step (2): carrying out model design and training on the data set in the step (1); and (3): and (3) generating a text after the model training based on the step (2) is finished. The invention has the advantages that: manual control writing is not needed, and manpower is saved; the work is efficient, and the cooperation efficiency is multiple times of that of manual writing; the generation is stable, the context is usually subject to control, and the yellow anti-riot terrorist content is avoided.

Description

Method for automatically generating house property news articles

Technical Field

The invention relates to a method for automatically generating a house property news article.

Background

The release of news articles can increase brand exposure and bring traffic, especially the collection of long tail traffic of search engines. However, the news article automatic generation robot in the real estate field is trained by using an artificial intelligence technology because the news article automatic generation robot directly transfers the news article, the originality is not good enough, and the SEO (search engine optimization) is influenced, but the cost is high because the news article automatic generation robot completely depends on artificial originality. And successfully applied to our content distribution system.

The existing news writing is mostly divided into two types, one is to transfer news, original articles from other sources are crawled at a fast speed by using a crawler technology and are released to a website of the user, the other is to manually write the original articles, manual creation is performed, the articles are automatically generated in the field of splicing, the generated contents are not friendly to people, and the articles are simply used for searching and guiding search engines.

The existing article release modes have the defects that manual creation consumes manpower, reprinting of reptiles is poor in originality, simple in disorganization and splicing and not friendly to people.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a method for automatically generating a house property news article, and the technical scheme of the invention is as follows:

the method for automatically generating the house property news articles comprises the following steps:

step (1): preparing a data set, collecting three parts including article titles, abstracts and texts, and summarizing the three parts to form sentences;

step (2): carrying out model design and training on the data set in the step (1);

and (3): and (3) generating a text after the model training based on the step (2) is finished.

The step (1) specifically comprises the following steps: carrying out statistics, Arabic number processing and initial character and termination character splicing on the word frequency; the word frequency statistics specifically includes performing word frequency statistics on the corpus of the whole data set, and replacing words with the frequency less than 10 times in the data set with set characters; the Arabic number processing is specifically to mark digits of Arabic numbers appearing together in a data set; the splicing of the starting character and the ending character is specifically that a mark character is added to the forefront of each article in the data set to indicate the beginning of the article, and a mark character is added to the back of the end of each article to indicate the end of the article.

The specific method of the step (2) is as follows: constructing a coder-decoder model, and separately training the coder and decoder of the model:

an encoder: searching all training sets by taking the initial character or the whole character generated at the current moment as a searching condition to obtain a plurality of related text sets, and selecting the first plurality of text sets according to an information searching algorithm; coding the taken information, and taking the coding result as a statement suitable for generation;

a decoder: encoding the currently generated sentence;

the specific steps of the attention mechanism are as follows: multiplying the encoder output matrix and the decoder output matrix to obtain a group of weight value vectors, multiplying the weight value vectors by the encoder output matrix again to obtain information of a predicted next character, and obtaining a final word vector for predicting the next character through a softmax activation function.

The specific method of the step (3) is as follows:

(3-1) inputting an initial sentence or randomly selecting an initial character;

(3-2) retrieving the whole data set according to the initial words and sentences, and taking the first plurality of data sets as input information of the encoder;

(3-3) encoding the result obtained by the retrieval;

(3-4) encoding the current generated sentence;

(3-5) obtaining a next character through an attention mechanism;

(3-6) generating a sentence by the beam search method until the terminator occurs.

The invention has the advantages that: manual control writing is not needed, and manpower is saved; the work is efficient, and the cooperation efficiency is multiple times of that of manual writing; the generation is stable, the context is usually subject to control, and the yellow anti-riot terrorist content is avoided.

Detailed Description

The invention will be further described with reference to specific embodiments, and the advantages and features of the invention will become apparent as the description proceeds. These examples are illustrative only and do not limit the scope of the present invention in any way. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention, and that such changes and modifications may be made without departing from the spirit and scope of the invention.

The invention relates to a method for automatically generating a house property news article, which comprises the following steps:

step (1): preparing a data set, collecting three parts including article titles, abstracts and texts, and summarizing the three parts to form sentences;

step (2): carrying out model design and training on the data set in the step (1);

and (3): and (3) generating a text after the model training based on the step (2) is finished.

The step (1) specifically comprises the following steps: carrying out statistics, Arabic number processing and initial character and termination character splicing on the word frequency; the word frequency statistics is specifically to perform word frequency statistics on the corpus of the whole data set, and replace words with the frequency less than 10 times in the data set with set characters, for example, words with the frequency less than 10 can be replaced with < unk >; the arabic number processing is specifically to mark the digits of the arabic numbers appearing together in the data set, for example, the arabic numbers appearing together are replaced with < numN >, and N in parentheses represents the digits of the digits, for example: 1234 should be replaced by < num4>, 342134 should be replaced by < num6>, and the splicing of the start character and the end character is specifically implemented by adding a mark character at the forefront of each article in the data set to indicate the beginning of the article, adding a mark character after the end of each article to indicate the end of the article, using a "< s >" character to indicate the beginning of the article, and adding a "</s >" character after the end of each article to indicate the end of the article.

The specific method of the step (2) is as follows: constructing a coder-decoder model, and separately training the coder and decoder of the model:

an encoder: searching all training sets by taking the initial character or the whole character generated at the current moment as a searching condition to obtain a plurality of related text sets, and selecting the first plurality of text sets according to an information searching algorithm; coding the taken information, and taking the coding result as a statement suitable for generation;

a decoder: encoding the currently generated sentence;

the specific steps of the attention mechanism are as follows: and multiplying the encoder output matrix and the decoder output matrix to obtain a group of weight value vectors, multiplying the weight value vectors by the encoder output matrix again to obtain the information of the next predicted character, and performing a softmax activation function to obtain the final word vector for predicting the next character.

The specific method of the step (3) is as follows:

(3-1) inputting an initial sentence or randomly selecting an initial character;

(3-2) retrieving the whole data set according to the initial words and sentences, and taking the first plurality of data sets as input information of the encoder;

(3-3) encoding the result obtained by the retrieval;

(3-4) encoding the current generated sentence;

(3-5) obtaining a next character through an attention mechanism;

(3-6) generating a sentence by the beam search method until the terminator occurs.

Claims (1)

1. The method for automatically generating the house property news articles is characterized by comprising the following steps of:

step (1): preparing a data set, collecting three parts including article titles, abstracts and texts, and summarizing the three parts to form sentences;

step (2): carrying out model design and training on the data set in the step (1);

and (3): after the model training based on the step (2) is finished, generating a text;

the step (1) specifically comprises the following steps: carrying out statistics, Arabic number processing and initial character and termination character splicing on the word frequency; the word frequency statistics specifically includes performing word frequency statistics on the corpus of the whole data set, and replacing words with the frequency less than 10 times in the data set with set characters; the Arabic number processing is specifically to mark digits of Arabic numbers appearing together in a data set; the splicing of the starting character and the ending character is specifically that a mark character is added to the forefront of each article in the data set to indicate the beginning of the article, and a mark character is added to the back of the end of each article to indicate the end of the article;

the specific method of the step (2) is as follows: constructing a coder-decoder model, and separately training the coder and decoder of the model:

an encoder: searching all training sets by taking the initial character or the whole character generated at the current moment as a searching condition to obtain a plurality of related text sets, and selecting the first plurality of text sets according to an information searching algorithm; coding the taken information, and taking the coding result as a statement suitable for generation;

a decoder: encoding the currently generated sentence;

the specific steps of the attention mechanism are as follows: multiplying the encoder output matrix with the decoder output matrix to obtain a group of weight value vectors, multiplying the weight value vectors with the encoder output matrix again to obtain information of a predicted next character, and obtaining a final word vector for predicting the next character through a softmax activation function;

the specific method of the step (3) is as follows:

(3-1) inputting an initial sentence or randomly selecting an initial character;

(3-2) retrieving the whole data set according to the initial words and sentences, and taking the first plurality of data sets as input information of the encoder;

(3-3) encoding the result obtained by the retrieval;

(3-4) encoding the current generated sentence;

(3-5) obtaining a next character through an attention mechanism;

(3-6) generating a sentence by the beamsearch method until the terminator occurs.