CN112507164B - Barrage filtering method, device and storage medium based on content and user ID - Google Patents

Abstract

The invention discloses a method and device for filtering bullet screens based on content and user identification. The method includes: preprocessing bullet screen data and user data of bullet screen video websites crawled by python crawler software; introducing word embedding, The short text representation method under the combined effect of word similarity, word and topic probability, and tag topic probability expands the short text of the bullet screen; constructs user platform features; splices the expanded text features and platform features into the classification model, Output the barrage classification result. The invention combines the advantages of external corpus expansion and content feature expansion of short text itself, and at the same time, introduces word vector into feature expansion, realizes semantic expansion of original text to the greatest extent, and adds user platform type features to the bullet screen feature space to enrich the bullet screen feature space. Improve the barrage recognition rate.

Description

Barrage filtering method, device and storage medium based on content and user ID

technical field

The present invention relates to the technical field of video bullet screens, in particular to a method, device and storage medium for filtering bullet screens based on content and user identification.

Background technique

In recent years, Bullet Subtitle video sharing websites (such as Station A, Station B, Station C, etc.) have developed rapidly, attracting the attention of a large number of contemporary young customers. The bullet screen video website relies on the advantages of high interaction and high degree of freedom of the bullet screen to attract users and increase the playback volume and spread of the website. However, the management and monitoring of the bullet screen content on the network platform is not reasonable enough, resulting in vulgar, violent and negative energy comments in different videos, which seriously affects the viewing experience of the customer group. At the same time, the audience of the bullet screen is mainly young people. The non-standard language, violence and vulgarity of the bullet screen will have a negative impact on the language literacy, value orientation and mental formation of young people.

After the bullet screen text is preprocessed, there are few remaining feature words, which are typical ultra-short texts. The sparsity of bullet screen text features makes traditional text classification methods unsatisfactory for bullet screen classification. In order to solve the problem of sparse short text features, researchers at home and abroad use feature expansion, mainly by means of external corpus expansion and text expansion by means of short text's own content features. External corpus expansion and short text own content feature expansion have their own advantages and disadvantages. External corpus expansion mainly depends on the quality of the corpus, which is computationally intensive and text-dependent. The extension of the content features of the short text itself is mainly to mine the semantic features of the short text itself, which is prone to overfitting. Therefore, a new text classification method can be proposed by combining the advantages of these two methods.

SUMMARY OF THE INVENTION

The present invention provides a method and device for filtering bullet screens based on content and user identification, combining external corpus expansion and content feature expansion of short texts, and introducing word vectors into feature expansion, and proposes a new text classification method to solve the bullet screen problem. The problem of sparse text features in video website bullet chat.

The present invention is achieved through the following technical solutions:

Since the traditional text classification method is very unsatisfactory for the classification effect of the bullet screen, the expansion of the text with the help of the external corpus expansion alone or with the help of the content features of the short text itself has its own shortcomings. Therefore, the present invention combines the expansion of the external corpus and the content feature expansion of the short text itself. The advantages of the two methods, while introducing the word vector into the feature extension, proposes a content and user identification-based bullet screen content classification method, so as to complete the bullet screen filtering, including steps S1-S4:

S1. Use python crawler software to crawl the bullet screen data and user data of the bullet screen video website, clean the crawled data and mark the bullet screen data into ordinary bullet screen and bad bullet screen. Among them, the bullet screen The data includes the short text of the bullet screen, that is, the text content of the bullet screen, and the user data includes the user's gender, the number of fans, the number of followers, and the user level;

S2, extending the short text of the bullet screen marked in step S1, and optimizing the feature representation of the text on the short text of the bullet screen after the expansion, so as to obtain the extended text feature;

S3. Constructing user platform class features, analyzing user data, and constructing new user credit rating features and user identity credibility features based on the original features of user data;

S4. Divide the bullet screen data marked in step S1 into a training set and a test set, use the five-fold cross-validation to train the SVM model, construct a bullet screen content classification model, and combine the extended text features obtained in step S2 with those obtained in step S3. The user platform class features are spliced and input into the bullet screen content classification model, the bullet screen classification result is output, and finally the bullet screen is filtered according to the bullet screen classification result.

Further, the specific process of marking the bullet screen data in step S1 is to mark the bullet screen data according to the content of the bullet screen. The bullet screen content of emoji is marked with 1, and the other bullet screen content is marked with 0, where 0 represents normal bullet screen and 1 represents bad bullet screen.

Further, step S2 specifically includes steps S21-S24:

S21. Pre-train the Word2Vec model according to the external corpus;

S22, construct the optimal feature space and the label topic feature space;

S23. According to the constructed optimal feature space and tag topic feature space, based on the pre-trained Word2Vec model, expand the qualified vocabulary in the short text of the bullet screen to obtain the extended short text;

S24 , improving the text representation method, and introducing an extended influence factor into the extended short text to indicate the degree of influence of the extended word on the short text of the bullet screen, so as to maximize the semantic expansion of the original bullet screen text and obtain extended text features.

Further, the external corpus for training the Word2Vec model comes from the comment data below the video of the bullet screen video website and the bullet screen data in the video. The content of the external corpus has domain similarity with the bullet screen data to be classified, which can ensure that the word coverage rate is high. It is used in the commonly used Wikipedia word vectors and Sogou news word vectors.

Further, constructing the optimal feature space and the label topic feature space specifically includes steps S221-S223:

S221, using the chi-square test method to extract the feature words with category tendency in the short text of the bullet screen, and construct the optimal feature space;

S222, adopt the aggregation strategy to merge all the short texts of the bullet screen under each label into long texts, and then input the long texts under each label into the LDA topic model for training;

S223 , using the LDA topic model to obtain a text-topic probability matrix, obtaining the probability of each tag under each topic, and selecting the top n topics with a larger probability under each tag to construct a tag topic feature space.

Further, expanding the qualified vocabulary specifically includes the following steps:

S231. According to the constructed tag topic feature space, use the LDA topic model to obtain a topic-topic word probability matrix, and select the top n topic words with high probability under each topic to form a topic word file;

S232, traverse the vocabulary in the short text of the bullet screen, if the vocabulary belongs to the optimal feature space, calculate the maximum probability topic to which the feature word belongs based on the topic-topic word distribution matrix;

S233. Check whether the word belongs to the theme word of the theme according to the theme word file. If not, it proves that the feature word does not have strong theme information. If the feature word is expanded, it is easy to introduce irrelevant feature words. the vocabulary is expanded;

S234. If it belongs, then check whether the maximum probability topic belongs to the tag topic feature space. If it belongs, use the Word2Vec model to add the top k words with high similarity as expansion words into the short text of the bullet screen; if not, it is proved that the For maximum probability topics that do not have label identification, the vocabulary is not expanded.

Further, the text representation method is improved by using the combined effect of word embedding, word similarity, word and topic probability, and label topic probability. The specific process includes using the direct addition of bullet screen short text and extended short text vector. method to construct a short text vector:

C(wi _,j )=sim( _wi ,wi _,j )*P(wi _, topic _m )*P(topic _m ,class)*D(wi _,j )#

Among them, C(d) represents the short text representation of the short text d based on the word vector synthesis, wi is the _ith word existing in the short text d, C( _wi ) is the word vector representation of the word wi, _{wi , j} is the j-th extended word of the i-th word, C(wi _,j ) is the final weighted vector representation of the extended word vector, D(wi _,j ) is the j-th extended word of the i-th word word vector representation. sim( _wi ,wi _,j ) represents the semantic similarity between the jth word extended by _wi and _wi , P(wi _, topic _m ) represents the probability of _wi in the largest topic topic _m to which it belongs, P(topic _m , class) represents the probability of generating the largest topic topic _m in the class label.

Further, the calculation steps of the user reputation level are:

The user's credit rating I _{credit-rating} is calculated according to the historical bullet chat behavior published by the user. The formula is as follows:

Among them, N _total represents the total number of published bullet screens, and N _bad represents the number of bad bullet screens published, and the user reputation level is periodically updated;

The calculation steps of the user identity credibility are:

According to the user's platform level and whether the user is VIP or not, the user's identity credibility I _{identity-credibility} is obtained. The formula is as follows:

Among them, I _level represents the normalized value of the user's platform level, and I _vip represents whether the user is a VIP.

In addition, the present invention proposes a content and user identification-based bullet screen filtering device, which supports the above-mentioned content and user identification-based bullet screen filtering method, including a data preprocessing module, a text extension module, and a user platform-like feature structure modules and classification modules, where,

Data preprocessing module: used to clean the missing data of the bullet screen video website and user data crawled by python crawler software, and mark the bullet screen data into ordinary bullet screen and bad bullet screen. The screen data includes the short text of the bullet screen, that is, the text content of the bullet screen;

Text extension module: It is used to construct the optimal feature space and label theme feature space to expand the marked bullet screen short text, improve the text representation method, and optimize the text feature representation of the expanded bullet screen short text to obtain the extended text. feature;

User platform class feature construction module: used to construct user platform class features, analyze user data, and construct user reputation level features and user identity credibility features based on the original features of user data;

Classification module: It is used to divide the marked bullet screen data set into training set and test set, use five-fold cross-validation to train the SVM model to construct a classification model, and splicing the expanded text features and user platform features into the In the classification model, output the bullet chat classification result.

A computer-readable storage medium on which a computer program is stored, the computer program implements the above-mentioned content- and user-identity-based bullet screen filtering method when running.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

The present invention is a method and device for filtering barrage based on content and user identification, which combines the advantages of external corpus expansion and content feature expansion of short text itself, and introduces word vector into feature expansion, and proposes an improved text expansion method. This method assigns different weights to the expanded words with different granular topics and different semantic complementarities, so as to maximize the semantic expansion of the original text. At the same time, a text representation method with word similarity, word and topic probability, tag topic probability, and word embedding is proposed. The improved text representation can learn richer semantic information and optimize the feature representation of text. In the user platform feature construction module, two new features, user reputation level and user identity credibility, are constructed. The bullet screen text is short and the amount of information is small. The recognition dimension based on the text content is single. Adding user features to the bullet screen feature space can further increase the information dimension, enrich the bullet screen feature space, improve the bullet screen recognition rate, and optimize the bullet screen. Classification algorithm.

Description of drawings

The accompanying drawings described herein are used to provide further understanding of the embodiments of the present invention, and constitute a part of the present application, and do not constitute limitations to the embodiments of the present invention. In the attached image:

Fig. 1 is the schematic flow chart of the method of the present invention;

Fig. 2 is a schematic diagram of a text expansion method;

Fig. 3 is the flow chart of constructing the optimal feature space and tag topic feature space;

FIG. 4 is a text expansion method according to an embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and the accompanying drawings. as a limitation of the present invention.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one of ordinary skill in the art that these specific details need not be employed to practice the present invention. In other instances, well-known structures, circuits, materials, or methods have not been described in detail in order to avoid obscuring the present invention.

Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in the present invention in at least one embodiment. Thus, appearances of the phrases "one embodiment," "an embodiment," "one example," or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures or characteristics may be combined in any suitable combination and/or subcombination in one or more embodiments or examples. Furthermore, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and that the drawings are not necessarily drawn to scale. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

In the description of the present invention, it should be understood that the terms "front", "rear", "left", "right", "upper", "lower", "vertical", "horizontal", "high", The orientation or positional relationship indicated by "low", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated device or Elements must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as limiting the scope of the invention.

Example 1

Since the traditional text classification method is very unsatisfactory for the classification effect of the bullet screen, the expansion of the text with the help of the external corpus expansion alone or the content feature of the short text itself has its own shortcomings. Therefore, the present invention combines the expansion of the external corpus and the content feature expansion of the short text itself. The advantages of the two methods, at the same time, the word vector is introduced into the feature extension, and a bullet screen filtering method based on content and user identification is proposed. The bullet screen video websites mentioned below include but are not limited to stations A, B, and C, etc. For the bullet screen video sharing website, as shown in Figure 1, the overall process of the bullet screen filtering method based on content and user identification includes steps S1-S4:

S1. Use python crawler software to crawl the bullet screen data and user data of the bullet screen video website, clean the crawled data and mark the bullet screen data into ordinary bullet screen and bad bullet screen. Among them, the bullet screen The data includes the short text of the bullet screen, that is, the text content of the bullet screen, and the user data includes the user's gender, the number of fans, the number of followers, and the user level;

S2, extending the short text of the bullet screen marked in step S1, and optimizing the feature representation of the text on the short text of the bullet screen after the expansion, so as to obtain the extended text feature;

S3. Constructing user platform class features, analyzing user data, and constructing new user credit rating features and user identity credibility features based on the original features of user data;

S4. Divide the bullet screen data marked in step S1 into a training set and a test set, use the five-fold cross-validation to train the SVM model, construct a bullet screen content classification model, and combine the extended text features obtained in step S2 with those obtained in step S3. The user platform class features are spliced and input into the bullet screen content classification model, the bullet screen classification result is output, and finally the bullet screen is filtered according to the bullet screen classification result.

Specifically, the specific process of marking the bullet screen data in step S1 is to mark the bullet screen data according to the content of the bullet screen. The bullet screen content of emoji is marked with 1, and the other bullet screen content is marked with 0, where 0 represents normal bullet screen and 1 represents bad bullet screen.

Specifically, in order to maximize the semantic expansion of the original bullet screen short text, the features of the bullet screen short text are extended, as shown in Figure 2, including steps S21-S24:

S21. Pre-train the Word2Vec model according to the external corpus;

S22, construct the optimal feature space and the label topic feature space;

S23. According to the constructed optimal feature space and tag topic feature space, based on the pre-trained Word2Vec model, expand the qualified vocabulary in the short text of the bullet screen to obtain the extended short text;

S24 , improving the text representation method, and introducing an extended influence factor into the extended short text to indicate the degree of influence of the extended word on the short text of the bullet screen, so as to maximize the semantic expansion of the original bullet screen text and obtain extended text features.

The above external corpus for training the Word2Vec model comes from the comment data below the video of the bullet screen video website and the bullet screen data in the video. Jieba is used to segment the corpus and remove stop words to ensure that the corpus scale is as large as possible. The optimized Word2Vec model builds a word vector dictionary, and the content of the external corpus has domain similarity with the bullet screen data to be classified, which can ensure that the word coverage is higher than the currently commonly used Wikipedia word vectors and Sogou news word vectors.

Specifically, constructing an optimal feature space and a tag topic feature space gives different weights to extended words with different granularity topics and different semantic complementarities, as shown in Figure 3, which specifically includes steps S221-S223:

S221, using the chi-square test method to extract the feature words with category tendency in the short text of the bullet screen, and construct the optimal feature space;

S222, adopt the aggregation strategy to merge all the short texts of the bullet screen under each label into long texts, and then input the long texts under each label into the LDA topic model for training;

S223 , using the LDA topic model to obtain a text-topic probability matrix, obtaining the probability of each tag under each topic, and selecting the top n topics with a larger probability under each tag to construct a tag topic feature space.

In actual operation, it is not necessary to expand all the vocabulary for the obtained bullet screen data. According to the above-constructed optimal feature space and tag topic feature space, the qualified vocabulary is expanded to improve the bullet screen classification efficiency, as shown in Figure 4. It includes the following steps:

S231. According to the constructed tag topic feature space, use the LDA topic model to obtain a topic-topic word probability matrix, and select the top n topic words with high probability under each topic to form a topic word file;

S232, traverse the vocabulary in the short text of the bullet screen, if the vocabulary belongs to the optimal feature space, calculate the maximum probability topic to which the feature word belongs based on the topic-topic word distribution matrix;

S233. Check whether the word belongs to the theme word of the theme according to the theme word file. If not, it proves that the feature word does not have strong theme information. If the feature word is expanded, it is easy to introduce irrelevant feature words. the vocabulary is expanded;

S234. If it belongs, then check whether the maximum probability topic belongs to the tag topic feature space. If it belongs, use the Word2Vec model to add the top k words with high similarity as expansion words into the short text of the bullet screen; if not, it is proved that the For maximum probability topics that do not have label identification, the vocabulary is not expanded.

Specifically, the text representation method is improved by using the combined effect of word embedding, word similarity, word and topic probability, and label topic probability. The improved text representation can learn richer levels of semantic information and optimize the text. The specific process includes using the method of directly adding the bullet screen short text and the extended short text vector to construct the short text vector:

C(wi _,j )=sim( _wi ,wi _,j )*P(wi _, topic _m )*P(topic _m ,class)*D(wi _,j )#

Among them, C(d) represents the short text representation of the short text d based on the word vector synthesis, wi is the _ith word existing in the short text d, C( _wi ) is the word vector representation of the word wi, _{wi , j} is the j-th extended word of the i-th word, C(wi _,j ) is the final weighted vector representation of the extended word vector, D(wi _,j ) is the j-th extended word of the i-th word Word vector representation, sim( _wi ,wi _,j ) represents the semantic similarity between the jth word extended by _wi and _wi , P(wi _, topic _m ) represents the largest topic topic _m to which _wi belongs The probability in , P(topic _m , class) represents the probability of generating the largest topic topic _m in the class label.

Because the bullet screen text is short and the amount of information is small, the recognition dimension based only on the text content is single, adding user features to the bullet screen feature space can further increase the information dimension, enrich the bullet screen feature space, improve the bullet screen recognition rate, and improve the recognition rate of the bullet screen. After analysis, two new features of user reputation level and user identity credibility are constructed on the basis of the original features of user data; the calculation steps of the user reputation level are as follows:

The user's credit rating I _{credit-rating} is calculated according to the historical bullet chat behavior published by the user. The formula is as follows:

Among them, N _total represents the total number of published bullet screens, and N _bad represents the number of bad bullet screens published, and the user reputation level is periodically updated;

The calculation steps of the user identity credibility are:

According to the user's platform level and whether the user is VIP or not, the user's identity credibility I _{identity-credibility} is obtained. The formula is as follows:

Among them, I _level represents the normalized value of the user's platform level, and I _vip represents whether the user is a VIP.

Example 2

A specific embodiment of the present invention also provides a barrage filtering device based on content and user identification, including a data preprocessing module, a text expansion module, a user platform class feature construction module and a classification module, wherein,

Data preprocessing module: It is used to clean the missing data of the bullet screen video website and user data crawled by python crawler software, and mark the bullet screen data into ordinary bullet screen and bad bullet screen;

Text extension module: It is used to construct the optimal feature space and label theme feature space to expand the marked bullet screen short text, improve the text representation method, and optimize the text feature representation of the expanded bullet screen short text to obtain the extended text. feature;

User platform class feature construction module: used to construct user platform class features, analyze user data, and construct user reputation level features and user identity credibility features based on the original features of user data;

Classification module: It is used to divide the marked bullet screen data set into training set and test set, use five-fold cross-validation to train the SVM model to construct a classification model, and splicing the expanded text features and user platform features into the In the classification model, the bullet screen classification results are output, and the bullet screens are filtered according to the classification results.

The device supports the content- and user-identity-based barrage filtering method described in Embodiment 1, and details are not described here.

Example 3

A specific embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program implements the content- and user-identity-based bullet screen filtering method described in Embodiment 1 when running.

Professionals may further realize that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, in order to clearly illustrate the possibilities of hardware and software. Interchangeability, the above description has generally described the components and steps of each example in terms of functionality. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

The steps of a method or algorithm described in conjunction with the embodiments disclosed herein may be directly implemented in hardware, a software module executed by a processor, or a combination of the two. A software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

It can be understood that, through the method provided by the present invention, an improved text expansion method is proposed by combining the advantages of external corpus expansion and content feature expansion of short text itself, and introducing word vectors into feature expansion. This method assigns different weights to the expanded words with different granular topics and different semantic complementarities, so as to maximize the semantic expansion of the original text. At the same time, a text representation method with word similarity, word and topic probability, tag topic probability, and word embedding is proposed. The improved text representation can learn richer semantic information and optimize the feature representation of text. In the user platform feature construction module, two new features, user reputation level and user identity credibility, are constructed. The bullet screen text is short and the amount of information is small. The recognition dimension based on the text content is single. Adding user features to the bullet screen feature space can further increase the information dimension, enrich the bullet screen feature space, improve the bullet screen recognition rate, and optimize the bullet screen. Classification algorithm.

The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A bullet screen filtering method based on content and user identification is characterized by comprising the following steps:

s1, crawling bullet screen data and user data of a bullet screen video website by using python crawler software, cleaning the crawled data, marking the bullet screen data into a common bullet screen and a bad bullet screen, wherein the bullet screen data comprise short bullet screen texts;

s2, expanding the barrage short text marked in the step S1, and optimizing the feature representation of the text of the expanded barrage short text to obtain the feature of the expanded text;

s3, constructing user platform class characteristics, analyzing user data, and newly constructing user reputation grade characteristics and user identity credibility characteristics on the basis of original characteristics of the user data;

and S4, dividing the marked bullet screen data in the step S1 into a training set and a testing set, constructing a bullet screen content classification model by utilizing a five-fold cross validation training SVM model, splicing the extended text features obtained in the step S2 and the user platform features obtained in the step S3, inputting the spliced extended text features and the user platform features into the bullet screen content classification model, and outputting bullet screen classification results.

2. The bullet screen filtering method based on the content and the user identifier as claimed in claim 1, wherein the specific process of marking the bullet screen data in step S1 is marking the bullet screen data according to the bullet screen content, marking the bullet screen content with the unintelligent wording, meaningless single characters and all emoticons as 1, and marking the other bullet screen content as 0, where 0 represents a normal bullet screen and 1 represents a bad bullet screen.

3. The bullet screen filtering method based on the content and the user identifier as claimed in claim 1, wherein the step S2 specifically includes the following steps:

s21, pre-training a Word2Vec model according to an external corpus;

s22, constructing an optimal feature space and a label theme feature space;

s23, expanding words meeting conditions in the bullet screen short text based on a pre-trained Word2Vec model according to the constructed optimal feature space and the label theme feature space to obtain an expanded short text;

and S24, improving a text representation method, and introducing an expansion influence factor into the expanded short text to represent the influence degree of the expansion words on the bullet screen short text to obtain the characteristics of the expanded text.

4. The bullet screen filtering method based on content and user identification as claimed in claim 3, wherein the external corpus includes comment data below the bullet screen video website video and bullet screen data in the video.

5. The bullet screen filtering method based on the content and the user identification as claimed in claim 3, wherein the constructing of the optimal feature space and the tag topic feature space specifically comprises the following steps:

s221, extracting feature words with category tendentiousness in the bullet screen short text by using a chi-square test method, and constructing an optimal feature space;

s222, combining all the bullet screen short texts under each label into a long text by adopting a polymerization strategy, and inputting the long text under each label into an LDA topic model for training;

s223, obtaining a text-theme probability matrix by using the LDA theme model, obtaining the probability of each label under each theme, and selecting the first n themes with high probability under each label to construct a label theme feature space.

6. The bullet screen filtering method based on the content and the user identification as claimed in claim 3, wherein the step of expanding the qualified vocabulary specifically comprises the steps of:

s231, forming a subject word file according to the subjects in the constructed tag subject feature space;

s232, traversing the words in the bullet screen short text, and calculating the maximum probability theme of the words based on a theme-subject word distribution matrix if the words belong to the optimal feature space;

s233, checking whether the vocabulary belongs to the theme words of the corresponding theme according to the theme word file, if not, not expanding the vocabulary;

s234, if the Word belongs to the label topic feature space, checking whether the maximum probability topic of the Word belongs to the label topic feature space, and if the Word belongs to the label topic feature space, adding the first k words with high similarity as extension words into the bullet screen short text by using a Word2Vec model; if not, the vocabulary is not expanded.

7. The bullet screen filtering method based on the content and the user identification as claimed in claim 3, wherein the text representation method is improved by using the combined action of word embedding, word similarity, word and topic probability and tag topic probability, and the specific process comprises the following steps of constructing a short text vector by using a method of directly adding bullet short text and an extended short text vector:

C(w_i,j)＝sim(w_i,w_i,j)*P(w_i,topic_m)*P(topic_m,class)*D(w_i,j)#

wherein C (d) represents short text d short text representation based on word vector synthesis, w_iFor the i-th word, C (w), present in short text d_i) Is a word w_iWord vector representation of, w_i,jThe j-th expansion word of the ith word, C (w)_i,j) For expanding the final weighted vector representation of the vocabulary vector, D (w)_i,j) Word vector representation of the jth expanded word for the ith word, sim (w)_i,w_i,j) Is represented by w_iExpanded jth word and w_iSemantic similarity of (c), P (w)_i,topic_m) Denotes w_iIn the belonged maximum topic_mProbability of (1), P (topic)_mClass) represents the largest topic yielding topic in a class label_mThe probability of (c).

8. The bullet screen filtering method based on the content and the user identification according to claim 1, wherein the user reputation level is calculated by the steps of:

calculating to obtain a user reputation grade I according to historical bullet screen behaviors issued by a user_{credit-rating}The formula is as follows:

wherein N is_totalIndicates total number of delivered barrages, N_badBad bullet screen for showing publicationUpdating the user credit rating periodically;

the user identity credibility calculation steps are as follows:

obtaining user identity credibility I according to user platform grade and whether the user is VIP_{identity-credibility}The formula is as follows:

wherein I_levelValue normalized by the platform level representing the user, I_vipIndicating whether the user is VIP.

9. A bullet screen filtering device based on content and user identification, which supports the bullet screen filtering method based on content and user identification according to any one of claims 1-8, and comprises a data preprocessing module, a text extension module, a user platform class feature construction module and a classification module, wherein,

a data preprocessing module: the system is used for cleaning missing data of bullet screen data and user data of a bullet screen video website crawled by python crawler software, and marking the bullet screen data into a common bullet screen and a bad bullet screen;

a text extension module: the method is used for constructing an optimal feature space and a label subject feature space to expand the marked bullet screen short text, and the improved text representation method is used for carrying out text feature representation optimization on the expanded bullet screen short text to obtain the expanded text features;

the user platform class feature construction module: the system is used for constructing user platform class characteristics, analyzing user data and newly constructing user reputation grade characteristics and user identity credibility characteristics on the basis of user data original characteristics;

a classification module: the method is used for dividing the marked bullet screen data set into a training set and a testing set, constructing a classification model by utilizing a five-fold cross validation training SVM model, splicing and inputting the expanded text characteristics and the user platform type characteristics into the classification model, and outputting bullet screen classification results.

10. A computer-readable storage medium, on which a computer program is stored which, when executed, implements the method of any one of claims 1-8.

Images