CN112000779A

CN112000779A - Automatic review and labeling system

Info

Publication number: CN112000779A
Application number: CN202011178471.3A
Authority: CN
Inventors: 王鹏翔
Original assignee: Beijing Zhidemai Technology Co ltd
Current assignee: Beijing Zhidemai Technology Co ltd
Priority date: 2020-10-29
Filing date: 2020-10-29
Publication date: 2020-11-27

Abstract

The invention discloses an automatic review and labeling system, which comprises: learning by utilizing comment data which are labeled by operators daily, and training a neural network model, a comment model and a primary label prediction model; auditing is carried out on a piece of comment data, and the auditing comprises the step of auditing the number of effective words of the comment data, wherein the number of the effective words is smaller than the preset number of the effective words and is judged to be not passed; whether illegal words exist in the comment data is checked, and if the illegal words exist, the comment data is judged to be not passed; if the neural network model passes, judging by using the neural network model; if the comment data pass through the search engine, the keywords of the comment data are mined and matched with the primary labels, and if the keywords are the same, the keywords are directly bound to the corresponding primary labels; when the first-level label is not hit, performing first-level label hit prediction by using the neural network model, and if the first-level label is predicted, binding the corresponding first-level label; the review cost of the e-commerce platform on the comments can be greatly improved.

Description

Automatic review and labeling system

Technical Field

The invention relates to the technical field of deep learning, in particular to natural language processing, and discloses an automatic review and labeling system.

Background

In recent years, with the development of electronic commerce, various large e-commerce platforms develop well, online shopping by users becomes a habit in life, after the online shopping behavior is finished, the users leave comments on commodities on e-commerce websites and platforms, the comments are also behaviors which do not need to be considered, the comments include the opinions of the users on the commodities, the use feelings and the personal use experiences and feelings, which are valuable wealth for the e-commerce platforms, effective comments which can be checked out are screened out, the feelings of the users on different dimensions of the commodities are found from the effective comments of the users, and then personalized display is carried out, the overall image of the commodities is enriched, guidance on the purchasing behaviors of subsequent users is provided, and the users can objectively know the commodities.

The method and the device for extracting the tags of the network comments, disclosed by CN105824898A, label comment objects and emotion types for the comment phrases. And then counting according to the comment objects, counting the number of comment short sentences of which the emotion types are positive emotions in the same comment object, and extracting by taking the counting result as a label. Compared with the method for extracting the label only by semantically removing the duplicate of the comment short sentence, the label contains the object to be commented by the comment short sentence and the information of the number of positive comments and negative comments of the object to be commented, so that the information of a certain aspect of the commodity can be displayed in a more concise label form, and the shopping experience of a user is improved.

CN107633007A provides a commodity comment data labeling system based on hierarchical AP clustering, which comprises a data capturing module, a word vector training module, a characteristic information extraction module and a characteristic information labeling module; the data capturing module stores the material information and the comment data; the word vector training module obtains a training corpus set; the characteristic information extraction module obtains a characteristic information set corresponding to the comment data; and the characteristic information labeling module obtains a labeling result of the clustered comment data. The invention has the beneficial effects that: the commodity comment data tagging system and method based on hierarchical AP clustering achieve the purpose of automatically completing comment data tagging, can mine the value orientation of the characteristic information, show the characteristic information to merchants and customers in a tag form, provide support for subsequent data analysis, and provide a tool for enterprises and consumers to conveniently, scientifically and intuitively obtain useful comment information.

CN108984523A relates to the field of natural language processing, in particular to a commodity comment sentiment analysis method based on a deep learning model, which comprises the steps of capturing commodity comment data, marking one-star and two-star evaluations in the commodity comment data as positive comments, marking four-star and five-star evaluations in the commodity comment data as negative comments, dividing the commodity comment data into a training set and a test set, and preprocessing the training set and the test set; constructing an emotion element dictionary set and an emotion feature vector, obtaining word vectors according to word sequences and emotion feature vectors obtained by preprocessing a training set, and connecting a plurality of word vectors to form text vectors; constructing a dynamic convolution neural network model, updating network parameters of the dynamic convolution neural network by taking a text vector as a training object through a BP algorithm and a random gradient descent algorithm, finally obtaining an emotion classification model and carrying out emotion marking on a test set; the method can improve the generalization capability of the classification model by combining with the dynamic convolution neural network, and can realize better classification effect.

At present, the e-commerce platform for mining comment contents depends on manual review of small editors, whether comments can pass or not, whether blacklist keywords are hit or not needs to be reviewed or not, whether the comments belong to marketing comments or not and the like. For the operation of the associated labels of the comments, characteristic labels associated with the comments need to be set in advance, the comments are understood and audited manually, and hit labels are selected for manual audit and binding, for example, the comments are considered to design labels such as 'commodity appearance', 'use experience', and the like. In addition, common corresponding keywords are collected under the tags, and if the comment content hits the keywords, the corresponding tags are bound. The time and labor cost of the methods are high, and the number of newly increased comments in one day is large for the prosperity of the current electric business, so that the methods are slightly labored. In addition, simply binding tags by keywords may miss many comments, because sometimes the comment may not hit keywords, but carefully understanding the meaning of the words in the comment may be consistent with the meaning of some tags.

Disclosure of Invention

The embodiment of the invention provides an automatic review and labeling system. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

According to an embodiment of the present invention, there is provided an automatic review and labeling system, including:

s1: learning by using data labeled by operators daily, and training a neural network model, wherein the neural network model comprises a marketing type comment or diversion type comment model and a label prediction model;

s2: auditing a comment, wherein the examination is specific to the comment, and comprises effective word number judgment and blacklist vocabulary scanning, and judging whether the comment can pass or not;

s3: if the judgment result is passed, judging whether the comment is a marketing type comment or a diversion type comment by using the neural network model;

s4: if the keyword passes through the first-level tag, the keyword is mined, the keywords which can be hit under the tag are found, and the keywords are directly bound to the corresponding tag;

s5: when the first-level label key words are not hit, label hit prediction is carried out by using the neural network model, and when the labels are predicted, corresponding labels are bound;

s6: and extracting the secondary label, and extracting the core subjective opinion of the user in the comment to serve as the secondary label, wherein the secondary label can show the comment content in a personalized manner.

Preferably, the training of the neural network model is performed off-line, and the normalized processing of the comment data is sent to the neural network model for training.

Preferably, the marketing type comment or diversion type comment model is 1 classifier, the specified target value of "1" is a marketing type or diversion type comment, and the target value of "0" is a non-marketing type or diversion type comment.

Preferably, the label prediction model is a two-classifier, n two-classifiers are established for n label variables, a target value of the classifier is specified to be "1", and corresponding labels are marked.

Preferably, the two classifiers are implemented by adopting a fasttext technology.

Preferably, in order to use the fasttext, the normalization processing of the comment data comprises data labeling, word segmentation, word filtering stopping, and a formatted data set required by the fasttext is constructed; the data marking means that whether each comment is manually marked or not is firstly carried out, and a label which is in accordance with the comment is bound to the comment; the word segmentation is to divide the Chinese text into a form of one character and one character; the stop word filtering removes the nonsense Chinese characters and all punctuation marks.

Preferably, the formatted data set required by the fasttext comprises a training set and a test set, the training set and the test set are txt files, and each line in the file is a piece of sample data.

Preferably, the tag hit prediction is performed, and the tag judged to be "1" by the model is associated with the comment.

Preferably, the extraction of the secondary labels adopts an unsupervised learning model, and through text analysis, the viewpoints of the commodities expressed by the user are extracted, and the extracted personalized labels are used as the results of the secondary labels.

Preferably, the unsupervised learning model is an ltp language analysis tool.

The technical scheme provided by the embodiment of the invention has the following beneficial effects:

the automatic review and automatic marking work of the user comments is carried out by utilizing the natural language processing technology, so that the review cost of the e-commerce platform on the comments can be greatly improved, and the review cost comprises time cost and labor cost. The operator of the e-commerce platform only needs to carry out data standard work in the early stage, after enough training set sample data are accumulated, the fast advantage of fasttext training can be utilized, a deep learning model can be rapidly obtained, the better generalization performance of the model can be utilized to replace manual review and labeling operation for comments, and subsequent manual work only needs to carry out sampling inspection work. In addition, the constructed secondary label system can display the original content of the user comment more abundantly, so that the user can feel more real and appropriate commodity evaluation and commodity description, further more attract greater attention of the user, and the referability and the external value of the comment are also improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a flow diagram illustrating an automatic review and tagging system in accordance with an exemplary embodiment;

FIG. 2 is a flowchart illustrating logic performed by an automatic review and tagging system in accordance with an exemplary embodiment.

Detailed Description

The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them. The examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in or substituted for those of others. The scope of embodiments of the invention encompasses the full ambit of the claims, as well as all available equivalents of the claims. Embodiments may be referred to herein, individually or collectively, by the term "invention" merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed. The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the structures, products and the like disclosed by the embodiments, the description is relatively simple because the structures, the products and the like correspond to the parts disclosed by the embodiments, and the relevant parts can be just described by referring to the method part.

It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

The invention is further described with reference to the following figures and examples:

an automatic review and tagging system as shown in fig. 1 includes,

s1: utilizing daily marked comment data of operators to learn and train a neural network model, wherein the neural network model comprises a marketing type comment model or a flow guiding type comment model and a primary label prediction model;

s2: auditing is carried out on a piece of comment data, and the auditing comprises the step of auditing the number of effective words of the comment data, wherein the number of the effective words is smaller than the preset number of the effective words and is judged to be not passed; whether illegal words exist in the comment data is checked, and if the illegal words exist, the comment data is judged to be not passed;

s4: if the comment data pass through the search engine, the keywords of the comment data are mined and matched with the primary labels, and if the keywords are the same, the keywords are directly bound to the corresponding primary labels;

s5: when the first-level label is not hit, performing first-level label hit prediction by using the neural network model, and if the first-level label is predicted, binding the corresponding first-level label;

s6: and extracting the secondary label, and extracting the core subjective opinion of the user in the comment to serve as the secondary label, wherein the secondary label displays the comment content in an individualized manner.

According to the scheme, further, the training of the neural network model is performed off-line, and the comment data is firstly subjected to normalization processing and then sent to the neural network model for training.

According to the above scheme, further, the marketing comment model or the diversion comment model is 1 classifier, the specified target value "1" is marketing or diversion comment, and the target value "0" is non-marketing or non-diversion comment.

According to the scheme, the primary label prediction model is a two-classifier, n two-classifiers are established for n label variables, a target value of the two-classifier is specified to be 1, and corresponding primary labels are marked. Specifically, the first-level labels are associated with the content of the comment, the specific first-level labels to be marked are sorted in advance, the first-level labels have generality and universality, and the large aspects related to the comment content, such as 'use experience', 'commodity quality' and the like, can be summarized. The label prediction model is a two-classifier, n two-classifiers are established for n label variables, a target value specified by the classifier is '1', and corresponding labels are marked. For example, 10 labels are extracted, and the 10 labels are regarded as 10 independent first-level labels, and the purpose is to train 10 classifiers, input a piece of comment content into the 10 classifiers in sequence for 10 times of classification, judge that the classifier is "yes", and apply the corresponding label. And the classifier is realized by adopting a fasttext technology. fasttext is suitable for training large data set text classification tasks because of the very fast training speed.

According to the scheme, further, in order to use the fasttext, the normalization processing of the comment data comprises the steps of labeling, word segmentation and word filtering stopping on the comment data to construct a data set which is required to be formatted by the fasttext; the annotation means that whether each comment is manually annotated or not is first carried out, and keywords which accord with a first-level label in the comment data are bound to the comment; the word segmentation is a form of dividing a Chinese text into one word and one character; the stop word filtering is to remove those meaningless Chinese characters and all punctuation marks. Specifically, we should first arrange the data set into the required format, i.e. perform data preprocessing including data labeling, word segmentation, stopping word filtering, and constructing a formatted training set. The data marking means that whether each comment is manually marked or not is firstly carried out, and whether the label is in line with the comment or not is bound to the comment. For example: the label is good-looking, practical and exquisite, the comment is in line with the label, the label can be printed, and the comment which is bought by a vintage son does not need to know how much, so that the comment which is evaluated again is not hit. We then perform participle and stop word filtering operations on the comments. For the review of the Chinese text, the form of dividing into words is better, and if the English words are encountered, the form of the words is kept. Stopping word filtering removes those meaningless Chinese characters and all punctuation marks, such as "ones", etc. And finally constructing a formatted data set required by the fasttext, wherein the formatted data set comprises a training set and a testing set, the training set and the testing set are both txt files, and each line in the file is a piece of sample data. For example, we put a comment into the following final arrangement: "screen large-definition original machine good __ label __ 0" without error. Where 0 is not, 1 is. And then, training the models, namely training the models by using a plurality of classifiers, and storing the models after the training is finished.

According to the scheme, the data set required to be formatted by the fasttext further comprises a training set and a testing set, the training set and the testing set are txt files, and each behavior in the txt files is one-striped comment sample data.

According to the scheme, the second-level labels are extracted by adopting an unsupervised learning model, the viewpoints of the commodities expressed by the user are extracted through text analysis, and the extracted personalized labels are used as the results of the second-level labels. And extracting the secondary label by using an ltp language analysis tool, performing word segmentation, part of speech, labeling and dependency syntactic analysis operation on a piece of comment content by using the ltp respectively, and performing detailed splitting and analysis on the text of the comment. And (3) part-of-speech tagging, namely, tagging the part-of-speech of each word of the word segmentation result, wherein the dependency syntax explains the syntax structure by analyzing the dependency relationship before the components in the language unit, and the core verb in the sentence is claimed to be the central component dominating other components. But is not itself subject to any other constituent, all subject constituents being subject to a subject in some relationship. Through detailed analysis and investigation of the comment content, the user's view basically appears in the following structures: (centering structure) + predicate structure, subject is noun, predicate is adjective, there may be an adjective before the subject and the subject to form a centering structure, for example: long standby time, good hand feeling and good sound effect. 2. The dynamic structure, i.e. the form of verb + complement, for example: the operation is smooth, and the real shooting is real. By the aid of the rules of the two structures, the combination which accords with the rules can be extracted and summarized according to the word segmentation result, the part of speech tagging result and the dependency syntax analysis result of the ltp, and the user viewpoint of the comment can be obtained and used as a secondary label for displaying. For example, one comment is: the user feels bright in front when just holding the hand, and the user looks good. The sound is good. The effect of shooing is bar-type. The running speed is high, and the memory is large. In normal use, the battery is very durable, and a large 5000 milliamp battery is really large. The seller has good attitude, express as before, have very high cost performance and can not be requested. ", the secondary label result list extracted according to the ltp analysis result is: 'cost performance is high', 'battery is durable', 'sound is good', 'photographing effect is thick-bar', 'big battery', 'operation speed is fast', 'seller attitude is good' ].

According to the scheme, further, the primary tag hit prediction comprises the following steps: the label judged as 1 by the primary label prediction model is associated with the comment, the secondary label is extracted by adopting an unsupervised learning model, the core subjective opinions of the user in the comment are extracted through text analysis, and the extracted personalized core subjective opinions are used as the result of the secondary label.

As shown in fig. 2, the real-time portion has no operation interface, only an interface is provided externally, the comment text content can be transmitted as a parameter through the access interface in real time, and the obtained result includes whether the review is passed and the result of tagging the second-level tag after the review is passed. Aiming at a piece of comment content, firstly, automatic examination is carried out to pass or not, matching of blacklist words and limitation of effective word number are firstly carried out, if the requirements are met, then, marketing or flow-guiding type comments are judged by a marketing or flow-guiding type classifier, and whether the comment contains words obviously belonging to flow-guiding type or marketing type is judged firstly, for example: the terms such as 'free from rest', 'return for profit', '0 click on' and the like exist, and the classifier directly outputs '1', namely, the assessment is made as marketing and marketing type or diversion type assessment. If the words are not hit, the words are judged by a model in the marketing or flow guide type classifier, if the output of the classifier is 1, the words belong to flow guide type or marketing type comments, and if the output of the classifier is 0, the words do not belong to flow guide type or marketing type comments. The whole auditing process is passed, and the following associated tag operation is carried out. For the associated tag operation, first, a level of tag association is performed. The first-level labels have respective corresponding keywords, which keywords can be hit under the respective labels corresponding to the comments are found, the hit keywords are directly bound with the corresponding labels, deep learning model judgment corresponding to the keywords is continuously carried out on the labels with the keywords not hit, and the labels with the model judgment of '1' are associated with the comments.

It is to be understood that the present invention is not limited to the procedures and structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

1. An automatic review and labeling system, comprising,

2. An automatic review and labeling system as claimed in claim 1, wherein the training neural network model is performed off-line, and the normalization processing of the review data is performed first, and then the review data is sent to the neural network model for training.

3. An automatic review and labeling system as claimed in claim 2, wherein the marketing review model or the guided review model is 1 classifier, the specified target value "1" is marketing or guided review, and the target value "0" is non-marketing or non-guided review.

4. An automatic review and labeling system as claimed in claim 2, wherein the primary label prediction model is a two-classifier, n of the two-classifiers are established for n label variables, a target value of "1" is specified for the two-classifiers, and a corresponding primary label is labeled.

5. An automated review and tagging system according to claim 4 wherein the two classifiers are implemented using fasttext techniques.

6. The system for automatically reviewing comments and labeling according to claim 2, wherein in order to use fasttext, the normalization processing of the comment data includes labeling, word segmentation and stop word filtering of the comment data to construct a data set formatted by the fasttext; the annotation means that whether each comment is manually annotated or not is first carried out, and keywords which accord with a first-level label in the comment data are bound to the comment; the word segmentation is a form of dividing a Chinese text into one word and one character; the stop word filtering is to remove those meaningless Chinese characters and all punctuation marks.

7. The automated review and tagging system of claim 6, wherein the fasttext requirement formatted data set comprises a training set and a testing set, the training set and the testing set are a txt file, and each action in the txt file is a piece of formatted review sample data.

8. An automated review and tagging system according to claim 1, wherein the primary tag hit prediction: the label judged as "1" by the primary label prediction model is associated with the comment.

9. An automatic review and labeling system as claimed in claim 1, wherein the secondary label extraction employs an unsupervised learning model to extract the user's core subjective opinion of the review through text analysis, and the extracted personalized core subjective opinions are used as the secondary label result.

10. An automated review and tagging system according to claim 9 and wherein said unsupervised learning model is an ltp language analysis tool.