CN112115703B - Article evaluation method and device - Google Patents
Article evaluation method and device Download PDFInfo
- Publication number
- CN112115703B CN112115703B CN202010917172.0A CN202010917172A CN112115703B CN 112115703 B CN112115703 B CN 112115703B CN 202010917172 A CN202010917172 A CN 202010917172A CN 112115703 B CN112115703 B CN 112115703B
- Authority
- CN
- China
- Prior art keywords
- article
- quality
- sample
- low
- articles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Abstract
The embodiment of the application discloses an article evaluation method and device based on artificial intelligence, wherein the method comprises the steps of firstly acquiring an article to be evaluated, then calling a trained neural network, training the trained neural network through a sample article and a low-quality article corresponding to the sample article to obtain an article evaluation result of the article to be evaluated, and finally using the trained neural network to obtain the article evaluation result of the article to be evaluated; the neural network used by the method is obtained by training sample articles and low-quality articles, the low-quality articles in the related training data are obtained by processing the sample articles, the requirement on the sample data size is greatly reduced, meanwhile, the training data comprise the sample articles and the low-quality articles corresponding to the sample articles, the evaluation results of the low-quality articles are lower than those of the sample articles, the low-quality articles do not need to be evaluated manually, and the influence of subjective factors on the training results of the neural network is reduced.
Description
Technical Field
The application relates to the field of artificial intelligence, in particular to an article evaluation method and device based on artificial intelligence.
Background
With the development of artificial intelligence technology, the occupancy rate in the field of test paper evaluation is increasingly increased based on the article evaluation technology of artificial intelligence, such as the scoring of English compositions, and the like, so that the labor cost is greatly reduced.
The accuracy of the article evaluation is directly limited by the training effect of the neural network, but in order to ensure the training effect of the neural network, the prior art needs to provide a large amount of sample data (including sample articles and corresponding article evaluation), so that the training cost is relatively high, and the article evaluation can reflect the quality of the articles, but also has subjective factors (such as personal preference and the like) of the evaluators to a certain extent when the sample data are acquired, so that the neural network obtained by training based on the sample data can introduce the subjective factors of the evaluators, and can not objectively reflect the quality of the articles.
That is, the current article evaluation technology has at least the technical problem that a large number of sample articles with evaluation are needed or the quality of the articles cannot be objectively reflected.
Content of the application
The embodiment of the application provides an article evaluation and identification method and device, which are used for at least relieving the technical problem that a large number of sample articles with evaluation are required in the current article evaluation technology.
In order to solve the technical problems, the embodiment of the application provides the following technical scheme:
the embodiment of the application provides an article evaluation method, which comprises the following steps:
acquiring an article to be evaluated;
invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and sending the article evaluation result of the article to be evaluated to a terminal.
The embodiment of the application provides an article evaluation device, which comprises:
the acquisition module is used for acquiring articles to be evaluated;
the invoking module is used for invoking the trained neural network, wherein the trained neural network is obtained by training a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
the evaluation module is used for processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
And the sending module is used for sending the article evaluation result of the article to be evaluated to the terminal.
An embodiment of the present application provides a computer device, including a processor and a memory, where the memory stores a plurality of instructions, the instructions being adapted to be loaded by the processor to perform the steps of the method described above.
Embodiments of the present application provide a computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps of the above-described method.
Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium; the processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the above-described method.
The embodiment of the application provides a new article evaluation method and device based on artificial intelligence, which comprises the steps of firstly acquiring an article to be evaluated, then calling a trained neural network, wherein the trained neural network is obtained by training a sample article and a low-quality article obtained by the sample article, and finally processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated; the neural network used by the method is obtained by training the sample articles and the low-quality articles corresponding to the sample articles, the low-quality articles in the related training data are obtained by processing the sample articles, the requirement on the sample data size is greatly reduced, meanwhile, the training data comprise the sample articles and the low-quality articles corresponding to the sample articles, the evaluation results of the low-quality articles are lower than those of the sample articles, the low-quality articles do not need to be manually evaluated, the influence of subjective factors on the training results of the neural network is reduced, namely the technical problem that a large number of sample articles with evaluation results or the article quality cannot be objectively reflected in the current article evaluation technology is relieved, the training speed is accelerated, and meanwhile, the article evaluation can also objectively reflect the quality of the articles to be evaluated.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a networking schematic diagram of an evaluation system according to an embodiment of the present application.
Fig. 2 is a schematic flow chart of a first method for evaluating an article according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a second flow of the article evaluation method according to the embodiment of the present application.
Fig. 4 is a third flowchart of an article evaluation method according to an embodiment of the present application.
Fig. 5 is a schematic structural diagram of an article evaluation device according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Fig. 7a to 7f are schematic diagrams of models according to embodiments of the present application.
Fig. 8a to 8b are schematic views of a user interface according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
The article evaluation method related to the embodiment of the application relates to the field of artificial intelligence, and can be specifically realized through an artificial intelligence cloud service in the field of cloud technology.
Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. In the application, the artificial intelligence technology is mainly used for realizing article evaluation.
Machine Learning (ML) is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like. In the application, the machine learning is mainly used for the parameter training of the neural network corresponding to the feature extraction module and the feature evaluation module.
Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.
The artificial intelligence cloud Service is also commonly called AIaaS (AI as a Service, chinese is "AI as Service"). The service mode of the artificial intelligent platform is the mainstream at present, and particularly, the AIaaS platform can split several common AI services and provide independent or packaged services at the cloud. This service mode is similar to an AI theme mall: all developers can access one or more artificial intelligence services provided by the use platform through an API interface, and partial deep developers can also use an AI framework and AI infrastructure provided by the platform to deploy and operate and maintain self-proprietary cloud artificial intelligence services.
In the embodiment of the present application, in order to train the neural network, a small number of sample articles are required to be collected, and the sample articles are articles which have been given by the user (the evaluation results may include a result of scoring, good or bad, for example, may be a numerical score or a letter grade score, and hereinafter, the numerical score is taken as an example), and are uniformly distributed; for example, the sample articles can be uniformly provided with different numbers of compositions from 50 equal numbers of different subjects, the compositions under the same subject can be uniformly distributed in different fractional segments, for example, the compositions are fully divided into 15 points, the actual scores of the compositions can be uniformly distributed between 1 and 15 points, for example, 500 compositions are sampled under a subject, the actual scores of the 500 compositions can be uniformly distributed between 1 and 15, and each fractional segment (3 points progressive) comprises 100 different compositions; based on this, the model training of the application only needs 2500 sample articles. Meanwhile, in order to prove the beneficial effects of the article evaluation method provided by the application, a large number of non-grading articles are collected while sample articles are collected, and the non-grading articles are used for proving the beneficial effects of the neural network.
In the embodiment of the application, the articles to be evaluated refer to articles which are uploaded by users such as teachers, students and the like and need to be evaluated, and can be articles of various languages such as Chinese, english and the like; the sample article refers to the article with the evaluation collected in the previous step, the low-quality article corresponding to the sample article refers to the simple replacement of the content of any article with other questions by deleting sentences/words randomly or regularly and adjusting the sequence of the sample article, and the operations can lead to the poorer content quality of the sample article.
In the embodiment of the application, a certain model or a certain module represents the same object, and the module is realized by an algorithm corresponding to the model.
Referring to fig. 1, fig. 1 is a schematic view of a scenario of an evaluation system provided by an embodiment of the present application, where the system may include a user side device and a service side device, and the user side device and the service side device are connected by means of an internet formed by various gateways, and are not described in detail, where the user side device includes a plurality of terminals 11, and the service side device includes a plurality of servers 12; wherein:
The terminal 11 includes, but is not limited to, mobile terminals such as mobile phones, tablet and the like, and fixed terminals such as computers, inquiry machines, advertisement machines and the like, and is a service port that can be used and operated by a user; for convenience of the following description, the terminal 11 is defined as a platform terminal 11a and a user terminal 11b, wherein the platform terminal 11a is used for uploading sample articles, test articles, setting model parameters and the like, and the user terminal 11b is used for uploading articles to be evaluated, displaying evaluation results and the like;
the server 12 provides various business services for the user, including an evaluation server 12a, a training server 12b, and the like, wherein the training server 12b is used for model training and the like, and the evaluation server 12a is used for receiving an evaluation request from a terminal, returning an evaluation result, and the like.
In the present application, the training server 12b is configured to obtain a sample article from the platform terminal 11a or other servers, and then generate a low-quality article corresponding to the sample article according to a preset low-quality article generation manner; constructing a training library based on the sample article and the low-quality article, wherein the element pairs in the training library comprise a first article, a second article, a true probability that the first article is better than the second article, and an article evaluation result of the sample article, the first article is any one of the sample article and the low-quality article, and the second article is the rest of the sample article and the low-quality article; acquiring a neural network to be trained, wherein the neural network comprises at least two feature extraction modules and a feature evaluation module connected with the output of the at least two feature extraction modules; and training the neural network to be trained by using the training library to obtain the trained neural network.
In the application, an evaluation server 12a acquires an article to be evaluated from a user terminal 11b, and a trained neural network is called from a training server 12b, wherein the trained neural network is obtained through training of a sample article and a low-quality article corresponding to the sample article; and processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated.
In the present application, the evaluation server 12a and the training server 12b may be independent physical servers, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.
It should be noted that, the schematic system scenario shown in fig. 1 is only an example, and the servers and the scenarios described in the embodiments of the present application are for more clearly describing the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application, and those skilled in the art can know that, with the evolution of the system and the appearance of a new service scenario, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.
Fig. 2 is a first flowchart of an article evaluation method according to an embodiment of the present application, please refer to fig. 2, wherein the article evaluation method includes the following steps:
201: the training server performs model training.
In one embodiment, the training server performs model training first to obtain a trained neural network, and provides the trained neural network to the evaluation server to invoke evaluation of the articles to be evaluated uploaded by the user.
In one embodiment, the method and the device can generate a large amount of training data based on a small amount of sample articles, and the evaluation result of the generated training data does not need to be manually determined, so that the influence of subjective factors of users on the neural network is avoided, and the neural network pays more attention to the article quality during evaluation; the method comprises the following steps: the training server obtains a sample article with article evaluation results from the platform terminal 11a, processes the sample article to generate a low-quality article corresponding to the sample article according to a preset low-quality article generation mode, builds a training library based on the sample article and the low-quality article, and uses elements in the training library to train the neural network to be trained by using the training library to obtain the trained neural network and then use the evaluation server, wherein the training library comprises a first article, a second article, the real probability of the first article being better than the second article, and the article evaluation results of the sample article, the first article is any one of the sample article and the low-quality article, the second article is the rest of the sample article and the low-quality article, and the neural network to be trained comprises at least two feature extraction modules and a feature evaluation module connected with the output of the at least two feature extraction modules.
In one embodiment, because the quality of an article includes aspects such as article integrity (e.g., the article lacks excessive content, etc.), article logic (e.g., the article content is in a disordered order, etc.), article cleanliness (e.g., there is content in the article that is irrelevant to other content, etc.), the dimension of interest may be sentence level, even word level, and may be set as desired. Based on the above, according to a preset low-quality article generating manner, the step of processing the sample article to generate a low-quality article corresponding to the sample article includes: acquiring a quality attention object, wherein the quality attention object comprises at least one of article integrity, article logic and article cleanliness; and selecting a generation mode matched with the quality attention object to process the sample article according to the quality attention object to generate a low-quality article corresponding to the sample article. Thus, the articles can be evaluated at different angles according to different objects of interest.
In one embodiment, the step of processing the sample article to generate a low-quality article corresponding to the sample article according to the quality attention object, wherein the generating mode of selecting the quality attention object matching includes at least one of the following modes: when the quality attention object comprises the integrity of the article, deleting the deleting unit of the sample article by taking sentences or words as deleting units to obtain the low-quality article; or when the quality attention object comprises article logicality, adjusting the sequence of the adjustment unit of the sample article by taking sentences or words as adjustment units to obtain the low-quality article; or when the quality attention object comprises the article cleanliness, replacing other irrelevant contents with the replacement unit of the sample article by taking sentences or words as the replacement unit, and obtaining the low-quality article. Of course, in other embodiments, the manner of generating the low quality article may be any other manner of reducing the quality of the article, for example, replacing a part of sentences or words in the article with incorrect sentences or words, etc. The purpose of the application is to generate the countermeasure data of the sample articles, and the evaluation results of the low-quality articles are objectively lower than those of the sample articles, if the evaluation results of the low-quality articles are manually performed, the evaluation results of the low-quality articles are possibly higher than those of the sample articles due to subjective factors, the application does not need to acquire the evaluation results for the low-quality articles, and the evaluation results of the low-quality articles are directly defaulted to be lower than those of the corresponding sample articles.
In one embodiment, the step of building a training library based on the sample articles and the low quality articles comprises: determining one article from the sample article and the low quality article as a first article and the other as a second article based on a random manner or other manner; determining the real probability that the first article is better than the second article according to the content of the first article; and constructing the first article, the corresponding second article, the probability that the first article is better than the second article, and the article evaluation result of the sample article as element pairs. By determining one article from the sample article and the low-quality article as a first article and the other article as a second article in a random manner or other manners, the neural network can be prevented from being wrongly trained to be certain that the first article is better than the second article in quality by default, so that an error exists in training results.
In one embodiment, the step of determining the true probability that the first article is better than the second article according to the content of the first article includes: when the content of the first article is the sample article, determining that the real probability that the first article is better than the second article is 1; and when the content of the first article is a low-quality article of the sample article, determining that the real probability that the first article is better than the second article is 0. It can be seen that when the number of the number sample articles is greater than the threshold (e.g., 1000), the number of element pairs with true probabilities of 1 and 0, which are better in the first article than in the second article, is approximately the same, so that the overfitting of the neural network is ensured.
In one embodiment, the training the neural network to be trained using the training library includes: respectively inputting a first article and a second article into different feature extraction modules to obtain first article features of the first article and second article features of the second article; the different feature extraction modules are the same neural network; the first article features and the second article features are spliced, the characteristics are input into the feature evaluation module, and the prediction evaluation result of the articles corresponding to the sample articles and the prediction probability that the quality of the first articles is better than that of the second articles are obtained; and correcting parameters of the feature extraction module and the feature evaluation module according to the loss function, the prediction evaluation result, the article evaluation result, the prediction probability and the real probability.
In one embodiment, the step of correcting the parameters of the feature extraction module and the feature evaluation module according to the loss function, the prediction evaluation result, the article evaluation result, the prediction probability, and the true probability includes: determining a regression task loss function in the loss function according to the prediction evaluation result and the article evaluation result; determining a classification task loss function in the loss function according to the prediction probability and the real probability; the loss function is minimized to reverse correct parameters of the feature extraction module and the feature evaluation module.
Specific training procedures will be described below in connection with specific embodiments.
202: and acquiring the article to be evaluated.
In one embodiment, a teacher or other user uses the user terminal 11b to upload a file including a plurality of articles to be evaluated to an evaluation server at an article upload interface of the interface shown in fig. 8a, where the file may be a file in various manners, such as word, PDF, and picture; and the evaluation server analyzes the articles to obtain articles to be evaluated.
203: invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article.
In one embodiment, this step is performed by the evaluation server by invoking the training results of the training server.
204: and processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated.
In one embodiment, how this step is evaluated includes: inputting the articles to be evaluated into different feature extraction modules simultaneously to obtain a plurality of article features of the articles to be evaluated; the different feature extraction modules are the same neural network; and splicing a plurality of article features of the article to be evaluated, and inputting the article features into the feature evaluation module to obtain a prediction evaluation result of the article corresponding to the article to be evaluated. This step will be described below.
205: and sending an article evaluation result of the article to be evaluated to the terminal.
In one embodiment, after the evaluation server returns the article evaluation results to the user terminal, the terminal may display the evaluation results of each article in the evaluation window as shown in fig. 8 a.
The embodiment provides an artificial intelligence-based article evaluation method, the neural network used by the method is obtained by training sample articles and low-quality articles corresponding to the sample articles, the low-quality articles in the related training data are obtained by processing the sample articles, the requirement on the sample data volume is greatly reduced, meanwhile, the training data comprise the sample articles and the low-quality articles corresponding to the sample articles, the evaluation results of the low-quality articles are lower than those of the sample articles, the low-quality articles do not need to be evaluated manually, the influence of subjective factors on the training results of the neural network is reduced, namely the technical problem that a large number of sample articles with evaluation results or the article quality cannot be objectively reflected in the current article evaluation technology is relieved, the training speed is accelerated, and the article quality of the articles to be evaluated can be objectively reflected.
In order to simplify the following description, the training process and the analysis process according to the present application will now be described with reference to the accompanying drawings.
As shown in fig. 7a, the training process of the present application includes a challenge data generation stage T2 in addition to a sample data acquisition stage T1, a model training stage T3, and a verification stage T4, as compared with the existing training process.
Specifically, as shown in fig. 7c, the current evaluation model includes only one encoding module E-Encoder (i.e., the feature extraction module above) and one evaluation module E-Score (i.e., the feature evaluation module above); as shown in fig. 7d, the evaluation model provided by the present application includes at least 2 encoding modules E-encodings (i.e. the feature extraction modules above) and at least one evaluation module E-Score (i.e. the feature evaluation modules above). The encoding module E-Encoder can be any neural network with feature extraction functions such as word vectors, and the same evaluation module E-Score can be any neural network with feature evaluation based on the features such as word vectors.
In sample data acquisition phase T1:
the current technology shares the same sample data and test data with the present application, and the sample data E includes 500 papers uniformly distributed between 1 and 15 minutes from 50 different titles respectively as described above for 2500 sample papers, wherein the article content of the i-th sample article is recorded as.
In the challenge data generation phase T2:
the prior art does not relate to processing sample data, and directly converting sample articles E in the sample data i0 And a corresponding true score (i.e. true assessment result above) s i0 As training data (E i0 ，s i0 ) Transmitted into the neural network for training.
The application generates challenge data at this stage T2, as shown in fig. 7b, and introduces a challenge data generator E-generator for generating corresponding low quality articles from the sample articles and processing the pairs of elements required for training of the application.
Specifically, as shown in FIG. 7b, the challenge data generator E-generator includes:
delete generator E-Delete: for randomly deleting sample articles E i0 Some parts (in terms of sentences and/or words deleted) of the sample article E i0 Corresponding low quality article E i1 ；
Random generator E-Shuffle: for randomly scrambling sample articles E i0 In the order of some parts (in terms of sentences and/or words) of the article, a sample article E is obtained i0 Corresponding low quality article E i1 ；
Substitution generator E-substitution: for randomly grouping sample articles E i0 Is replaced by the content of another article (for example, other article under the topic in the sample data) by taking sentences and/or words as replacement units to obtain a sample article E i0 Corresponding low quality article E i1 ；
Random module Random for Random slave sample article E i0 And low quality article E i1 One of the articles E is selected as the first article i3 And a second article E i4 And according to the first article E i3 Whether or not it is a sample article E i0 Determining a first article quality E i3 Than the second article E i4 High probability of realism p i0 The method comprises the steps of carrying out a first treatment on the surface of the Wherein, if E i3 ＝E i0 Then p i0 =1, if E i4 ＝E i0 Then p i0 ＝0；
Final output includes element pairs corresponding to all sample articles (E i3 ，E i4 ，s i0 ，p i0 ) Training data of the neural network.
In the application, the operation dimension of the deletion generator E-Delete, the random generator E-Shuffle and the replacement generator E-Replace can be sentences or words; the present application may select one or more from the deletion generator E-Delete, the random generator E-Shuffle, and the substitution generator E-Replace according to the configuration of the neural network (configuration parameters of the quality attention object), which will be described below in connection with the actual scenario.
In model training phase T3:
as shown in fig. 7c and 7d, the current training mode is also different from the training mode of the present application.
Specifically, as shown in fig. 7c, the current training method is: obtaining a sample article E by a coding module E-Encoder i0 Characteristic data T of (2) i0 Then the characteristic data T i0 Input evaluationThe estimation module E-Score obtains a sample article E i0 Is (i.e. predictive assessment results above) s i1 Then based on the minimized regression task loss function MsE-i (s i0 ,s i1 ) And (3) back-propagating the gradient to correct the parameters of the E-Encoder and the E-Score, and completing model training.
Specifically, as shown in fig. 7d, the training mode of the present application is: obtaining a first article E by a first encoding module E-Encoder1 i3 Characteristic data T of (2) i3 Obtaining a second article E by a second encoding module E-Encoder2 i4 Characteristic data T of (2) i4 Then the first article E i3 Characteristic data T of (2) i3 And a second article E i4 Characteristic data T of (2) i4 After splicing, inputting the first articles E to an evaluation module E-Score i3 Predictive assessment of (a) si3 Second article E i4 Predictive scoring of (2) si4 The method comprises the steps of carrying out a first treatment on the surface of the Thereafter, according to the first article E i3 Predictive scoring of (2) si3 And a second article E i4 Is the predictive score s of (2) i4 Obtaining a first article quality E i3 Than the second article E i4 High predictive probability p i1 Wherein, if si3 ＞s i4 Then p i1 =1, if s i3 ＜s i4 Then p i1 =0 according to the first article E i3 Whether or not it is a sample article E i0 Obtaining sample article E i0 Is the predictive score s of (2) i1 Wherein, if E i3 ＝E i0 S is then i1 ＝s i3 If E i4 ＝E i0 S is then i1 ＝s i4 The method comprises the steps of carrying out a first treatment on the surface of the Finally, calculating model loss function according to sample article E i0 Is the true score s of (2) i0 And a predictive score s i1 Obtaining a regression task loss function MSE-i (s i0 ,s i1 ) First article E according to E-generation i3 True probability p of whether the quality is higher i0 And first article E i3 Higher quality predictive probability p i1 Obtaining a classification task loss function l c-i (p i0 ,p i1 ) Further construct the final loss function L i (s i0 ,s i1 ,p i0 ,p i1 ) Wherein the final loss function is a weighted average of the two loss functions: l (L) i (s i0 ,s i1 ,p i0 ,p i1 )＝αl MSE-i (s i0 ,s i1 )+(1-α)l c-i (p i0 ,p i1 ) Where α is an adjustable parameter that can be determined based on the desired degree of interest in the quality of the article that the model achieves, e.g., can be set to 0.6, and then the final loss function L is minimized i (s i0 ,s i1 ,p i0 ,p i1 ) And propagates its gradient back to correct the parameters of the E-Encoder and E-Score. In the application, the model structures of the first encoding module E-Encoder1 and the second encoding module E-Encoder2 are the same, and parameters are shared.
In the verification phase T4:
as shown in fig. 7E, the current technology will test data (i.e., a van article E t ) Inputting the encoding module E-Encoder to obtain a template article E t Characteristic data T of (2) t Will characteristic data T t Input evaluation module E-Score obtains Van article E t Is the predictive score s of (2) t And outputting.
As shown in FIG. 7f, the present application is to test data (i.e. the van article E t ) Respectively inputting the first encoding module E-Encoder1 and the second encoding module E-Encoder2 to obtain two template articles E t Characteristic data T of (2) t Two parts of characteristic data T t After splicing, inputting the E-Score to an evaluation module to obtain a van article E t Is the predictive score s of (2) t And outputting.
In particular, the inventors have conducted experiments to verify the validity of a model, the experiments comprising two batches of data, one batch being scored composition data, but the batch being derived from about 50 english composition topics, about 500 pieces of data per topic, and the other batch being non-scored template data, each composition having a different topic, the total number being about tens of thousands. Two test sets were then constructed, one Limited test set (Limited) from the fifty articles and the other Unlimited test set (Unlimited) from the Van data. Two models, a base model (as the current model in fig. 7 e) based on the conventional score-single-objective training with Limited data as the training set and a multi-objective training method (multi-task) according to the present application were then trained using the Limited test set (Limited), respectively, and the results of the two models and the different results on the test set are shown in table 1 below.
TABLE 1
As can be seen from table 1, the two models showed no significant difference in acceptability in the limited test set, but the Multi-task model was significantly better than the Base model in the non-failed test set, unlimited. In the Unlimited test set, the data come from the model text data, the article quality is higher, the average score of the Base model is 10.3 minutes in the total score of 15 minutes, the average score is slightly higher than the average score of 9.7 of the Limit test set, the model text quality is obviously not reflected, and compared with the model Multi-task model provided by the application, the average score level 13.4 can be quite high, so that the Multi-task model shows better effect on the composition which is not seen.
Based on the description, the method and the device generate a large amount of unlabeled data by using the sample articles, thereby improving the robustness of the model. The normal composition scoring model can cause the model to have very different behaviors in different compositions due to the lack of training data or single training data, the model effect is very unstable, and the model can understand compositions of more subjects by utilizing a large amount of unlabeled data, so that the robustness is enhanced; the application utilizes multi-task learning to make the deep features extracted by the model more relevant to the article quality, the normal model training process takes the article score as the only target, the deep features extracted by the model are not relevant to the article quality, for example, scoring deviation caused by difference of scoring habits of teachers can be learned by the model, the application increases a training target directly relevant to the article quality to judge which article is better, which makes the article deep feature space constrained in the learning process, and the features relevant to the article quality are more likely to be learned; according to the method, at least three article generating methods are used for generating low-quality articles, the learning difficulty and the learning controllability of the model are reduced, the training targets of article quality judgment are combined, the model is easier to learn about the difference of article quality on the basis of ensuring the correctness of the article quality judgment labels, and the generating methods are controllable, so that the method has the capability of determining which characteristics (such as integrity, logic, cleanliness and the like) need to be learned.
In the present application, the specific loss function may be any function having a reduced numerical value difference, and the present application is not limited thereto; in the above embodiment, 2 articles are taken as an example to illustrate, and in practice, more low-quality articles can be provided, and the two-by-two combination of the low-quality articles and the sample articles are respectively used for training the neural network to obtain the neural network with different quality attention objects.
FIG. 3 is a second flowchart of an article evaluation method according to an embodiment of the present application, in this embodiment, scoring an article is taken as an example of an article evaluation result; referring to fig. 3, the article evaluation method includes the following steps:
the embodiment mainly aims at describing the scene of composition evaluation by a teacher.
Since the teacher is paying more attention to the article logic when evaluating the composition, in this scenario the training server has already trained the neural network based on the low quality article generated by the random generator E-Shuffle for the sample article in the sentence dimension, a specific training process can take part in the above description, in this scenario the low quality article E i1 Generated by a random generator E-Shuffle.
301: the teacher uploads a file including a plurality of articles to be evaluated.
In this embodiment, when a teacher needs to evaluate a large number of compositions, the teacher may complete the evaluation by using the artificial intelligence cloud service based on the file uploaded by the user terminal.
A teacher or the like uses an evaluation client, such as an APP, a web page window, or the like, installed in the user terminal 11b, to upload a file including an article to be evaluated, which may be a file of various manners, such as a word, PDF, picture, or the like, to an evaluation server at an article upload interface C1 of an interface as shown in fig. 8 a.
302: the evaluation server acquires the article to be evaluated.
In this embodiment, the evaluation server parses the file uploaded by the user to obtain a plurality of articles to be evaluated.
303: the evaluation server invokes the neural network from the training server.
In this embodiment, the evaluation server determines, according to a manner in which the user uploads the article to be evaluated, for example, through file uploading, that the identity of the uploading user is a teacher, and the requirement is to evaluate the article according to the article logic, and call the trained neural network corresponding to the article logic.
304: the evaluation server uses the trained neural network to score articles.
In this embodiment, the evaluation server uses the neural network to score articles, as shown in fig. 7b, and this step includes: will be evaluated articles E in turn x Respectively inputting the first encoding module E-Encoder1 and the second encoding module E-Encoder2 to obtain an article E to be evaluated x Characteristic data T of (2) x Two parts of characteristic data T x Inputting the spliced articles into an evaluation module E-Score to obtain articles E to be evaluated x Is the predictive score s of (2) x And outputting.
305: the evaluation server sends the article scores of the articles to be evaluated to the user terminal.
306: and the client displays the evaluation result.
As shown in fig. 8a, the client may display the scores of the articles in the evaluation display window C2, or may directly output the scores in a file manner.
According to the embodiment, scoring of the articles in the scenes of examination, operation and the like is completed, the burden of users such as teachers is reduced, and scoring results are only relevant to the quality of the articles and are more objective.
FIG. 4 is a third flowchart of an article evaluation method according to an embodiment of the present application, in this embodiment, scoring an article is also taken as an example of an article evaluation result; referring to fig. 4, the article evaluation method includes the following steps:
the embodiment is mainly described for students to modify the scenes of articles based on evaluation.
Since students need to pay attention to all directions of the article quality when learning to write articles, in this scenario, the training server has already completed training on 6 neural networks based on low-quality articles generated by the deletion generator E-Delete, the random generator E-Shuffle, and the replacement generator E-replay for sample articles in sentence dimension and word dimension, respectively, a specific training process can participate in the above description, in this scenario, the low-quality article E of the first neural network i1 Low quality article E generated in sentence dimension by random generator E-Shuffle, second neural network i1 Low quality article E generated in word dimension by random generator E-Shuffle, third neural network i1 Low quality article E generated in sentence dimension by Delete generator E-Delete, fourth neural network i1 Low quality article E generated in word dimension by Delete generator E-Delete, fifth neural network i1 Low quality article E generated in sentence dimension by substitution generator E-substitution, sixth neural network i1 Generated in the word dimension by the replacement generator E-Replace.
401: the student inputs an article to be evaluated.
In this embodiment, the student may complete the assessment feedback using the artificial intelligence cloud service based on user input while making the article contact.
The user such as student inputs the articles to be evaluated at the article input interface C3 of the interface shown in fig. 8b using the evaluation client, such as APP, web page window, etc. installed in the user terminal 11b, and the client sends the articles to the evaluation server in real time through the evaluation request, and the user such as student can adjust the articles in the article input interface C3 in real time according to the feedback result.
402: the evaluation server acquires the article to be evaluated.
In this embodiment, the evaluation server parses an evaluation request sent by the client to obtain an article to be evaluated.
403: the evaluation server invokes the neural network from the training server.
In this embodiment, the evaluation server determines, according to the manner in which the user uploads the article to be evaluated, for example, through the article input window C3, that the identity of the uploading user is a student, and the requirement is to evaluate the article according to 2 dimensions and 3 objects of interest, and invokes the 6 trained neural networks.
404: the evaluation server uses the trained neural network to score articles.
In this embodiment, the evaluation server uses each neural network to score articles from different dimensions and different objects of interest, as shown in fig. 7b, and this step includes: will be evaluated articles E in turn x Respectively inputting the first encoding module E-Encoder1 and the second encoding module E-Encoder2 to obtain an article E to be evaluated x Characteristic data T of (2) x Two parts of characteristic data T x Inputting the spliced articles into an evaluation module E-Score to obtain articles E to be evaluated x Predictive evaluation s of (2) x Outputting; the output results include 6 scores, and in other embodiments may also include coefficient-based average scores, and the like.
405: the evaluation server transmits the respective article scores of the articles to be evaluated to the user terminal.
406: and the client displays the evaluation result.
As shown in fig. 8b, the client may display 6 scores of the articles in the evaluation display window C4, or may directly output in a file manner.
The embodiment finishes the scoring of the articles in the scenes of learning and writing the articles and the like, so that the users such as students can know how to adjust the content to improve the scoring, and the scoring result is only relevant to the quality of the articles and is more objective.
Accordingly, fig. 5 is a schematic structural diagram of an article evaluation device provided in an embodiment of the present application, please refer to fig. 5, the article evaluation device includes the following modules:
a training module 501 for training the neural network;
an obtaining module 502, configured to obtain an article to be evaluated;
a calling module 503, configured to call a trained neural network, where the trained neural network is obtained by training a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
the evaluation module 504 is configured to process the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and the sending module 505 is configured to send an article evaluation result of the article to be evaluated to a terminal.
In one embodiment, training module 501 is configured to: acquiring a sample article with an article evaluation result; according to a preset low-quality article generating mode, processing the sample articles to generate low-quality articles corresponding to the sample articles; constructing a training library based on the sample article and the low-quality article, wherein the element pairs in the training library comprise a first article, a second article, a true probability that the first article is better than the second article, and an article evaluation result of the sample article, the first article is any one of the sample article and the low-quality article, and the second article is the rest of the sample article and the low-quality article; acquiring a neural network to be trained, wherein the neural network comprises at least two feature extraction modules and a feature evaluation module connected with the output of the at least two feature extraction modules; and training the neural network to be trained by using the training library to obtain the trained neural network.
In one embodiment, training module 501 is configured to: acquiring a quality attention object, wherein the quality attention object comprises at least one of article integrity, article logic and article cleanliness; and selecting a generation mode matched with the quality attention object to process the sample article according to the quality attention object to generate a low-quality article corresponding to the sample article.
In one embodiment, training module 501 is configured to implement at least one of the following: when the quality attention object comprises the integrity of the article, deleting the content of the sample article by taking sentences or words as a deleting unit to obtain the low-quality article; or when the quality attention object comprises article logicality, moving the content of the sample article by taking sentences or words as adjustment units to obtain the low-quality article; or when the quality attention object comprises the article cleanliness, replacing other irrelevant contents with the contents of the sample article by taking sentences or words as replacement units, so as to obtain the low-quality article.
In one embodiment, training module 501 is configured to: determining one article from the sample article and the low-quality article as a first article, and the other article as a second article; determining the real probability that the first article is better than the second article according to the content of the first article; and constructing the first article, the corresponding second article, the probability that the first article is better than the second article, and the article evaluation result of the sample article as element pairs.
In one embodiment, training module 501 is configured to: when the content of the first article is the sample article, determining that the real probability that the first article is better than the second article is 1; and when the content of the first article is a low-quality article corresponding to the sample article, determining that the real probability that the first article is better than the second article is 0.
In one embodiment, training module 501 is configured to: respectively inputting a first article and a second article into different feature extraction modules to obtain first article features of the first article and second article features of the second article; the different feature extraction modules are the same neural network; the first article features and the second article features are spliced, the characteristics are input into the feature evaluation module, and the prediction evaluation result of the articles corresponding to the sample articles and the prediction probability that the quality of the first articles is better than that of the second articles are obtained; and correcting parameters of the feature extraction module and the feature evaluation module according to the loss function, the prediction evaluation result, the article evaluation result, the prediction probability and the real probability.
In one embodiment, training module 501 is configured to: determining a regression task loss function in the loss function according to the prediction evaluation result and the article evaluation result; determining a classification task loss function in the loss function according to the prediction probability and the real probability; the loss function is minimized to reverse correct parameters of the feature extraction module and the feature evaluation module.
In one embodiment, the evaluation module 504 is configured to: inputting the articles to be evaluated into different feature extraction modules simultaneously to obtain a plurality of article features of the articles to be evaluated; the different feature extraction modules are the same neural network; and splicing a plurality of article features of the article to be evaluated, and inputting the article features into the feature evaluation module to obtain a prediction evaluation result of the article corresponding to the article to be evaluated.
Correspondingly, the embodiment of the application also provides computer equipment, which comprises a server or a terminal and the like.
As shown in fig. 6, the computer device may include Radio Frequency (RF) circuitry 601, memory 602 including one or more computer readable storage media, input unit 603, display unit 604, sensor 605, audio circuitry 606, wireless fidelity (WiFi, wireless Fidelity) module 607, processor 608 including one or more processing cores, and power supply 609. Those skilled in the art will appreciate that the computer device structure shown in FIG. 6 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:
The RF circuit 601 may be used for receiving and transmitting signals during a message or a call, and in particular, after receiving downlink information of a base station, the downlink information is processed by one or more processors 608; in addition, data relating to uplink is transmitted to the base station. The memory 602 may be used to store software programs and modules that are stored in the memory 602 for execution by the processor 608 to perform various functional applications and data processing. The input unit 603 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
The display unit 604 may be used to display information entered by a user or provided to a user as well as various graphical user interfaces of a computer device, which may be composed of graphics, text, icons, video, and any combination thereof.
The computer device may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. The audio circuit 606 includes a speaker and the microphone can provide an audio interface between the user and the computer device.
WiFi belongs to a short-distance wireless transmission technology, and computer equipment can help a user to send and receive emails, browse webpages, access streaming media and the like through a WiFi module 607, so that wireless broadband Internet access is provided for the user. Although fig. 6 shows a WiFi module 607, it is understood that it does not belong to the necessary constitution of the computer device, and can be omitted entirely as needed within the scope of not changing the essence of the application.
Processor 608 is the control center of the computer device, and uses various interfaces and lines to connect the various parts of the overall handset, perform various functions of the computer device and process data by running or executing software programs and/or modules stored in memory 602, and invoking data stored in memory 602.
The computer device also includes a power supply 609 (e.g., a battery) for powering the various components, which may be logically connected to the processor 608 via a power management system so as to perform functions such as managing charge, discharge, and power consumption via the power management system.
Although not shown, the computer device may further include a camera, a bluetooth module, etc., which will not be described herein. In particular, in this embodiment, the processor 608 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 608 executes the application programs stored in the memory 602, so as to implement the following functions:
Acquiring an article to be evaluated;
invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and sending the article evaluation result of the article to be evaluated to a terminal.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and the portions of an embodiment that are not described in detail in the foregoing embodiments may be referred to in the foregoing detailed description, which is not repeated herein.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, embodiments of the present application provide a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the following functions:
Acquiring an article to be evaluated;
invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and sending the article evaluation result of the article to be evaluated to a terminal.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Wherein the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The steps in any method provided by the embodiment of the present application may be executed by the instructions stored in the storage medium, so that the beneficial effects that any method provided by the embodiment of the present application may be achieved, which are detailed in the previous embodiments and are not repeated herein.
Meanwhile, the embodiment of the application provides a computer program product or a computer program, which comprises computer instructions, wherein the computer instructions are stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described above. For example, the following functions are implemented:
Acquiring an article to be evaluated;
invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and sending the article evaluation result of the article to be evaluated to a terminal.
The foregoing describes in detail a method and apparatus for article assessment, a computer device and a computer readable storage medium provided by embodiments of the present application, and specific examples are applied to illustrate the principles and embodiments of the present application, where the foregoing examples are only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will vary in the specific embodiments and application scope according to the ideas of the present application, the present description should not be construed as limiting the present application in summary.
Claims (9)
1. An article evaluation method, comprising:
acquiring a sample article with an article evaluation result;
According to a preset low-quality article generating mode, processing the sample articles to generate low-quality articles corresponding to the sample articles;
constructing a training library based on the sample article and the low-quality article, wherein the element pairs in the training library comprise a first article, a second article, a true probability that the first article is better than the second article, and an article evaluation result of the sample article, the first article is any one of the sample article and the low-quality article, and the second article is the rest of the sample article and the low-quality article;
acquiring a neural network to be trained, wherein the neural network comprises at least two feature extraction modules and a feature evaluation module connected with the output of the at least two feature extraction modules;
training the neural network to be trained by using the training library to obtain a trained neural network;
acquiring an article to be evaluated;
invoking a trained neural network, wherein the trained neural network is obtained through training of a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
Processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and sending the article evaluation result of the article to be evaluated to a terminal.
2. The article evaluation method according to claim 1, wherein the step of processing the sample article to generate the low-quality article corresponding to the sample article according to a preset low-quality article generation manner includes:
acquiring a quality attention object, wherein the quality attention object comprises at least one of article integrity, article logic and article cleanliness;
and selecting a generation mode matched with the quality attention object to process the sample article according to the quality attention object to generate a low-quality article corresponding to the sample article.
3. The article evaluation method according to claim 2, wherein the step of selecting a generation mode of the quality attention object matching to process the sample article to generate a low-quality article corresponding to the sample article according to the quality attention object includes at least one of the following modes:
when the quality attention object comprises the integrity of the article, deleting the content of the sample article by taking sentences or words as a deleting unit to obtain the low-quality article;
When the quality attention object comprises article logicality, moving the content of the sample article by taking sentences or words as adjustment units to obtain the low-quality article;
and when the quality attention object comprises article cleanliness, replacing the content of the sample article with other irrelevant content by taking sentences or words as replacement units, so as to obtain the low-quality article.
4. The article evaluation method of claim 1, wherein the step of constructing a training library based on the sample articles and the low quality articles comprises:
determining one article from the sample article and the low-quality article as a first article, and the other article as a second article;
determining the real probability that the first article is better than the second article according to the content of the first article;
and constructing the first article, the corresponding second article, the probability that the first article is better than the second article, and the article evaluation result of the sample article as element pairs.
5. The article evaluation method according to claim 4, wherein the step of determining a true probability that the first article is better than the second article based on the content of the first article, comprises:
When the content of the first article is the sample article, determining that the real probability that the first article is better than the second article is 1;
and when the content of the first article is a low-quality article corresponding to the sample article, determining that the real probability that the first article is better than the second article is 0.
6. The article evaluation method of claim 1, wherein the training the neural network to be trained using the training library comprises:
respectively inputting a first article and a second article into different feature extraction modules to obtain first article features of the first article and second article features of the second article; the different feature extraction modules are the same neural network;
the first article feature and the second article feature are spliced, the feature evaluation module is input, and a prediction evaluation result of the article corresponding to the sample article and a prediction probability that the quality of the first article is better than that of the second article are obtained;
and correcting parameters of the feature extraction module and the feature evaluation module according to the loss function, the prediction evaluation result, the article evaluation result, the prediction probability and the real probability.
7. The article evaluation method according to claim 6, wherein the step of correcting parameters of the feature extraction module and the feature evaluation module according to a loss function, the predictive evaluation result, the article evaluation result, the predictive probability, the true probability, comprises:
determining a regression task loss function in the loss function according to the prediction evaluation result and the article evaluation result;
determining a classification task loss function in the loss function according to the prediction probability and the real probability;
and minimizing a final loss function to inversely correct parameters of the feature extraction module and the feature evaluation module, wherein the final loss function is a weighted average of the regression task loss function and the classification task loss function.
8. The article evaluation method according to any one of claims 1 to 7, wherein the step of processing the article to be evaluated using the trained neural network to obtain an article evaluation result of the article to be evaluated includes:
inputting the articles to be evaluated into different feature extraction modules simultaneously to obtain a plurality of article features of the articles to be evaluated; the different feature extraction modules are the same neural network;
And splicing a plurality of article features of the article to be evaluated, and inputting the article features into the feature evaluation module to obtain a prediction evaluation result of the article corresponding to the article to be evaluated.
9. An article evaluation apparatus, comprising:
the training module is used for acquiring a sample article with an article evaluation result; according to a preset low-quality article generating mode, processing the sample articles to generate low-quality articles corresponding to the sample articles; constructing a training library based on the sample article and the low-quality article, wherein the element pairs in the training library comprise a first article, a second article, a true probability that the first article is better than the second article, and an article evaluation result of the sample article, the first article is any one of the sample article and the low-quality article, and the second article is the rest of the sample article and the low-quality article; acquiring a neural network to be trained, wherein the neural network comprises at least two feature extraction modules and a feature evaluation module connected with the output of the at least two feature extraction modules; training the neural network to be trained by using the training library to obtain a trained neural network;
The acquisition module is used for acquiring articles to be evaluated;
the invoking module is used for invoking the trained neural network, wherein the trained neural network is obtained by training a sample article and a low-quality article; wherein the low quality article is obtained by preprocessing the sample article;
the evaluation module is used for processing the article to be evaluated by using the trained neural network to obtain an article evaluation result of the article to be evaluated;
and the sending module is used for sending the article evaluation result of the article to be evaluated to the terminal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917172.0A CN112115703B (en) | 2020-09-03 | 2020-09-03 | Article evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010917172.0A CN112115703B (en) | 2020-09-03 | 2020-09-03 | Article evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112115703A CN112115703A (en) | 2020-12-22 |
| CN112115703B true CN112115703B (en) | 2023-10-17 |
Family
ID=73801741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010917172.0A Active CN112115703B (en) | 2020-09-03 | 2020-09-03 | Article evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112115703B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112989816B (en) * | 2021-04-20 | 2021-10-01 | 中译语通科技股份有限公司 | Text content quality evaluation method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999483A (en) * | 2011-09-16 | 2013-03-27 | 北京百度网讯科技有限公司 | Method and device for correcting text |
| CN107193805A (en) * | 2017-06-06 | 2017-09-22 | 北京百度网讯科技有限公司 | Article Valuation Method, device and storage medium based on artificial intelligence |
| CN108446813A (en) * | 2017-12-19 | 2018-08-24 | 清华大学 | A kind of method of electric business service quality overall merit |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10417328B2 (en) * | 2018-01-05 | 2019-09-17 | Searchmetrics Gmbh | Text quality evaluation methods and processes |
-
2020
- 2020-09-03 CN CN202010917172.0A patent/CN112115703B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999483A (en) * | 2011-09-16 | 2013-03-27 | 北京百度网讯科技有限公司 | Method and device for correcting text |
| CN107193805A (en) * | 2017-06-06 | 2017-09-22 | 北京百度网讯科技有限公司 | Article Valuation Method, device and storage medium based on artificial intelligence |
| CN108446813A (en) * | 2017-12-19 | 2018-08-24 | 清华大学 | A kind of method of electric business service quality overall merit |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于深度学习的科普文章评论情感分析;姚彬 等;科研信息化技术与应用;第9卷(第4期);第27-33页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112115703A (en) | 2020-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220180882A1 (en) | Training method and device for audio separation network, audio separation method and device, and medium | |
| US10922866B2 (en) | Multi-dimensional puppet with photorealistic movement | |
| US20190303768A1 (en) | Community Question Answering-Based Article Recommendation Method, System, and User Device | |
| US20150243279A1 (en) | Systems and methods for recommending responses | |
| CN111754985B (en) | Training of voice recognition model and voice recognition method and device | |
| CN109478204A (en) | The machine of non-structured text understands | |
| WO2017186050A1 (en) | Segmented sentence recognition method and device for human-machine intelligent question-answer system | |
| CN106649345A (en) | Automatic session creator for news | |
| US11868738B2 (en) | Method and apparatus for generating natural language description information | |
| CN110491218A (en) | A kind of online teaching exchange method, device, storage medium and electronic equipment | |
| CN110008326B (en) | Knowledge abstract generation method and system in session system | |
| CN109978139B (en) | Method, system, electronic device and storage medium for automatically generating description of picture | |
| CN105374249A (en) | On-line question test method based on picture display in mobile terminal | |
| CN110516749A (en) | Model training method, method for processing video frequency, device, medium and calculating equipment | |
| WO2020186484A1 (en) | Automatic image description generation method and system, electronic device, and storage medium | |
| CN113962965A (en) | Image quality evaluation method, device, equipment and storage medium | |
| CN112115703B (en) | Article evaluation method and device | |
| CN104900094A (en) | Implementation method and device for writing simulation in terminal | |
| Cao et al. | Construction of sports safety information mining platform based on multimedia data sharing technology | |
| CN113257060A (en) | Question answering solving method, device, equipment and storage medium | |
| CN112843681A (en) | Virtual scene control method and device, electronic equipment and storage medium | |
| CN117271745A (en) | Information processing method and device, computing equipment and storage medium | |
| JP2023078411A (en) | Information processing method, model training method, apparatus, appliance, medium and program product | |
| Wang et al. | Online mobile teaching methods based on Android in the 5G environment | |
| CN113033209A (en) | Text relation extraction method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |