CN116127367A

CN116127367A - Method and device for auditing service evaluation and computer readable storage medium

Info

Publication number: CN116127367A
Application number: CN202111330965.3A
Authority: CN
Inventors: 石路路; 孟平; 史超
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-11-10
Filing date: 2021-11-10
Publication date: 2023-05-16

Abstract

The application provides a method and a device for auditing service evaluation and a computer readable storage medium. The method comprises the steps of obtaining a service evaluation text of a target service business, and detecting whether the service evaluation text comprises a target health keyword or not. If the service evaluation text comprises the target health keywords, outputting the service evaluation text to an auditing platform, and determining the target auditing category of the service evaluation text through the auditing platform. If the service evaluation text does not comprise the target health keywords, the emotion features and the word vector features corresponding to the service evaluation text are obtained, and the emotion features and the word vector features are input into a text classification model. And determining a target audit class of the service evaluation text based on the initial audit class output by the text classification model or outputting the service evaluation text to an audit platform, and determining the target audit class of the service evaluation text through the audit platform. By adopting the method and the device, the auditing efficiency of the service evaluation text can be improved, and the auditing result is high in objectivity and high in applicability.

Description

Method and device for auditing service evaluation and computer readable storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for auditing service evaluation, and a computer readable storage medium.

Background

In service industries (such as hotel, restaurant, e-commerce and other service industries), a service provider provides a service, a user can subscribe or purchase the service provided by the service provider, the service provided by the service provider can be evaluated after enjoying the service, the service provider can receive the service evaluation from the user, and an improvement scheme can be extracted based on the evaluation of the user for improving the service quality. In order to extract effective information for improving the quality of service from a large number of service evaluations, a service provider analyzes (and/or reviews) the service evaluations of users to distinguish user-satisfactory evaluations from user-unsatisfactory evaluations from the service evaluations, to present the user-satisfactory evaluations or extract improvement comments from the user-unsatisfactory evaluations, and the like.

The inventor of the application finds that in the research and experiment process, the prior art is mainly based on the fact that staff of a service provider analyze a large number of service evaluations provided by the user by pieces of service evaluation texts manually, so that the manual consumption is large, the auditing efficiency is low, the subjectivity of auditing results is strong, and the applicability is poor.

Disclosure of Invention

The embodiment of the application provides a method and a device for auditing service evaluation and a computer readable storage medium, which can improve the auditing efficiency of service evaluation, and have more accurate auditing results and strong applicability.

In a first aspect, an embodiment of the present application provides a method for auditing service evaluation, where the method includes: and acquiring a service evaluation text of the target service business, and detecting whether the service evaluation text comprises the target health keywords. Here, the target service may be a service such as hotel, restaurant, travel, and e-commerce. If the service evaluation text comprises the target health keywords, the service evaluation text can be output to an auditing platform, and the target auditing category of the service evaluation text is determined through the auditing platform. Here, the audit platform may be a manual audit platform to determine a target audit category of the service evaluation text based on the manual audit. If the service evaluation text does not comprise the target health keywords, the emotion features and the word vector features corresponding to the service evaluation text can be obtained, and the emotion features and the word vector features are input into a text classification model. And determining the target audit category of the service evaluation text based on the initial audit category of the service evaluation text output by the text classification model or outputting the service evaluation text to an audit platform so as to determine the target audit category of the service evaluation text through the audit platform. In the embodiment of the application, the target health keywords are detected on the service evaluation text, and the service evaluation text including the target health keywords is output to an auditing platform to obtain the target auditing category of the service evaluation text. Meanwhile, emotion features constructed based on an emotion word library of the target service business are added in feature elements of the input text classification model. The text classification model in the business server can output an initial audit category of the service evaluation text based on the input word vector characteristics and emotion characteristics, and determine a target audit category of the service evaluation text based on the initial audit category or output the service evaluation text to an audit platform to acquire the target audit category. The operation is simple, the auditing efficiency is high, the auditing result objectivity is strong, and the applicability is strong.

With reference to the first aspect, in a first possible implementation manner, the service evaluation text may be segmented to obtain a plurality of independent words. And comparing the independent words with the health keywords in the health word stock to detect whether the service evaluation text comprises the target health keywords in the health word stock. Here, the health keyword may be related to an elegant word (such as a dirty word, etc.), or may be an industry sensitive word (such as money, etc.), etc. In the embodiment of the application, for the service evaluation text which contains the target health keywords (such as the unqualified vocabulary) is detected, the auditing result can be directly confirmed through a manual auditing mode, and the service evaluation text which does not contain the target health keywords can be subjected to auditing type identification through a text classification model, so that the number of service evaluation texts to be audited for manual auditing can be reduced, and the auditing efficiency of the service evaluation text is improved.

With reference to the first aspect and any one of the first possible implementation manners of the first aspect, in a second possible implementation manner, the obtaining the emotion feature and the word vector feature of the service evaluation text may specifically be obtaining the emotion feature of the service evaluation text based on the emotion word stock of the target service and the service evaluation text, extracting a keyword from the service evaluation text based on a target feature word extraction algorithm, and generating a word vector feature of the service evaluation text based on the extracted keyword. In the embodiment of the application, the emotion characteristics constructed based on the emotion word library of the target service are newly added on the basis of the word vector characteristics, so that the characteristic expression capacity of the text classification model is enhanced, and the auditing accuracy is improved.

With reference to the second possible implementation manner of the first aspect, in a third possible implementation manner, obtaining the emotion feature of the service evaluation text based on the emotion word library of the target service and the service evaluation text may specifically be: firstly, dividing the service evaluation text into a plurality of clauses, determining the word types, the words of the word types and the emotion weights contained in each clause based on an emotion word stock of a target service, and determining the sentence patterns of each clause and the emotion weights of each sentence pattern. Here, the emotion word stock may include an emotion word stock, a degree adverb stock, a query word stock, and a turning word stock. Finally, based on the words and emotion weights of the word types in each clause, the sentence patterns of each clause and the emotion weights of each sentence pattern, the emotion characteristics of the service evaluation text. In the embodiment of the application, the emotion characteristics constructed based on the emotion word library of the target service are newly added on the basis of the word vector characteristics, so that the characteristic expression capacity of the text classification model is enhanced, and the auditing accuracy is improved.

With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, determining, based on an emotion word library of the target service, a word type included in each clause, a word of each word type, and an emotion weight may specifically be: matching words included in each clause with emotion words in an emotion word library of the target service to determine positive emotion words and/or negative emotion words included in each clause, and determining emotion weights corresponding to the positive emotion words and/or the negative emotion words included in each clause. The emotion words in the emotion word library at least comprise two types of emotion words, the at least two types of emotion words at least comprise positive emotion words and negative emotion words, one type of emotion words comprises one or more words, and one type of emotion words corresponds to one emotion weight. In the embodiment of the application, the positive emotion words and/or the negative emotion words and the emotion weights corresponding to the positive emotion words and/or the negative emotion words are determined through the emotion word library, so that the operation is simple, and the extraction efficiency of emotion features is high.

With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner, the emotion word library of the target service further includes a degree adverb, and after determining the positive emotion word and/or the negative emotion word included in each clause, the words included in each clause may be further matched with the degree adverb in the emotion word library of the target service. If the target degree adverbs included in the emotion word library are determined from all clauses and the target degree adverbs are positioned before the positive emotion words and/or the negative emotion words, determining emotion weights corresponding to the target degree adverbs so as to obtain emotion weights corresponding to words with the word types of the degree adverbs in all clauses. In the embodiment of the application, the target degree adverbs and the emotion weights thereof are determined to be included in each clause through the emotion word library, so that the operation is simple, and the extraction efficiency of emotion features is high.

With reference to the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner, determining a sentence pattern of each clause and an emotion weight of each sentence pattern may specifically be: detecting whether each clause comprises an exclamation mark or not by taking a sentence as a unit, if any target clause in each clause comprises the exclamation mark and the target clause comprises one or more emotion words of a target service, determining that the sentence pattern of the target clause is the exclamation mark, and determining that the target clause is an emotion weight corresponding to the exclamation mark.

With reference to the fifth possible implementation manner of the first aspect, in a seventh possible implementation manner, determining a sentence pattern of each clause and an emotion weight of each sentence pattern may specifically be: detecting whether each clause comprises a question mark or not by taking a sentence as a unit, determining whether each clause comprises a query word or not based on an emotion word library of a target service, if any target clause in each clause comprises the question mark and the target clause comprises one or more query words, determining the sentence pattern of the target clause as the question sentence, and determining that the target clause is an emotion weight corresponding to the question sentence.

With reference to the fifth possible implementation manner of the first aspect, in an eighth possible implementation manner, determining a sentence pattern of each clause and an emotion weight of each sentence pattern may specifically be to match a word included in each clause with a turning word in an emotion word library of the target service, and if any target clause in each clause includes one or more target turning words in an emotion word library of the target service, determining that the sentence pattern of the target clause is a turning sentence, and determining that the target clause is an emotion weight corresponding to the turning sentence.

In the embodiment of the application, the sentence patterns of each clause and the emotion weight of each sentence pattern are determined through the emotion word library, the sentence pattern determining modes are various, the operation is simple, and the extraction efficiency of emotion features is high.

With reference to any one of the sixth possible implementation manner to the eighth possible implementation manner of the first aspect, in a ninth possible implementation manner, the emotion score of each clause may be calculated based on the positive emotion words and their corresponding emotion weights, the negative emotion words and their corresponding emotion weights, and/or the target degree adverbs and their corresponding emotion weights, which are included in each clause. Updating emotion scores of the clauses based on the sentence patterns of the clauses and emotion weights of the sentence patterns, and obtaining emotion characteristics of the service evaluation text based on the emotion scores of the clauses. In the embodiment of the application, the emotion characteristics constructed based on the emotion word library of the target service are newly added on the basis of the word vector characteristics, so that the characteristic expression capacity of the text classification model is enhanced, and the auditing accuracy is improved.

With reference to the ninth possible implementation manner of the first aspect, in a tenth possible implementation manner, calculating the emotion score of each clause based on the positive emotion word and its corresponding emotion weight, the negative emotion word and its corresponding emotion weight, and/or the target degree adverb and its corresponding emotion weight included in each clause may specifically be: and calculating emotion scores of the clauses based on the active emotion words and the emotion weights corresponding to the active emotion words and/or the passive emotion words and the emotion weights corresponding to the passive emotion words, wherein the emotion scores comprise active emotion scores and/or passive emotion scores. And updating emotion scores of the clauses based on emotion weights of target degree adverbs contained in the clauses, wherein the emotion weights of the target degree adverbs are 1 when the target degree adverbs are not contained in the clauses. In the embodiment of the application, the emotion score of each clause is calculated through the active emotion words and the emotion weights corresponding to the active emotion words and/or the passive emotion words and the emotion weights corresponding to the passive emotion words contained in each clause, and the emotion score is updated through the emotion weights of the target degree adverbs, so that the feature expression capability of the text classification model is enhanced, and the auditing accuracy is improved.

With reference to the tenth possible implementation manner of the first aspect, in an eleventh possible implementation manner, a sum of positive emotion values, a sum of negative emotion values, a positive emotion value average, a negative emotion value average, and a total emotion value of all clauses are obtained based on emotion scores of the clauses. And constructing the emotion characteristics of the service evaluation text based on one or more of the sum of positive emotion values, the sum of negative emotion values, the average of positive emotion values, the average of negative emotion values and the total emotion value. In the embodiment of the application, the emotion characteristics constructed based on the emotion word library of the target service are newly added on the basis of the word vector characteristics, so that the characteristic expression capacity of the text classification model is enhanced, and the auditing accuracy is improved.

With reference to the second possible implementation manner of the first aspect, in a twelfth possible implementation manner, extracting a keyword from the service evaluation text based on the target feature word extraction algorithm, and generating a word vector feature of the service evaluation text based on the extracted keyword may specifically be: the method comprises the steps of segmenting a service evaluation text to obtain a plurality of independent words, and extracting one or more keywords from the plurality of independent words obtained by segmentation based on a target feature word extraction algorithm. And carrying out vectorization processing on one or more keywords through a word vector conversion model to obtain a keyword vector sequence, and obtaining word vector characteristics of the service evaluation text based on the keyword vector sequence. In the embodiment of the application, the redundant components in the service evaluation text can be reduced by carrying out keyword extraction, vectorization processing, partial vector extraction and other processing on the service evaluation text, so that the auditing efficiency is improved, and the auditing accuracy is higher.

With reference to the twelfth possible implementation manner of the first aspect, in a thirteenth possible implementation manner, the target feature word extracting algorithm may be a word frequency-inverse document frequency algorithm, and extracting, based on the target feature word extracting algorithm, one or more keywords from a plurality of independent words may specifically be: and calculating word frequency-inverse document frequency values of the words of the independent words by using a word frequency-inverse document frequency algorithm, sorting the words in a descending order according to the corresponding word frequency-inverse document frequency values, and determining the first N words in the sorting result as keywords, wherein N is a positive integer not greater than the total number of the independent words. Or selecting one or more words with word frequency-inverse document frequency value not smaller than a preset value from the plurality of independent words as keywords. By selecting partial words as key words by using the word frequency-inverse document frequency algorithm, redundant components in the service evaluation text can be reduced, so that the auditing efficiency is improved, and the auditing accuracy is higher.

With reference to the twelfth possible implementation manner of the first aspect, in a fourteenth possible implementation manner, the target feature word extraction algorithm may be a chi-square test algorithm, and extracting, based on the target feature word extraction algorithm, one or more keywords from a plurality of independent words may specifically be: and obtaining the chi-square value of each word in the plurality of independent words through a chi-square test calculation formula, sorting the plurality of independent words in descending order according to the chi-square value of each word, and selecting the first M words in the sorting result as key words, wherein M is a positive integer not greater than the total number of the independent words. By using the chi-square checking algorithm to select partial words as keywords, redundant components in the service evaluation text can be reduced, so that the auditing efficiency is improved, and the auditing accuracy is higher.

With reference to any one of the twelfth possible implementation manner of the first aspect to the fourteenth possible implementation manner of the first aspect, in a fifteenth possible implementation manner, the obtaining the word vector feature of the service valuation text based on the keyword vector sequence may specifically be: and acquiring the spearman correlation coefficient between each vector in the keyword vector sequence, if the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in any two vectors and reserving the other vector to obtain a keyword vector sequence with the vector removed, and obtaining word vector characteristics based on the keyword vector sequence with the vector removed. And partial keyword vectors are removed based on the Szelman correlation coefficients, so that more simplified word vector features can be obtained, and the auditing efficiency and the auditing accuracy are improved.

With reference to any one of the first aspect to the fifteenth possible implementation manner of the first aspect, in a sixteenth possible implementation manner, determining a target audit category of the service valuation text or outputting the service valuation text to the audit platform based on the initial audit category of the service valuation text output by the text classification model may specifically be: the initial audit category of the service evaluation text output by the text classification model (which may be a text classification model based on the FastText algorithm) is obtained, where the initial audit category may include a first audit category (i.e., a failure, poor or negative evaluation on behalf of the service evaluation text) and a second audit category (i.e., a good or positive evaluation on behalf of the service evaluation text). And if the initial audit category is the first audit category, outputting the service evaluation text to an audit platform, and if the initial audit category is the second audit category, determining the initial audit category as a target audit category of the service evaluation text. The first audit category comprises audit failing, the second audit category comprises audit passing, or the first audit category is poor, and the second audit category is good. In the embodiment of the application, the initial audit category is the second audit category, and accuracy judgment and/or correction are carried out through manual audit, so that the accuracy of a final audit result is improved, and the false audit result caused by misjudgment of a model is reduced.

With reference to the sixteenth possible implementation manner of the first aspect, in a seventeenth possible implementation manner, sample service evaluation texts of at least two audit categories are obtained from a service evaluation sample library of the target service business, where the at least two categories include a first audit category and a second audit category, and an audit category label of the sample service evaluation text is included in the sample service evaluation text of any audit category. And inputting the sample service evaluation text into a text classification model, and learning the sample service evaluation text through the text classification model to acquire the capability of identifying the auditing category of any service evaluation text.

With reference to the seventeenth possible implementation manner of the first aspect, in a seventeenth possible implementation manner, after determining the target audit category of the service valuation text, the target audit category may be further added as a category label of the service valuation text, and the service valuation text and the category label thereof may be added to the service valuation sample library to update the service valuation sample library. In the embodiment of the application, the robustness of the model can be enhanced by continuously updating the sample service evaluation text contained in the service evaluation sample library and iterating the text classification model in real time or periodically, and the model auditing result is more accurate.

In a second aspect, an embodiment of the present application provides an auditing apparatus for service evaluation, where the auditing apparatus includes a unit and/or a module for performing the method for auditing service evaluation provided by the first aspect and/or any one of possible implementation manners of the first aspect, so that the beneficial effects (or advantages) provided by the method provided by the first aspect can also be achieved.

In a third aspect, embodiments of the present application provide a terminal device, including a memory, a transceiver, and a processor; wherein the memory, transceiver and processor are connected by a communication bus or the processor and transceiver are configured to couple with the memory. The memory is used for storing a set of program codes, and the transceiver and the processor are used for calling the program codes stored in the memory to execute the auditing method of the service evaluation provided by the first aspect and/or any one of the possible implementation manners of the first aspect, so that the beneficial effects of the method provided by the first aspect can be achieved.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, where an instruction is stored, where the instruction, when executed on a network device, causes a terminal device to execute an auditing method of service evaluation provided by the foregoing first aspect and/or any possible implementation manner of the first aspect, and may also achieve the beneficial effect provided by the method provided by the first aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product including instructions, where the computer program product when executed on a terminal device causes the terminal device to perform the method for auditing service assessment provided in the first aspect, and also achieves the beneficial effects provided by the method provided in the first aspect.

Drawings

FIG. 1 is a schematic service interaction diagram provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of a system architecture according to an embodiment of the present application;

FIG. 3 is a flow chart of an auditing method of service valuation according to an embodiment of the present application;

FIG. 4 is another flow chart of an auditing method of service valuation provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of training and optimizing a text classification model according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an auditing apparatus for service evaluation according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

Detailed Description

The auditing method of service evaluation provided by the embodiment of the application can be executed by computer equipment (or simply referred to as equipment) such as terminal equipment or a service server. The description of the business server (or simply called server), the equipment terminal, the computer equipment, the equipment and the like in the application can be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, and a cloud server for providing basic cloud computing services such as a cloud database, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, big data, artificial intelligent platform and the like. The terminal device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a palm computer, a mobile internet device (mobile internet device, MID), a wearable device (e.g., a smart watch, a smart bracelet, etc.), a smart computer, a smart vehicle, etc. which may run the above application. The terminal device and the service server may be directly or indirectly connected through a wired or wireless manner, which is not limited herein. For convenience of description, an example of the apparatus will be described below.

The auditing method of the service evaluation is suitable for auditing and classifying the service evaluation texts of the service industries such as hotels, restaurants, electronic commerce and the like, and the service evaluation texts can be texts obtained by evaluating, sharing or complaining the service provided by the service providers of the service industries such as hotels, restaurants, traveling, electronic commerce and the like. The service evaluation in the service industry may be obtained through an internet platform, which may be a third party consumption comment website, a life service e-commerce platform, an Online travel service providing platform (such as Online travel agency (Online TravelAgency, OTA) and business travel management company (Travel Management Companies, TMC)), an Online shopping retail platform, and a consumer service website or a mobile application independently developed by a service provider in each service industry, and the internet platform may be specifically determined according to an actual application scenario, and is not limited herein. After the service provider of each service industry (hotel, restaurant, travel, e-commerce, etc.) is resident on the relevant internet platform (third party consumption review website, living service e-commerce platform, online travel agency, online shopping retail platform, etc.), the user can subscribe to and use the relevant product and/or service with the service provider through the internet platform. In addition, the user may send a service rating to the internet platform based on the experience of the service provider's product and/or service after the end of using the product or service. The internet platform can display the received service evaluation to all users using the platform, and can also feed back the service evaluation to the resident service provider. On the one hand, the user can screen out better service providers through service evaluation on the Internet platform, and on the other hand, the service providers can extract improvement ideas of the user from a large number of service evaluation, so that the service quality is further improved. Taking the hotel services industry as an example, a service provider (e.g., a hotel) in the hotel services industry may be resident on an online travel services providing platform, such as an online travel agency (i.e., OTA) or a business administration company (i.e., TMC), through which a user may subscribe to and use related products and/or services with the service provider (e.g., the hotel). Referring to fig. 1, fig. 1 is a schematic service interaction diagram provided in an embodiment of the present application. As shown in fig. 1, taking an online travel service providing platform such as an Online Travel Agency (OTA) as an example, first, for a hotel (or simply referred to as a hotel) that cooperates with a cooperation platform (such as the OTA), the hotel can enter the OTA, and the hotel and the OTA negotiate about a cooperation item such as a room price and a commission proportion, and the two parties agree to have a post agreement. The user (e.g., consumer) can then query and book the hotel on the OTA consumer-oriented platform. The user can send the check-in information of the hotel to the OTA, and the OTA confirms the check-in information and feeds back the check-in information to the hotel partner. After confirming the check-in information of the OTA and the cooperation hotels, the OTA returns the reservation information of the user to the user, and the user checks in the reserved hotels according to the reservation information of the platform. Finally, the user completes the check-in, confirms the completion of the order and evaluates. The user can send the service evaluation to the OTA, and the OTA can also feed back the service evaluation to the corresponding cooperation hotel. The foregoing internet platform may be embodied as a single device or multiple devices (may be a computer device such as a terminal device or a service server), and the service server is used as an execution body in the auditing method of service evaluation provided in the embodiments of the present application for illustration, which is not described in detail below.

The auditing method of the service evaluation provided by the embodiment of the invention can automatically audit and classify the service evaluation of a certain service business (for convenience of description, the service evaluation can be a target service business), classify the service evaluation based on audit categories (audit pass or fail, good or bad evaluation, positive evaluation or negative evaluation and the like) so as to determine the audit categories (such as audit pass or fail, good or bad evaluation, positive evaluation or negative evaluation and the like) of the service evaluation, and extract improvement comments of the corresponding service business based on the service evaluation belonging to audit fail, bad evaluation or negative evaluation. Here, the service evaluation may be a service evaluation obtained by a user performing a comment, a core sharing, a complaint, or the like on a service provided by a service provider in a service industry such as a hotel, a restaurant, a travel, and an e-commerce, and the service evaluation may include contents in various expressions such as text, a picture, and a voice. When the service evaluation is a picture or the service evaluation includes a picture, a service evaluation text of the service evaluation may be obtained from the picture based on image text recognition. When the service rating is voice or voice is included in the service rating, a service rating text of the service rating may be obtained from the picture based on voice text recognition. For convenience of description, the auditing and classifying of the service evaluation text will be exemplified below, and will not be described in detail.

In the embodiment of the application, before classifying the service evaluation text of the service evaluation, the health degree detection can be performed to screen the service evaluation text containing the target health keywords (such as the elegant vocabulary or the industry sensitive vocabulary and the like) from the received service evaluation text, so that the auditing and classifying efficiency of the service evaluation text can be improved. In addition, the auditing method of the service evaluation provided by the embodiment of the application can audit and classify the service evaluation text through the text classification model, and can solve the problems of large artificial consumption, low auditing efficiency, strong subjectivity of auditing results and the like in manual auditing. In the auditing method of service evaluation provided by the embodiment of the application, before auditing and classifying the service evaluation text through the text classification model, keyword extraction, vectorization processing, partial vector extraction and other processing can be performed on the service evaluation text so as to reduce redundant components in the text, and emotion features constructed based on an emotion word stock of a target service are added in feature elements input into the text classification model. The text classification model in the business server can output the auditing category of the service evaluation text based on the input word vector characteristics and emotion characteristics, and has the advantages of simple operation, high auditing efficiency, strong auditing result objectivity and strong applicability.

Referring to fig. 2, fig. 2 is a schematic diagram of a system architecture according to an embodiment of the present application. As shown in fig. 2, the system architecture may include a service server 100 and a terminal cluster, where the terminal cluster may include: terminal device 200a, terminal device 200b, terminal devices 200c, … …, terminal device 200n, and the like. The service server 100 may establish communication connection with each terminal device in the terminal cluster, and may also establish communication connection between each terminal device in the terminal cluster. In other words, the service server 100 may establish a communication connection with each of the

terminal apparatuses

200a, 200b, 200c, … …, 200n, for example, a communication connection may be established between the terminal apparatus 200a and the service server 100. A communication connection may be established between terminal device 200a and terminal device 200b, and a communication connection may also be established between terminal device 200a and terminal device 200 c. The communication connection is not limited to a connection manner, and may be directly or indirectly connected through a wired communication manner, or may be directly or indirectly connected through a wireless communication manner, and the like, and may be specifically determined according to an actual application scenario, which is not limited herein.

It should be understood that each terminal device in the terminal cluster shown in fig. 2 may be provided with an application client, and when the application client runs in each terminal device, data interaction may be performed between the application client and the service server 100 shown in fig. 2, so that the service server 100 may receive service data from each terminal device. The application client may be an application client corresponding to the target service (may be a service such as hotel, restaurant, travel, e-commerce, etc.), that is, the user may subscribe to and use related products and/or services from a service provider of the target service through the application client, and after the use of the products or services ends, the user may send a service evaluation text to the service provider through the application client according to the experience of the products and/or services of the service provider. The service server 100 serves as a service provider server, and may receive service evaluation text issued by a user through an application client and display the received service evaluation text to other users to provide a selection reference for the user, or extract effective information for improving service quality of the service provider based on the received service evaluation text.

Taking hotel services as an example, each terminal device (i.e., terminal device 200a, terminal device 200b, terminal device 200c, … …, terminal device 200 n) in the terminal cluster shown in fig. 2 may install an application client related to the hotel services, through which a user may send service data about a reservation and use of related products and/or services to the service server 100, and after the use of the products or services ends, the user may send service evaluation text to the service server 100 through the application client according to the experience of the products and/or services of the service provider. It may be understood that the service evaluation text obtained by the service server 100 may be an internet platform, and the internet platform may be an OTA, that is, the OTA may be embodied as a terminal cluster shown in fig. 2 (may also include other service servers supporting the operation of the platform, etc.), and each terminal device may be installed with an application client corresponding to the OTA (abbreviated as an OTA application client) for performing service data interaction with the service server 100. The service server 100 may receive the service evaluation text sent by the user through the OTA application client, and display the received service evaluation text to other users to provide a selection reference for the user, or perform auditing and classification based on the received service evaluation text, so as to extract improvement comments of the corresponding service based on the service evaluation text belonging to the auditing failing, poor rating or negative rating.

In the process of auditing and classifying the service server 100 based on the received service evaluation text, the auditing result can be confirmed by manual auditing aiming at part of the service evaluation text, so as to improve auditing efficiency and accuracy of the final auditing result. For example, in the above health degree detection process, for the service evaluation text including the target health keyword, the service evaluation text can be determined to belong to the failed, poor or negative evaluation from the language usage habit, so that the auditing result can be directly confirmed by the manual auditing mode for the service evaluation text (including the target health keyword), thereby reducing the number of the service evaluation texts to be audited and improving the auditing efficiency. In addition, for the service evaluation text which does not detect the target health keywords, the auditing result is obtained through the text classification model after relevant processing, so that the auditing efficiency of the service evaluation text can be improved, and the auditing manual workload is reduced. Meanwhile, in order to avoid the error of the auditing result caused by the misjudgment of the model, manual auditing can be carried out to correct the auditing result aiming at the service evaluation text of which the auditing result belongs to the auditing failure, poor evaluation or negative evaluation, so that the accuracy of the final auditing result is improved. The manual auditing process can be completed through an auditing platform (or called manual auditing platform), and the auditing platform is used for displaying the received service evaluation text to an auditor so that the auditor can determine the target auditing category (i.e. the final auditing result) of the service evaluation text or perform checking and/or correcting operation on the initial auditing category (i.e. the auditing result obtained by the text classification model) of the service evaluation text output by the model based on the received service evaluation text to obtain the target auditing category.

In some possible embodiments, the auditing platform may be located in a device where the text classification model is located (the service server 100), or may be located in another device separately (any one of the

terminal devices

200a, 200b, 200c, … …, and 200 n). The audit platform may be embodied as the above-mentioned OTA application client, and is adapted to enable an auditor to manually audit the received service evaluation text by using the OTA application client, perform a checking and/or correcting operation on an initial audit category of the service evaluation text, and send, through the audit platform, the target audit category and the service evaluation text to the service server 100. Optionally, the user and the auditor who use the OTA application client can be represented by different login accounts, that is, different login accounts have different use rights, and the login account corresponding to the auditor can have rights to view and/or correct the audit category of the service evaluation text. Alternatively, the above-mentioned auditing platform may also be an independent application client, where the application client only performs the operation of checking and/or correcting the auditing result of the service evaluation text for the auditor, and the embodiment of the present application will be described by using the auditing platform separately located in other terminal devices (such as the terminal device 200 n) and embodied as an OTA application client.

The method provided in the embodiment of the present application may be performed by the service server 100 shown in fig. 2, may be performed by a terminal device (such as any one of the terminal device 200a, the terminal devices 200b, … …, and the terminal device 200n shown in fig. 2), or may be performed by the terminal device and the service server together, which may specifically be determined according to an actual application scenario, and is not limited herein. In this embodiment of the present application, the service server 100 is used as a server of a service provider of a hotel service, after a product is used or a service is finished, a user may send a service evaluation text to the service server 100 through an OTA application client installed in the terminal device 200a, and the terminal device 200n is used as an audit platform (embodied as an OTA application client) to perform the description together as an example, which will not be described in detail below.

In some possible embodiments, the service server 100 receives the service evaluation text from the terminal device 200a, where the service evaluation text is written by the user a through the OTA application client in the terminal device 200a and is sent to the service server 100, and the service server 100 may detect whether the target health keyword is included in the service evaluation text based on the health word stock. If the service evaluation text hit includes one or more health keywords in the health word stock by comparison, it is determined that the service evaluation text includes a target health keyword in the health word stock (i.e., the one or more health keywords may be target health keywords at this time). And outputting part of the service evaluation texts (including the target health keywords) to an auditing platform in the terminal equipment 200n, and receiving target auditing categories of the service evaluation texts sent by the auditing platform in the equipment 200n, so that the number of the service evaluation texts to be audited is reduced, and the auditing efficiency is improved. If the service evaluation text does not include the target health keyword, the service server 100 may obtain the emotion feature and the word vector feature corresponding to the service evaluation text. The service server 100 may obtain the emotion feature of the service evaluation text based on the emotion word stock of the target service (such as hotel) and the service evaluation text. By adding emotion characteristics to audit the service evaluation text, the characteristic expression capability of the text classification model is enhanced, the accuracy of auditing the service evaluation text can be further improved, and the auditing erroneous judgment can be maximally reduced. And meanwhile, keyword extraction is carried out on the service evaluation text based on a target feature word extraction algorithm, and word vector features of the service evaluation text are generated based on the extracted keywords, so that redundant words in the service evaluation text can be reduced, meanwhile, training time and testing time of a model are reduced, and accuracy and auditing efficiency of text auditing are further improved. The service server 100 may input the emotion feature and the word vector feature into a text classification model, obtain an initial audit category of the service evaluation text based on the text classification model, and if the initial audit category is a first audit category (such as audit failed, poor or negative), output the service evaluation text to an audit platform in the terminal device 200n, where the audit platform performs accuracy judgment and/or correction on the initial audit category by a manual audit manner based on the received service evaluation text and the initial audit category corresponding thereto, and returns to the service server 100 target audit category (corrected initial audit category). If the initial audit category is the second audit category (audit pass, good or positive rating), the audit platform determines the initial audit category as the target audit category of the service rating text and returns to the service server 100. The auditing platform carries out manual auditing on the service evaluation text of which the auditing result belongs to auditing failure, poor evaluation or negative evaluation, and can correct the error auditing result caused by misjudgment of the model, thereby improving the accuracy of the final auditing result.

Referring to fig. 3, fig. 3 is a flow chart of an auditing method of service evaluation according to an embodiment of the present application. As shown in fig. 3, the method comprises the steps of:

s101, acquiring a service evaluation text of a target service business.

In some possible embodiments, the service server may obtain a service rating of the target service (such as a hotel), where the service rating may be written by a user of the target service (such as a hotel) according to the product and/or service experience of the service provider after the product is used or the service is finished, and send a service rating text to the service server through the corresponding application client after the writing. Namely, a hotel user can subscribe to a service provider of the hotel through the OTA application client and use related products or services, and after the service is finished, the service server can receive service evaluation issued by the user through the OTA application client. For example, the user a subscribes to the service provider of the hotel through the OTA application client and uses the related product or service, and after the service is finished, the service server may receive the service evaluation issued by the user a through the OTA application client as "the environment of the hotel is in dislike". "etc. In addition, the service valuations received by the business server include, but are not limited to, "this time the hotel in which it is resident is very satisfied-! First, the hotel surroundings are quite good. Secondly, the server attitudes of hotels are good. Finally, breakfast in hotels is generally tasty. "," how do someone recommend such a hotel? "Hotel's fees can be received, however the service is not satisfactory" and "this time the resident hotel is very satisfactory-! First, the hotel surroundings are quite good. Secondly, the server attitudes of hotels are good. However, breakfast in hotels is generally not palatable. The specific content and/or the expression form of the service evaluation may be determined according to the actual application scenario, and are not limited herein.

It can be understood that the service evaluation may be a service evaluation obtained by a user performing a comment, a core sharing, a complaint, or the like on a service provided by a service provider in service industries such as hotels, restaurants, tours, electronic commerce, and the like, and the service evaluation may include contents in various manifestations such as texts, pictures, voices, and the like. When the service evaluation is a picture or the service evaluation includes a picture, a service evaluation text of the service evaluation may be obtained from the picture based on image text recognition. When the service rating is voice or voice is included in the service rating, a service rating text of the service rating may be obtained from the picture based on voice text recognition. Auditing the service valuations may be equivalent to auditing the service valuation texts, i.e., the auditing class of the service valuations is equivalent to the auditing class of their corresponding service valuation texts. The auditing result of the service evaluation text obtained by auditing and classifying the service evaluation text by the service server will be described below.

S102, detecting whether the service evaluation text comprises the target health keywords, if yes, executing a step S103, otherwise, executing a step S104.

In some possible embodiments, after the service evaluation text is obtained, the service evaluation text may be segmented to obtain a plurality of independent words. By segmenting the service evaluation text to obtain a plurality of independent words, the service evaluation text can be conveniently compared with the health keywords included in the health word stock to determine whether the service evaluation text contains an elegant word (or a target health keyword). Specifically, for the acquired service evaluation text, a word segmentation tool is used for segmenting the service evaluation text so as to obtain a plurality of independent words corresponding to the service evaluation text. Here, the word segmentation tools include, but are not limited to jieba, THULAC, snowNLP, pynlpir, coreNLP and pyLTP, etc. For example, the acquired service evaluation text is "environment of hotel is especially praised". ", a plurality of independent words can be obtained through the jieba word segmentation tool, and the method can comprise the following steps: "hotel", "environment", "extra" and "like" multiple independent words. And obtaining independent words after word segmentation, and detecting the health degree based on the words in the service evaluation text.

In some possible embodiments, the multiple independent words obtained after word segmentation may be compared with the health keywords included in the health word stock, so as to detect whether the service evaluation text includes the target health keywords. Specifically, the health word library includes a plurality of preset health keywords, where the health keywords in the health word library may be related inelegant words (such as visceral phones, etc.), or may be industry sensitive words (such as money, etc.), and may be specifically determined according to an actual application scenario, which is not limited herein. Here, the health keywords included in the health word stock may be directly obtained from the related field of the internet, or directly obtained from the obtained service evaluation text, or the service provider may be customized, etc., which may be specifically determined according to the actual application scenario, and is not limited herein. The obtained health keywords can be put into a designated storage space of the service server to form the health word stock, and word sets in the health word stock can be added and deleted periodically to update the health keywords in the health word stock, wherein the frequency of periodically updating the health keywords in the health word stock can be once a week, once a month and the like, and the frequency can be specifically determined according to the actual application scene requirements without limitation.

In some possible embodiments, if the service evaluation text is found to contain the target health keyword through comparison, the service evaluation text is indicated to contain the offensive word, and the offensive word can be output to the auditing platform to determine the target auditing category of the service evaluation text. For example, the service server receives a service evaluation text, and the independent words obtained after the service evaluation text is segmented include: and the service evaluation text is output to an auditing platform to determine the target auditing category of the service evaluation text through manual auditing when the health degree detection of the service evaluation text is not passed after the word 1, the word 2 and the word 3 are compared with the health word stock and the word 2 is the target health keyword. The service evaluation texts containing the target health keywords are output to the auditing platform to confirm auditing results in a manual auditing mode, so that the number of partial service evaluation texts to be audited is reduced, and the auditing efficiency of the service evaluation texts is improved.

S103, determining the target audit category of the service evaluation text through the audit platform.

In some possible embodiments, if the service valuation text includes a target health keyword (e.g., an elegance vocabulary), the business server may output the service valuation text to the audit platform to determine a target audit category of the service valuation text, where the target audit category may be a final determined audit category of the service valuation text, such as a first audit category (i.e., if the service valuation text is represented as audit failed, bad, or negatively rated) or a second audit category (i.e., if the service valuation text is represented as audit passed, good, or positively rated). The auditing platform may be used to present service assessment text that includes target health keywords (or offensive words) for an auditor to audit to determine their target audit category. Here, the auditing platform may be an application client (such as an OTA application client) corresponding to a target service business (such as a hotel). Alternatively, the target audit category of the service evaluation text may be determined by means of manual audit, and the process may be completed by an auditor through an application client (such as an OTA application client) corresponding to the target service (such as a hotel). For example, the auditor B may perform OTA application client login through a designated terminal device using an auditor account dedicated to manual audit of service evaluation text, which is distinguished from a normal user account and may have authority to view and audit the received service evaluation text. After logging, the auditor B can check the received service evaluation text at the OTA application client, wherein the service evaluation text can comprise a target health keyword, and the auditor judges the target audit category to be a second audit category (the service evaluation text is not approved, poorly rated or negatively rated) based on the text and the audit category. After the auditing is finished, the service server can receive the service evaluation text and the target auditing category sent by the auditing platform. The service evaluation texts containing the target health keywords are output to the auditing platform to confirm auditing results in a manual auditing mode, so that the number of partial service evaluation texts to be audited is reduced, and the auditing efficiency of the service evaluation texts is improved.

S104, acquiring emotion features and word vector features corresponding to the service evaluation text, and inputting the emotion features and the word vector features into the text classification model.

In some possible embodiments, for service evaluation texts that do not include target health keywords (such as elegance words), emotion features and word vector features may be extracted from the service evaluation texts, and input into a text classification model to obtain their corresponding initial audit categories. For easy understanding, the emotion feature acquisition and the word vector feature acquisition of the service evaluation text will be described below, respectively:

and (3) emotion feature acquisition:

in some possible embodiments, the emotion feature of the service evaluation text can be obtained through the emotion word library of the target service (such as a hotel) and the service evaluation text, and the service evaluation text is divided into multiple clauses to facilitate the construction of the emotion feature. Specifically, the service evaluation text may be divided into a plurality of independent clauses according to kanji punctuation marks included in the service evaluation text, or the service evaluation text may be divided into a plurality of independent clauses according to target punctuation marks, where the target punctuation marks include periods, question marks, exclamation marks, and the like. The embodiment of the application will be described by taking the example of dividing the service evaluation text into a plurality of independent clauses according to the target punctuation marks. For example, the service server obtains the service evaluation text: "the hotel where this check-in is very satisfied-! First, the hotel surroundings are quite good. Secondly, the server attitudes of hotels are good. Finally, breakfast in hotels is generally tasty. "based on the service evaluation text, the service evaluation text can be divided into: "the hotel where this check-in is very satisfied-! First, the surrounding environment of the hotel is very elegant. "," secondly, the hotel's attendant is very specialized. "and" finally, breakfast for hotels is generally tasty. Four separate clauses of ".

In some possible embodiments, the word type, the word of each word type and the emotion weight value included in each clause can be determined through an emotion word library of a target service (such as hotel service), and the sentence pattern and the emotion weight value of each sentence pattern of each clause are determined. Specifically, the emotion word library may include a plurality of preset emotion words, or may include a degree adverb that modifies the emotion words, a question word included in a question sentence, a turning word included in a turning sentence, and the like. The various emotion words contained in the emotion word library can be directly obtained from a webpage page in the hotel field of the internet, can be obtained according to the opinion of the expert in the related field, or can be directly obtained from the obtained service evaluation text, can be specifically determined according to the actual application scene, and is not limited herein. The obtained words can be put into a designated storage space of the service server to form the emotion word library, and words in the emotion word library can be updated by adding and deleting word sets in the emotion word library at regular intervals, wherein the frequency of updating the emotion word library at regular intervals can be determined once a week, once a month and the like, and the frequency can be specifically determined according to the actual application scene requirements without limitation.

The emotion lexicon is shown in table 1 below:

TABLE 1

As shown in table 1, the emotion word stock may include an emotion word stock, a degree adverb stock, a query word stock, and a inflected word stock. The emotion word library can comprise active emotion words and passive emotion words, the active emotion words can comprise words such as comfort, happiness and the like, and the passive emotion words can comprise words such as disappointment, boredom and the like; the term library of the degree adverbs is further divided into terms types such as "More", "Very", "More", "Ish", "Instrufficiently" and "Inverse", and the like, and the terms types also have corresponding terms, wherein the terms of the "More" term type may include "extraordinary", "polar", "Very" term type may include "extraordinary", "More" term type may include "More", "override", "Ish" term type may include "some", "one point", "Instrufficiently" term type may include "opposite", "micro", "Inverse" term type may include "none", "other". The query word stock and the turning word stock respectively contain query words and turning words and corresponding words thereof, for example, the query words can comprise "how can be" and the turning words can comprise "but", "however".

In some possible embodiments, each word type in the emotion word library may include one or more words, and each word in each word type corresponds to one emotion weight, each different word may correspond to a different emotion weight, or each word in the same word type may correspond to the same emotion weight, which may be specifically determined according to an actual application scenario, and is not limited herein. In the embodiment of the application, the words of the same word type are taken as examples to correspond to the same emotion weight.

In some possible embodiments, on the basis of dividing the service evaluation text into a plurality of clauses, matching words included in each clause with positive emotion words and negative emotion words in the emotion word library to determine positive emotion words and/or negative emotion words included in each clause, and determining emotion weights corresponding to the positive emotion words and/or the negative emotion words included in each clause. For example, for the above-mentioned "this time the hotel that is resident is very satisfied-! First, the surrounding environment of the hotel is very elegant. "," secondly, the hotel's attendant is very specialized. "and" finally, breakfast for hotels is generally tasty. The four independent clauses of the phrase are matched with positive emotion words contained in each clause to be 'satisfied', 'elegant', 'professional' and 'delicious' respectively through the emotion word library, and emotion weights (such as 2) corresponding to the positive emotion words contained in each clause are determined.

In some possible embodiments, it may also be determined whether the clause includes a word in the level adverb library through the level adverb library in the level adverb library, so as to further update the emotion score of the clause based on the level adverb in the clause and the corresponding emotion weight. Optionally, if the degree adverbs included in the emotion word library are determined from each clause and the degree adverbs are located before the positive emotion words and/or the negative emotion words, the degree adverbs are the degree adverbs corresponding to the positive emotion words and/or the negative emotion words, that is, from the aspect of Chinese grammar, the degree adverbs can be determined to play a role in reinforcing the positive emotion words and/or the negative emotion words. For example, in "very satisfied," the degree adverb "very" is the degree adverb corresponding to the positive emotion word "satisfied. And then determining the emotion weight corresponding to the degree adverb to obtain the emotion weight corresponding to the words belonging to the degree adverb word stock in each clause. Specifically, for different word types in the degree adverb word stock, words in different word types have different emotion weights. For example, regarding the clause "this time the hotel that is resident is very satisfied-! The word "very" matching the word type "Most" in the clause-containing degree adverb word library determines the emotion weight (e.g., 5) corresponding to the degree adverb. Whereas for the clause "next," the attendant of the hotel is very specialized. The word "Very" matching the word type "Very" in the word stock of the clause containing the degree adverb is used for determining the emotion weight (such as 4) corresponding to the degree adverb. In addition, the environment of the hotel is poor for the clause. The word "Very" matching the word type "Very" in the clause-containing degree adverb word stock and the word "none" of the "Inverse" word type determine the emotion weights (e.g., 4 and-1) corresponding to the degree adverbs, respectively.

In some possible embodiments, on the basis of determining the word type, the word of each word type and the emotion weight value included in each clause through the emotion word library, the sentence pattern and the emotion weight value of each sentence pattern can be determined, so that emotion scores of each clause obtained based on the word type, the word of each word type and the emotion weight value included in each clause can be further updated based on the sentence pattern and the emotion weight value of each sentence, and corresponding emotion characteristics can be extracted from the service evaluation text more effectively. Specifically, if any clause in each clause contains an exclamation mark, and the clause contains words in one or more emotion word libraries of the emotion word library, that is, the exclamation mark contained in the clause has emotion degree reinforcing effect on the words in the emotion word library contained in the clause, the sentence pattern of the clause can be determined to be the exclamation sentence, and the emotion weight corresponding to the exclamation sentence can be determined. For example, regarding the clause "this time the hotel that is resident is very satisfied-! If the clause is detected to contain an exclamation mark and contains a positive emotion word satisfactory, determining that the sentence pattern of the clause is the exclamation mark and the emotion weight (such as 2) corresponding to the exclamation mark.

In some possible embodiments, if any clause in each clause contains a question mark and the clause contains one or more words in the query word stock, that is, the question mark contained in the clause has emotion degree reinforcing effect on the words in the query word stock contained in the clause, determining that the sentence pattern of the clause is a question and determining that the clause is an emotion weight corresponding to the question. For example, get clauses from service valuations "how somebody recommends such hotels? If the clause is detected to contain a question mark and a query word of "how", determining that the sentence pattern of the clause is a question sentence and exclamating the emotion weight (such as-2) corresponding to the sentence.

In some possible embodiments, if any clause in each clause includes one or more words in the word stock of the turning words, determining the sentence pattern of the clause as the turning sentence, and determining that the clause is the emotion weight corresponding to the turning sentence. For example, the clause "the expense of the hotel can be received from the service evaluation, however, the service is not satisfied," the clause is detected to contain the turning word "however", the sentence pattern of the clause is determined to be the turning sentence, and the emotion weight (such as 1.5) corresponding to the turning sentence.

In some possible embodiments, the emotion score of each clause may be calculated based on the positive emotion words and their corresponding emotion weights, the negative emotion words and their corresponding emotion weights, and/or the degree adverbs and their corresponding emotion weights, which are included in each clause, and the emotion score of each clause may be updated based on the sentence pattern and the emotion weights of each sentence pattern, and the emotion characteristics of the service evaluation text may be obtained according to the emotion scores of each clause. Specifically, the emotion score of each clause is calculated based on the positive emotion words and the emotion weights corresponding to the positive emotion words and/or the negative emotion words and the emotion weights corresponding to the positive emotion words and the negative emotion words, wherein the emotion score can comprise a positive emotion score and/or a negative emotion score. And then updating the emotion score of each clause based on the emotion weight value of the degree adverb contained in each clause, wherein the emotion weight value of the degree adverb is 1 when the degree adverb is not contained in each clause. After the emotion score of each clause is obtained, the sum of the positive emotion values, the sum of the negative emotion values, the positive emotion value average value, the negative emotion value average value and the emotion total value of all the clauses are obtained based on the emotion score of each clause. And constructing the emotion characteristics of the service evaluation text by one or more of the sum of positive emotion values, the sum of negative emotion values, the average of positive emotion values, the average of negative emotion values and the total emotion value. By adding emotion characteristics to audit the service evaluation text, the characteristic expression capability of the text classification model is enhanced, and the accuracy rate of auditing the service evaluation text can be further improved.

For example, the service evaluation text is acquired: "the hotel where this check-in is very satisfied-! First, the hotel surroundings are quite good. Secondly, the server attitudes of hotels are good. However, breakfast in hotels is generally not palatable. "based on the service evaluation text, the service evaluation text can be divided into: "the hotel where this check-in is very satisfied-! First, the surrounding environment of the hotel is very elegant. "," secondly, the hotel's attendant is very specialized. "and" but breakfast for hotels is generally unpalatable. Four separate clauses of ". Wherein, in clause "this time the hotel where the stay is very satisfied-! The "satisfaction with active emotion words" in the sentence, the emotion weight corresponding to the active emotion words may be 2, and the emotion weight corresponding to the corresponding degree adverb before the active emotion words may be 5, at this time, the active emotion words and the emotion weight corresponding to the degree adverb may be multiplied to obtain the active emotion score 10 of the clause. Meanwhile, the clause is an exclamation sentence, the emotion weight corresponding to the exclamation sentence can be 2, the positive emotion score of the clause is multiplied by the emotion weight corresponding to the exclamation sentence, the positive emotion score of the clause is updated to be 20, and the negative emotion score of the clause is 0. Similarly, "first, the hotel's surrounding environment is very elegant" can be calculated. "positive emotion score 10, negative emotion score 0; second, the hotel attendant is very specialized. "positive emotion score 8, negative emotion score 0; however, breakfast in hotels is generally unpalatable. "positive emotion score 0, negative emotion score-3. After the emotion score of each clause is obtained, the sum of the positive emotion values, the sum of the negative emotion values, the positive emotion value average value, the negative emotion value average value and the emotion total value of all the clauses in the evaluation text are respectively 38, -3, 9.5, -0.75 and 35 based on the emotion score of each clause. And selecting the sum of the positive emotion values, the sum of the negative emotion values, the average value of the positive emotion values, the average value of the negative emotion values and the total emotion value as the emotion characteristics of the service evaluation text. By adding emotion characteristics to audit the service evaluation text, the characteristic expression capability of the text classification model is enhanced, the accuracy of auditing the service evaluation text can be further improved, and the auditing erroneous judgment is reduced.

Word vector feature acquisition:

in some possible embodiments, keyword extraction may be performed on the service evaluation text based on the target feature word extraction algorithm, and word vector features of the service evaluation text may be generated based on the extracted keywords. Specifically, the service evaluation text may be segmented to obtain a plurality of independent words, and one or more keywords may be extracted from the plurality of independent words based on the target feature word extraction algorithm. And carrying out vectorization processing on the keywords through a word vector conversion model to obtain a keyword vector sequence, and obtaining word vector characteristics of the service evaluation text based on the keyword vector sequence.

In some possible embodiments, the service evaluation text is segmented using a segmentation tool, and the service evaluation text may be segmented using a segmentation tool jieba, THULAC, snowNLP, pynlpir, coreNLP and a pyLTP segmentation tool to obtain independent words, so that the independent words are extracted by keywords based on a target feature word extraction algorithm. The target feature word extraction algorithm may be a word frequency-inverse document frequency algorithm, which is a statistical method for evaluating the importance of a word to one of the documents in a document set or a corpus. The importance of a word increases proportionally with the number of times it appears in the file, but at the same time decreases inversely with the frequency with which it appears in the corpus. Therefore, by calculating each word in the word segmentation result of the service evaluation text and arranging the calculated word frequency-inverse document frequency values (or TF-IDF values) in a descending order, the words in the ranking result, which are positioned in front by a preset number, can be selected as keywords. Or, selecting words with scores higher than a preset value as keywords, and in the embodiment of the application, selecting the words with the preset number in front in the sorting result as keywords for description. Specifically, word frequency-inverse document frequency value=tf×idf=word frequency of a word in an article (total number of occurrences/article word number) ×log (total number of documents of corpus/document number of containing the word+1). For example, for the acquired service evaluation text, the plurality of independent words obtained after word segmentation are: "Sound insulation", "hygiene", "too", "praise", "cost performance", "ultra", "high", "worth" and "recommended". Calculating each word in the word segmentation result of the service evaluation text through a word frequency-inverse document frequency value calculation formula, and arranging the calculated word frequency-inverse document frequency values in a descending order, wherein the ordering result is as follows: "Sound insulation", "hygiene", "praise", "cost performance", "high", "worth", "recommended", "too", "super". Selecting the words of the 7 words which are positioned in the front in the sorting result as keywords, and obtaining the keywords comprises the following steps: "Sound insulation", "hygiene", "praise", "cost performance", "high", "worth" and "recommended".

In some possible embodiments, the target feature word extraction algorithm may be a chi-square test algorithm, where the chi-square test may be used to count the degree of deviation between the actual observed value and the theoretical inferred value of the sample, and the degree of deviation between the actual observed value and the theoretical inferred value determines the magnitude of the chi-square value. If the chi-square value is larger, the deviation degree of the chi-square value and the chi-square value is larger; conversely, the smaller the deviation between the two. And calculating the relevance of each word to the auditing result, wherein the bigger the relevance is, the more beneficial to classifying the text classification model, otherwise, the text classification model can be discarded as useless features. The correlation can be embodied by a chi-square value (or X2), and the chi-square value can be obtained by using a chi-square test calculation formula and is used for measuring the difference degree between an actual value and a theoretical value.

The chi-square value calculation formula is:

wherein A is an observed value, E is a theoretical value, k is the number of observed values, n is a frequency number, and p is a theoretical frequency.

For example, for the acquired service evaluation text, the plurality of independent words obtained after word segmentation are: "Sound insulation", "hygiene", "too", "praise", "cost performance", "ultra", "high", "worth" and "recommended". Calculating chi-square values of each word in word segmentation results of the service evaluation text, and arranging the calculated chi-square values in a descending order, wherein the ordering results are as follows: sound insulation, hygiene, cost performance, praise, desertification, high, recommendation, superb, too. Selecting the words of the 7 words which are positioned in the front in the sorting result as keywords, and obtaining the keywords comprises the following steps: sound insulation, hygiene, cost performance, praise, desertification, high, and recommendation.

In some possible embodiments, the keyword is vectorized through a word vector conversion model to obtain a keyword vector sequence, and word vector features of the service evaluation text are obtained based on the keyword vector sequence. Specifically, on the basis of obtaining a keyword through the target feature word extraction algorithm, the keyword is subjected to vectorization processing through a word vector conversion model (which may be one or more of a word2vec model, a CBOW model and a glove model), so as to obtain a vector sequence of the keyword. For the vector sequence of the key words, the spearman correlation coefficient (or the spearman correlation coefficient) between each vector in the vector sequence is calculated. The spearman correlation coefficient is a non-parametric indicator of the dependence of two variables, which evaluates the dependence of two statistical variables using a monotonic equation. If there are no duplicate values in the data, and when the two variables are perfectly monotonically correlated, the spearman correlation coefficient is either +1 or-1. If the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in the two vectors and reserving the other vector to obtain a keyword vector sequence after the vector is removed, and obtaining word vector characteristics based on the keyword vector sequence after the vector is removed.

The spearman correlation coefficient is often expressed by greek letter ρ, and the spearman correlation coefficient is calculated by assuming that there are vector X and vector Y:

for example, obtaining keywords includes: the method comprises the steps of carrying out vectorization processing on the keywords through a word vector conversion model to obtain a vector sequence of the keywords, enabling a word vector obtained after vectorization of the keywords in a sound insulation mode to be M1, and pushing the vector sequence of the keywords to obtain the vector sequence of the keywords, wherein the vector sequence of the keywords can be as follows: m1, M2, … …, M7. The target threshold of the spearman correlation coefficient is 0.6, the spearman correlation coefficient between any two vectors in the keyword vector sequence is calculated, the spearman correlation coefficient between the word vector M3 and the word vector M5 is found to be 0.8, and if the spearman correlation coefficient exceeds the target threshold, the word vector M3 is removed (or the word vector M3 is removed). And if the spearman correlation coefficient between any two vectors in the rest keyword vector sequences does not exceed the target threshold value by 0.6, the keyword vector sequences are as follows: m1, M2, M4, M5, M6, M7 constitute word vector features. By extracting keywords in the process of constructing the word vector features and further selecting the keyword vectors as final word vector features, redundant words in the text can be reduced, meanwhile, training time and testing time of the model are reduced, and accuracy and auditing efficiency of text auditing are further improved.

In some possible embodiments, the obtained emotion features and word vector features are input into a text classification model, and when classifying text data, various text classification models can be selected, such as TextCNN, textRNN and FastText, etc., and the text classification model based on the FastText algorithm is selected in the embodiment of the application. FastText is a fast text classification algorithm, and is characterized by simple models, so that the fast text classification algorithm has faster training speed and testing speed under the condition of high precision, and is several orders of magnitude faster than the training of depth models. The FastText comprises an input layer, a hidden layer and an output layer, wherein the emotion characteristics and the word vector characteristics are input into a text classification model based on the FastText algorithm through the input layer, and the classification result is obtained through the output layer. Specifically, the input layer has N-gram characteristics, namely, the text content performs window sliding operation with the size of N according to the sequence of subsections, and finally a byte fragment sequence with the window of N is formed. The gram in the n-gram has different meanings according to different granularity, namely, the granularity of a word can be realized, and the granularity of a word can be realized. Taking word granularity as an example, for the above keywords: the term combination obtained by adding the N-gram feature with the window N value of 2 is as follows: "Sound insulation", "hygiene", "praise", "cost performance", "high", "worth", "recommended", "sound insulation hygiene", "sanitary praise", "praise cost performance", "high worth" and "worth recommending". Further, the word combination can be used as a new keyword sequence, and the keyword sequence is vectorized to obtain a keyword vector sequence: m1, M2, … …, M13, the word vector features of the service valuation text can be obtained based on the keyword vector sequences (by calculating the Szechwan correlation coefficients between the vectors in the vector sequences and removing part of the keyword vectors). The word vector features (keyword vector sequences with part of the keyword vectors removed) and the emotion features are input into a hidden layer through an input layer, the hidden layer adds and averages the input vectors to obtain a new vector, and then the new vector is input into an output layer. The output layer adopts a hierarchical softmax method to convert the multi-classification problem into a plurality of two-classification problems, so that the computational complexity is reduced from O (V) to O (log V), namely, a Huffman tree is established according to each category, each category corresponds to one Huffman code, each Huffman tree node has a vector as a parameter to update, the hidden layer output and each Huffman node vector do point multiplication during prediction, which direction to move left and right is determined according to the result, and finally the hidden layer output and each Huffman node vector fall on a node corresponding to a certain category. Finally, the probability value of the service evaluation text as the second examination category (examination is passed, good or positive evaluation) can be obtained through the output layer, and the probability value range is 0-1. If the probability value exceeds the audit threshold value, the text classification model outputs the initial audit category of the service evaluation text as a second audit category, otherwise, the initial audit category of the output service evaluation text is a first audit category (audit failed, poor or negative). For example, if the audit threshold value is set to 0.9 and an output probability value of 0.96 is obtained by the text classification model (such as a text classification model based on the FastText algorithm) based on the word vector features (such as keyword vector sequences: M1, M2, … …, M13) and the emotion features (which may include the sum of positive emotion values, the sum of negative emotion values, the positive emotion value average, the negative emotion value average, and the emotion total), the initial audit category of the service evaluation text may be obtained as the second audit category.

In some possible embodiments, in the keyword extraction process of the service evaluation text through the target feature word extraction algorithm, before the service evaluation text is segmented to obtain a plurality of independent words, invalid elements in the service evaluation text may be removed, where the invalid elements may include invalid characters and invalid symbols. Specifically, the invalid words may be "invalid words", "having been" and "first", and the invalid symbol may be a Chinese and English punctuation symbol. For example, for the service evaluation text, "first, the surrounding environment of the hotel is very elegant. The method comprises the steps of removing invalid elements before word segmentation, obtaining updated service evaluation text which is quite elegant in hotel surrounding environment, and word segmentation is carried out on the service evaluation text to obtain a plurality of independent words: "hotel", "ambient", "environment", "ten", "elegant". By processing the invalid elements in the service evaluation text, redundant components in the text can be reduced, and extraction of keywords is facilitated.

Referring to fig. 4, fig. 4 is another flow chart of an auditing method of service evaluation according to an embodiment of the present application. As shown in fig. 4, in terms of word vector feature extraction, the service server first performs text processing on the service evaluation text, where the text processing may include removing invalid elements in the service evaluation text, where the invalid elements may include invalid words and invalid symbols. The business server then performs word segmentation on the service evaluation text to obtain a plurality of independent words, which can be accomplished using a jieba word segmentation tool. The keyword extraction can be performed on the words after word segmentation, namely, the TF-IDF value of each word in the word segmentation result can be calculated based on a word frequency-inverse document frequency algorithm, and then partial words are selected as keywords based on the TF-IDF value (the words with the preset number positioned in front in the sorting result can be selected after the TF-IDF values are arranged in a descending order, or the words with the scores higher than the preset value are selected as keywords). The chi-square value of each word in the word segmentation result can be calculated through the chi-square checking algorithm, and then partial words are selected as keywords based on the chi-square value. After the keyword extraction is completed, the service server performs word vector conversion and vector extraction on the keyword, and a word vector conversion model (which may be one or more of a word2vec model, a CBOW model and a glove model) is used for obtaining a vector sequence of the keyword. And calculating the spearman correlation coefficient (or the spearman correlation coefficient) between each vector in the vector sequence aiming at the vector sequence of the keyword. If the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in the two vectors and reserving the other vector to obtain a keyword vector sequence after the vector is removed, and obtaining word vector characteristics based on the keyword vector sequence after the vector is removed. The final word vector feature may be n word vectors of the word vector sequences M1, M2, … …, mn. The business server inputs the obtained word vector features and emotion features into a text classification model (which can be a text classification model based on Fasttext algorithm), and obtains the auditing result of the service evaluation based on the text classification model. The business server extracts keywords in the process of constructing the word vector features, and further selects the keyword vectors as final word vector features, so that redundant words in the text can be reduced, training time and testing time of the model are also reduced, and accuracy and auditing efficiency of text auditing are further improved. And by adding emotion characteristics to audit the service evaluation text, the characteristic expression capability of the text classification model is enhanced, and the accuracy rate of auditing the service evaluation text can be further improved.

S105, determining whether the initial audit category of the service evaluation text output by the text classification model is the first audit category, if so, executing the step S103, otherwise, executing the step S106.

In some possible embodiments, for service valuation texts that do not include target health keywords (or unqualified vocabulary), their corresponding initial audit categories may be derived by a text classification model (such as a text classification model based on the FastText algorithm), which may include a first audit category (i.e., representing the service valuation text as audit failed, poorly rated, or negatively rated) and a second audit category (i.e., representing the service valuation text as audit passed, poorly rated, or positively rated). For the service evaluation text with the initial audit category being the first audit category, the service evaluation text and the corresponding initial audit category thereof can be output to the audit platform to confirm the target audit category of the service evaluation text (obtained by correcting the initial audit category by the audit platform). By manually conducting secondary auditing on the service evaluation text and the initial auditing category thereof, the false auditing result caused by misjudgment of the model can be corrected, thereby improving the accuracy of the final auditing result.

The initial audit category is corrected based on an audit platform as shown in the following table 2, and table 2 is an audit structure correction table of service evaluation text:

TABLE 2

Service valuation text	Initial audit results	Correction of results by auditors
			The check-in experience is poor and will not come again.	First audit category	First audit category
Sound insulation and hygiene are too poor.	First audit category	First audit category
			And as a whole, is good.	First audit category	Second audit category

As shown in table 2, there are service evaluations from users including: the "check-in experience is poor and will not come again. "," sound insulation and hygiene are too poor. "and" as a whole. According to the service evaluations, the text classification model can respectively obtain a model audit result (or an initial audit category) which is a first audit category (audit is not passed, poor audit or negative audit), the evaluation is output to an audit platform, the initial audit category is corrected through the audit platform, and a feedback target audit category is obtained from the audit platform. Wherein the service evaluation "check-in experience is poor and will not come again. And sound insulation and hygiene are too poor. The auditing result (initial auditing category) is the same as the auditor correction result (target auditing category), and the model prediction is accurate. And the service valuation is good overall. The "audit result (initial audit category) is the first audit category, but the audit person correction result (target audit category) fed back from the audit platform is the second audit category, and the model prediction is wrong.

And S106, determining the initial audit category as a target audit category of the service evaluation text.

In some possible embodiments, the service server may obtain an initial audit category of the service evaluation text output by a text classification model (such as a text classification model based on the FastText algorithm), and if the initial audit category is a first audit category (audit failed, poor or negative), output the service evaluation text to an audit platform to determine a target audit category of the service evaluation text by the audit platform. If the initial audit category is the second audit category (audit pass, good or positive rating), the initial audit category is determined to be the target audit category of the service rating text. For example, the text "hotel surroundings are very elegant for service evaluation". And if the initial audit category of the service evaluation text output by the text classification model is the second audit category, directly determining the target audit category of the service evaluation text as the second audit category.

In some possible embodiments, for the service evaluation text of the second audit category determined by the target audit category, the service evaluation text may be output to an internet platform (OTA) corresponding to the target service (such as a hotel) for display, for example, the user may view the displayed service evaluation text through an OTA related user interaction page (such as a web interface, a mobile application client).

In some possible embodiments, sample service evaluation texts of at least two audit categories may be obtained from a service evaluation sample library of a target service (such as a hotel), where the at least two categories include a first audit category and a second audit category, and audit category labels of the sample service evaluation texts are included in the sample service evaluation texts of any audit category. It will be appreciated that the service valuation sample library may be a designated storage space located in a business server, or the service valuation sample library may be embodied as other devices independent of the business server. And inputting the sample service evaluation text into a text classification model, and learning the sample service evaluation text through the text classification model to acquire the capability of identifying the auditing category of any service evaluation text.

Referring to fig. 5, a schematic diagram of text classification model training and optimization is provided in an embodiment of the present application. As shown in fig. 5, in the initial stage of building a text classification model (such as a text classification model based on the FastText algorithm), the audit class labels of the sample service evaluation texts in the service evaluation sample library are mainly obtained through manual labeling, and the manual labeling process can be completed by an auditor. I.e., first communicate with auditors to determine the category of audit categories, which may include a first audit category (audit failed, bad or negative rating) and a second audit category (audit passed, good or positive rating). After the auditing category is determined, an auditor can audit the historical service evaluation text (or service evaluation text in a training set) used for model training, determine the auditing category of the historical service evaluation text and label the auditing category label for the historical service evaluation text, and input the sample service evaluation text with the auditing category label into an evaluation sample library for text classification model training. The sample service evaluation text with the audit category label at least comprises sample service evaluation texts of two audit categories (a first audit category and a second audit category), and the historical service evaluation text for model training can be obtained from a webpage of a relevant target service business in the Internet or directly obtained from a service provider of the target service business. Meanwhile, in the text classification model training process, the service evaluation text can be audited based on the text classification model obtained through training to obtain the audit category of the service evaluation text. The audit category obtained by the text classification model can be audited by an audit platform to obtain a target audit category, the target audit category is added to serve as an audit category label of the service evaluation text, the service evaluation text added with the audit category label can serve as a sample service evaluation text, and the sample service evaluation text is added to a service evaluation sample library for training of the text classification model, so that the capability of identifying the audit category of any service evaluation text is obtained. Specifically, in the training process of the text classification model, a model prediction file can be obtained based on an input sample service evaluation text, and the model prediction file can include a text feature file (can include emotion features and word vector features obtained based on the sample service evaluation text) and a text classification algorithm model prediction file. When the sample service evaluation text or the service evaluation text acquired in real time is subjected to auditing and classification, the sample service evaluation text can be input into a model, and an auditing result of the sample service evaluation text can be obtained by executing the model prediction file. Meanwhile, the optimal model prediction file can be selected by comparing the auditing junctions obtained when model prediction files obtained in different training periods are executed, so that a final text classification model is determined, and the method is used for auditing service evaluation texts obtained in real time, so that the accuracy of auditing service evaluation texts based on the text classification model is improved, and the method is simple to operate and high in applicability.

In some possible embodiments, in the text classification model training process, after the service evaluation sample library receives the new sample service evaluation text, the sample service evaluation text can be input into the text classification model for training the text classification model in real time. Or, part of sample service evaluation texts can be periodically extracted from the service evaluation sample library, the text classification model is input to perform periodic model update training, wherein the frequency of the periodic model update training can be once a week, once a month, etc., and the periodic model update training can be specifically determined according to the actual application scene requirements without limitation.

The model robustness can be enhanced by continuously updating the sample service evaluation text contained in the service evaluation sample library and iterating the text classification model in real time or periodically, and the model auditing result is more accurate. Since the text classification model is built, the auditing performance of the text classification model can be shown in the following table 3:

TABLE 3 Table 3

	Verification pass	Pass of audit is not passed
			Pass of actual audit	131103	60
Actual audit is not passed	28	15905

As shown in table 3, the text classification model includes 147096 pieces of audit service evaluation text, wherein audit pass service evaluation text 131131 pieces (actual audit pass 131103 pieces, actual audit fail 28 pieces), audit pass service evaluation text 15965 pieces (actual audit pass 60 pieces, actual audit fail 15906 pieces). It can be seen that, for all service evaluation texts subjected to model audit, the service evaluation texts predicted to be correct (audit passed and actual audit passed, audit failed and actual audit failed) account for 99.94% of the total audit service evaluation texts, i.e. the accuracy of the text classification model is 99.94%. For the service evaluation text passing the audit, the service evaluation text with correct prediction (the actual audit passing) accounts for 99.97% of the total audit passing service evaluation text, namely the accuracy rate of the text classification model is 99.97%. And for the service evaluation texts of the actually-passed audit category in the service evaluation texts, the proportion of the service evaluation texts of the part of the service evaluation texts audited as the passed audit category by the model to the service evaluation texts of the total actually-passed audit category is 99.95 percent, namely the recall rate of the text classification model is 99.97 percent. From the perspective of each model evaluation index, the text classification model is stable in performance, and the prediction result is accurate and reliable.

In the embodiment of the application, the service server may obtain service evaluation text, where the service evaluation text may be a user from a related service (e.g. hotel) and may be obtained from an application client of a related internet platform (e.g. an OTA application client). Based on the service evaluation text, firstly, word segmentation is carried out on the text to obtain a plurality of independent words, and the independent words obtained after word segmentation are compared with health keywords included in a health word stock. If the service evaluation text is found to contain the target health keywords in the health word stock through comparison, the service evaluation text is indicated to contain the offensive words, and the offensive words can be output to an auditing platform to determine the target auditing category of the service evaluation text. If the service evaluation text does not include the target health keyword, the service server can acquire the emotion feature and the word vector feature corresponding to the service evaluation text. For emotion characteristics, the service evaluation text can be divided into a plurality of clauses, then words included in each clause are matched with active emotion words and passive emotion words in an emotion word library to determine active emotion words and/or passive emotion words included in each clause, and emotion weights corresponding to the active emotion words and/or the passive emotion words included in each clause are determined. If the degree adverbs included in the emotion word library are determined from all clauses and the degree adverbs are positioned before the positive emotion words and/or the negative emotion words, determining emotion weights corresponding to the degree adverbs so as to obtain emotion weights corresponding to the words belonging to the degree adverb word library in all clauses. And then, if any clause in each clause contains an exclamation mark and the clause contains words in one or more emotion word libraries of the emotion word library, determining that the sentence pattern of the clause is the exclamation sentence and determining that the clause is the emotion weight corresponding to the exclamation sentence. If any clause in each clause contains a question mark and the clause contains one or more words in the query word stock, determining the sentence pattern of the clause as a question, and determining that the clause is the emotion weight corresponding to the question. If any clause in each clause contains one or more words in the turning word library, determining the sentence pattern of the clause as a turning sentence, and determining that the clause is the emotion weight corresponding to the turning sentence. And calculating emotion scores of the clauses based on the active emotion words and the emotion weights corresponding to the active emotion words and/or the passive emotion words and the emotion weights corresponding to the passive emotion words, wherein the emotion scores can comprise active emotion scores and/or passive emotion scores. And then updating the emotion score of each clause based on the emotion weight value of the degree adverb contained in each clause. After the emotion score of each clause is obtained, the sum of the positive emotion values, the sum of the negative emotion values, the positive emotion value average value, the negative emotion value average value and the emotion total value of all the clauses are obtained based on the emotion score of each clause. And constructing the emotion characteristics of the service evaluation text by one or more of the sum of positive emotion values, the sum of negative emotion values, the average of positive emotion values, the average of negative emotion values and the total emotion value. For word vector features, a word segmentation tool can be used for segmenting a service evaluation text to obtain a plurality of independent words, word frequency-inverse document values (or TF-IDF values) are calculated for each word in a word segmentation result, the calculated word frequency-inverse document frequency values are arranged in a descending order, and a preset number of words in the front of the ordering result are selected as keywords. In addition, the chi-square value is calculated for each word in the word segmentation result of the service evaluation text, the chi-square values obtained by calculation are arranged in a descending order, and the words in the front preset number in the sorting result are selected as keywords. And then, carrying out vectorization processing on the keywords through a word vector conversion model to obtain a vector sequence of the keywords. And aiming at the vector sequence of the keyword, calculating the spearman correlation coefficient between each vector in the vector sequence, if the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in the two vectors and reserving the other vector to obtain a keyword vector sequence after vector removal, and obtaining word vector characteristics based on the keyword vector sequence after vector removal. And finally, inputting the obtained emotion characteristics and word vector characteristics into a text classification model based on a Fasttext algorithm, wherein the business server can acquire an initial audit category of a service evaluation text output by the text classification model, and if the initial audit category is a first audit category (audit is not passed, poor or negative), outputting the service evaluation text to an audit platform so as to determine a target audit category of the service evaluation text through the audit platform. If the initial audit category is the second audit category (audit pass, good or positive rating), the initial audit category is determined to be the target audit category of the service rating text. Therefore, the emotion characteristics and the word vector characteristics are obtained from the service evaluation text, and are input into the text classification model based on the Fasttext algorithm, so that the auditing category corresponding to the service evaluation text can be automatically obtained, the problem of high manual auditing labor consumption is solved, the auditing efficiency is improved, and the applicability is higher.

The embodiment of the application further provides an auditing device for service evaluation, please refer to fig. 6, fig. 6 is a schematic structural diagram of the auditing device for service evaluation provided in the embodiment of the application, and in the embodiment of the application, the device may operate the following modules:

an acquisition module 61, configured to acquire a service evaluation text of a target service;

a health detection module 62, configured to detect whether the service evaluation text acquired by the acquisition module 61 includes a target health keyword;

a first auditing module 63, configured to output the service evaluation text to an auditing platform when the health detection module 62 detects that the service evaluation text includes the target health keyword, and determine a target auditing category of the service evaluation text through the auditing platform;

a feature generation module 64, configured to, when the health detection module 62 detects that the service evaluation text does not include the target health keyword, obtain an emotion feature and a word vector feature corresponding to the service evaluation text obtained by the obtaining module 61, and input the emotion feature and the word vector feature into a text classification model;

the second auditing module 65 is configured to output a target auditing category of the service evaluation text based on the initial auditing category of the service evaluation text output by the text classification model, or output the service evaluation text to the auditing platform, so as to output the target auditing category of the service evaluation text through the auditing platform.

In one possible design, the health detection module 62 is configured to:

the service evaluation text acquired by the acquisition module is segmented to obtain a plurality of independent words;

and comparing the independent words with the health keywords in the health word stock to detect whether the service evaluation text comprises the target health keywords in the health word stock.

In one possible design, the feature generation module 64 is configured to:

obtaining emotion characteristics of the service evaluation text based on the emotion word library of the target service and the service evaluation text obtained by the obtaining module;

and extracting keywords from the service evaluation text based on a target feature word extraction algorithm, and generating word vector features of the service evaluation text based on the extracted keywords.

In one possible design, the feature generation module 64 is configured to:

dividing the service evaluation text acquired by the acquisition module into a plurality of clauses;

determining the word types, the words of the word types and the emotion weights of the emotion word bases of the target service business, and determining the sentence patterns of the clauses and the emotion weights of the sentence patterns;

And obtaining the emotion characteristics of the service evaluation text based on the words and emotion weights of the word types and the sentence patterns of the clauses and the emotion weights of the sentence patterns.

In one possible design, the feature generation module 64 is configured to:

calculating emotion scores of the clauses based on the positive emotion words and the corresponding emotion weights thereof, the negative emotion words and the corresponding emotion weights thereof, and/or the target degree adverbs and the corresponding emotion weights thereof, which are contained in the clauses;

updating emotion scores of the clauses based on the sentence patterns of the clauses and emotion weights of the sentence patterns, and obtaining emotion characteristics of the service evaluation text based on the emotion scores of the clauses.

In one possible design, the feature generation module 64 is configured to:

obtaining the sum of positive emotion values, the sum of negative emotion values, the average value of the positive emotion values, the average value of the negative emotion values and the total emotion value of all clauses based on the emotion scores of all clauses;

and constructing the emotion characteristics of the service evaluation text based on one or more of the sum of positive emotion values, the sum of negative emotion values, the average of positive emotion values, the average of negative emotion values and the total emotion value.

In one possible design, the feature generation module 64 is configured to:

the service evaluation text acquired by the acquisition module is segmented to obtain a plurality of independent words, and one or more keywords are extracted from the plurality of independent words based on a target feature word extraction algorithm;

and carrying out vectorization processing on the one or more keywords through a word vector conversion model to obtain a keyword vector sequence, and obtaining word vector characteristics of the service evaluation text based on the keyword vector sequence.

In one possible design, the feature generation module 64 is configured to:

acquiring a spearman correlation coefficient between vectors in the keyword vector sequence;

if the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in the any two vectors and reserving the other vector to obtain the keyword vector sequence with the vectors removed;

and obtaining word vector characteristics based on the keyword vector sequence with the vector removed.

In one possible design, the second audit module 65 is configured to:

acquiring an initial audit category of the service evaluation text acquired by the acquisition module output by the text classification model;

If the initial audit category is a first audit category, outputting the service evaluation text to the audit platform;

if the initial audit category is a second audit category, determining the initial audit category as a target audit category of the service evaluation text;

the first audit category includes audit not passing, the second audit category includes audit passing, or the first audit category is poor, and the second audit category is good.

According to the embodiment corresponding to fig. 3, the implementation described in steps S101 to S106 in the auditing method of service evaluation shown in fig. 3 may be performed by each module of the apparatus shown in fig. 6. For example, in the above-described auditing method for service assessment shown in fig. 3, the implementation described in step S101 may be performed by the in-device acquisition module 61 shown in fig. 6, the implementation described in step S102 may be performed by the health detection module 62, the implementation described in step S103 may be performed by the first and

second auditing modules

63 and 65, the implementation described in step S104 may be performed by the feature generation module 64, and the implementation described in step S105 and step S106 may be performed by the second auditing module 65. The implementation manners performed by the acquiring module 61, the health detecting module 62, the first checking module 63, the feature generating module 64, and the second checking module 65 may be referred to the implementation manners provided by the steps in the embodiment corresponding to fig. 3, and are not described herein.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal provided in the embodiment of the present application includes a processor 701, a memory 702, a user interface 703, a communication interface 704, a coupler 705, an antenna 706, and other functional modules. The memory 702 is used for storing programs. In particular, the program may include program code including computer-operating instructions. Memory 702 includes, but is not limited to, RAM, ROM, EPROM, or CD-ROM, etc., without limitation. The memory 702 may be a memory in the processor 701, and is not limited thereto.

The memory 702 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:

operation instructions: including various operational instructions for carrying out various operations.

Operating system: including various system programs for implementing various basic services and handling hardware-based tasks.

The processor 701 controls the operation of the terminal, and the processor 701 may be one or more CPUs. The method of the terminal disclosed in the embodiment corresponding to fig. 3 above may be applied to the processor 701 or implemented by the processor 701. The processor 701 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 701 or by instructions in the form of software. The processor 701 described above may be a general purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 702, and the processor 701 reads information in the memory 702 and performs the method steps of the terminal described in connection with the embodiment of fig. 3 in connection with its hardware.

The user interface 703 of the terminal is mainly used for providing an input interface for a user and acquiring data input by the user. The user interface 703 may include, without limitation, a multimedia input and/or output device 7031, a camera 7032, a display 7033, and so forth. The user interface 703 may be an information input and/or output module for interacting with a user of the terminal, such as a microphone and/or a speaker of the terminal, a front and/or rear camera, a touch screen, etc., which are not limited herein. Optionally, the user interface 703 may also include a standard wired interface, a wireless interface, etc., without limitation.

The processor 701 of the terminal may be coupled to the antenna 706 through one or more communication interfaces 704 and a coupler 705, and the implementation performed by the terminal described in the embodiment of fig. 3 may be performed in conjunction with other functional modules, which may be specifically referred to as the implementation provided by the above embodiment, which is not limited herein. Herein, "coupled" means that two components are directly or indirectly joined to each other. Such a combination may be fixed or removable and may allow fluid, electrical signals or other types of signals to be communicated between the two components.

The embodiment of the present application provides a computer readable storage medium, in which instructions are stored, when the instructions run on a terminal, the instructions cause the terminal to execute an implementation manner executed by the terminal described in the embodiment of fig. 3, and specifically, reference may be made to an implementation manner provided in the foregoing embodiment, which is not described herein again.

The embodiments of the present application also provide a computer program product comprising instructions which, when run on a terminal device, cause the terminal device to perform the implementation performed by the terminal described in the embodiment of fig. 3 above.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method of auditing a service assessment, the method comprising:

acquiring a service evaluation text of a target service business, and detecting whether the service evaluation text comprises a target health keyword or not;

If the service evaluation text comprises the target health keywords, outputting the service evaluation text to an auditing platform, and determining a target auditing category of the service evaluation text through the auditing platform;

if the service evaluation text does not contain the target health keywords, acquiring emotion features and word vector features corresponding to the service evaluation text, and inputting the emotion features and the word vector features into a text classification model;

and determining a target audit category of the service evaluation text based on the initial audit category of the service evaluation text output by the text classification model or outputting the service evaluation text to the audit platform so as to determine the target audit category of the service evaluation text through the audit platform.

2. The method of claim 1, wherein the detecting whether the target health keyword is included in the service valuation text comprises:

word segmentation is carried out on the service evaluation text to obtain a plurality of independent words;

3. The method of claim 1 or 2, wherein the obtaining emotion features and word vector features of the service valuation text comprises:

acquiring emotion characteristics of the service evaluation text based on the emotion word library of the target service and the service evaluation text;

4. The method of claim 3, wherein the obtaining the emotion feature of the service evaluation text based on the emotion word library of the target service and the service evaluation text comprises:

dividing the service evaluation text into a plurality of clauses;

determining the word types, the words of the word types and the emotion weights of the words based on the emotion word library of the target service business, and determining the sentence patterns of the sentences and the emotion weights of the sentence patterns;

and obtaining the emotion characteristics of the service evaluation text based on the words and emotion weights of the word types in the clauses and the sentence patterns and emotion weights of the sentence patterns.

5. The method of claim 4, wherein the determining the word type included in each clause, the word of each word type, and the emotion weight based on the emotion word library of the target service comprises:

matching words included in each clause with emotion words in an emotion word library of the target service business to determine positive emotion words and/or negative emotion words included in each clause, and determining emotion weights corresponding to the positive emotion words and/or the negative emotion words included in each clause;

the emotion words in the emotion word library at least comprise two types of emotion words, the at least two types of emotion words at least comprise positive emotion words and negative emotion words, one type of emotion words comprises one or more words, and one type of emotion words corresponds to one emotion weight.

6. The method according to claim 5, wherein the emotion word library of the target service further includes a degree adverb, and after determining the positive emotion words and/or the negative emotion words included in each clause, the method further includes:

matching the words included in each clause with the degree adverbs in the emotion word bank of the target service business;

And if the target degree adverbs included in the emotion word library are determined from the clauses and the target degree adverbs are positioned before the positive emotion words and/or the negative emotion words, determining emotion weights corresponding to the target degree adverbs so as to obtain emotion weights corresponding to words with the word types of the degree adverbs in the clauses.

7. The method of claim 6, wherein determining the sentence patterns and emotion weights for each sentence pattern comprises:

detecting whether each clause comprises an exclamation mark or not by taking a sentence as a unit;

if any target clause in each clause contains an exclamation mark and the target clause contains one or more emotion words of the target service, determining that the sentence pattern of the target clause is the exclamation mark and determining that the target clause is an emotion weight corresponding to the exclamation mark.

8. The method of claim 6, wherein the emotion word library of the target service further comprises query words, and wherein determining the sentence patterns and emotion weights of the sentence patterns comprises:

detecting whether each clause comprises a question mark or not by taking a sentence as a unit, and determining whether each clause comprises a query word or not based on an emotion word stock of the target service business;

If any target clause in each clause comprises a question mark and the target clause comprises one or more query words, determining that the sentence pattern of the target clause is a question, and determining that the target clause is an emotion weight corresponding to the question.

9. The method of claim 6, wherein the emotion word library of the target service further comprises turning words, and the determining the sentence patterns and emotion weights of the sentence patterns comprises:

matching the words included in each clause with turning words in the emotion word bank of the target service business;

if any target clause in each clause contains one or more target turning words in the emotion word library of the target service, determining that the sentence pattern of the target clause is a turning sentence, and determining that the target clause is an emotion weight corresponding to the turning sentence.

10. The method according to any one of claims 7-9, wherein the obtaining emotion characteristics of the service valuation text based on the terms and emotion weights of the term types in the clauses, and the sentence patterns and emotion weights of the sentence patterns comprises:

Calculating emotion scores of the clauses based on the positive emotion words and the corresponding emotion weights thereof, the negative emotion words and the corresponding emotion weights thereof and/or the target degree adverbs and the corresponding emotion weights thereof;

11. The method of claim 10, wherein said calculating the emotion score for each clause based on the positive emotion words and their corresponding emotion weights, the negative emotion words and their corresponding emotion weights, and/or the target degree adverbs and their corresponding emotion weights included in each clause comprises:

calculating emotion scores of the clauses based on the positive emotion words and the emotion weights corresponding to the positive emotion words and/or the negative emotion words and the emotion weights corresponding to the positive emotion words and the negative emotion words contained in the clauses, wherein the emotion scores comprise positive emotion scores and/or negative emotion scores;

and updating the emotion score of each clause based on the emotion weight of the target degree adverb contained in each clause, wherein the emotion weight of the target degree adverb is 1 when the target degree adverb is not contained in each clause.

12. The method of claim 11, wherein the deriving emotional characteristics of the service valuation text based on the emotional scores of the clauses comprises:

13. The method of claim 3, wherein the keyword extraction of the service valuation text based on the target feature word extraction algorithm and generating word vector features of the service valuation text based on the extracted keywords comprises:

segmenting the service evaluation text to obtain a plurality of independent words, and extracting one or more keywords from the plurality of independent words based on a target feature word extraction algorithm;

14. The method of claim 13, wherein the target feature word extraction algorithm is a word frequency-inverse document frequency algorithm; the extracting one or more keywords from the plurality of independent words based on the target feature word extraction algorithm comprises:

calculating word frequency-inverse document frequency values of each word of the plurality of independent words by using a word frequency-inverse document frequency algorithm;

the words are ordered in a descending order according to the corresponding word frequency-inverse document frequency values, and the first N words in the ordering result are determined to be keywords, wherein N is a positive integer not greater than the total number of the independent words;

or selecting one or more words with the word frequency-inverse document frequency value not smaller than a preset value from the plurality of independent words as keywords.

15. The method of claim 13, wherein the target feature word extraction algorithm is a chi-square test algorithm; the extracting one or more keywords from the plurality of independent words based on the target feature word extraction algorithm comprises:

obtaining the chi-square value of each word in the plurality of independent words through a chi-square test calculation formula, and sorting the plurality of independent words in a descending order according to the chi-square value of each word;

And selecting the first M words in the sorting result as key words, wherein M is a positive integer not greater than the total number of the independent words.

16. The method of any of claims 13-15, wherein the deriving word vector features of the service valuation text based on the keyword vector sequence comprises:

if the spearman correlation coefficient between any two vectors in the keyword vector sequence exceeds a target threshold, removing one vector in the any two vectors and reserving the other vector to obtain a keyword vector sequence with the vectors removed;

17. A method according to any one of claims 1-16, wherein the determining a target audit category of the service valuation text or outputting the service valuation text to the audit platform based on the initial audit category of the service valuation text output by the text classification model comprises:

acquiring an initial audit category of the service evaluation text output by the text classification model;

the first audit category includes audit failing, the second audit category includes audit failing, or the first audit category is a bad comment, and the second audit category is a good comment.

18. The method of claim 17, wherein the method further comprises:

acquiring sample service evaluation texts of at least two audit categories from a service evaluation sample library of the target service business, wherein the at least two categories comprise a first audit category and a second audit category, and the sample service evaluation text of any audit category comprises audit category labels of the sample service evaluation texts;

and inputting the sample service evaluation text into the text classification model, and learning the sample service evaluation text through the text classification model to acquire the capability of identifying the audit category of any service evaluation text.

19. The method of claim 18, wherein after determining the target audit category for the service valuation text, the method further comprises:

And adding the target audit category as a category label of the service evaluation text, and adding the service evaluation text and the category label thereof into the service evaluation sample library to update the service evaluation sample library.

20. An auditing apparatus for service evaluation, comprising:

the acquisition module is used for acquiring the service evaluation text of the target service business;

the health detection module is used for detecting whether the service evaluation text acquired by the acquisition module comprises a target health keyword or not;

the first auditing module is used for outputting the service evaluation text to an auditing platform when the health detection module detects that the service evaluation text comprises the target health keywords, and determining the target auditing category of the service evaluation text through the auditing platform;

the feature generation module is used for acquiring emotion features and word vector features corresponding to the service evaluation text acquired by the acquisition module when the health detection module detects that the service evaluation text does not contain the target health keywords, and inputting the emotion features and the word vector features into a text classification model;

The second auditing module is used for outputting the target auditing category of the service evaluating text or outputting the service evaluating text to the auditing platform based on the initial auditing category of the service evaluating text output by the text classification model so as to output the target auditing category of the service evaluating text through the auditing platform.

21. The apparatus of claim 20, wherein the health detection module is to:

22. The apparatus of claim 20 or 21, wherein the feature generation module is configured to:

acquiring emotion characteristics of the service evaluation text based on the emotion word library of the target service and the service evaluation text acquired by the acquisition module;

23. The apparatus of claim 22, wherein the feature generation module is configured to:

24. The apparatus of claim 23, wherein the feature generation module is configured to:

25. The apparatus of claim 24, wherein the feature generation module is configured to:

26. The apparatus of claim 22, wherein the feature generation module is configured to:

27. The apparatus of claim 26, wherein the feature generation module is configured to:

28. The apparatus of any one of claims 20-27, wherein the second audit module is configured to:

29. A terminal device, characterized in that the terminal device comprises: a processor, transceiver, and memory;

the processor and the transceiver are configured to couple to the memory, read and execute instructions in the memory, to implement the method of any one of claims 1-19.

30. A computer readable storage medium, characterized in that the computer readable storage medium stores therein program instructions, which when run, cause the method of any of claims 1-19 to be performed.