WO2020258904A1 - Method for determining quality check effectiveness, device, apparatus, and storage medium - Google Patents

Abstract

The present application discloses a method for determining quality check effectiveness, a device, an apparatus, and a storage medium, pertaining to the technical field of computers. The method comprises: acquiring a machine quality check result of a computer apparatus with respect to a target content object, the machine quality check result comprising a trigger segment detected by the computer apparatus and an untriggered rule, the trigger segment being a content segment in the target content object that satisfies a detection rule, and the untriggered rule being a detection rule that the target content object does not satisfy; acquiring a manual quality check result of a quality check employee with respect to the target content object, the manual quality check result comprising a trigger segment detected by the quality check employee and an untriggered rule; and comparing the machine quality check result and the manual quality check result, and determining, according to a comparison result, effectiveness of a quality check conducted by the computer apparatus on the content object. The technical solution provided by embodiments of the present application provides a method for evaluating the effectiveness of a quality check of a computer apparatus, thereby solving the problem of a lack of quality check effectiveness evaluation of computer apparatuses.

Description

Method, device, equipment and storage medium for determining quality inspection effect

Cross references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on June 28, 2019. The application number is 201910571707.0, and the application title is "Methods, Devices, Equipment and Storage Media for Determining Quality Inspection Effects". The citation is incorporated in the present invention.

Technical field

This application relates to the field of computer technology, in particular to methods, devices, equipment and storage media for determining quality inspection effects.

Background technique

Currently, in order to provide users with better services, many companies are equipped with customer service personnel who directly serve users. In order to ensure the quality of service, in actual applications, it is usually necessary to perform quality inspection on the dialogue between the customer service personnel and the user (that is, the content of the customer service dialogue).

In the prior art, in order to improve the efficiency of quality inspection, a computer device may perform quality inspection on the content of the customer service dialogue according to a preset inspection rule.

However, how to evaluate the effect of computer equipment quality inspection has become an urgent problem to be solved.

Summary of the invention

Based on this, it is necessary to provide a method, device, equipment and storage medium for determining the effect of quality inspection for the problem that the effect of quality inspection of computer equipment cannot be evaluated.

Obtain the machine quality inspection result of the target content object by the computer equipment. The machine quality inspection result includes the trigger segment and the untriggered rule detected by the computer equipment. The trigger segment is the content segment in the target content object that meets the checkout rule, and the untriggered rule is the target Check out rules that the content object does not satisfy;

Obtain the manual quality inspection result of the target content object by the quality inspector. The manual quality inspection result includes the triggered fragments and untriggered rules detected by the quality inspector;

Compare machine quality inspection results with manual quality inspection results, and determine the effect of computer equipment quality inspection on content objects based on the comparison results.

In an embodiment of the present application, comparing the machine quality inspection result with the manual quality inspection result includes:

Compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to get the number of correct trigger segments and the number of missed trigger segments;

Among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes the trigger segment not detected by the computer device but detected by the quality inspector.

In an embodiment of the present application, determining the effect of the computer device's quality inspection on the content object according to the comparison result includes:

Obtain the first sum of the number of correctly triggered segments and the number of missed trigger segments;

Obtain the first ratio of the number of correctly triggered segments to the first sum;

The first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.

In an embodiment of the present application, comparing the machine quality inspection result with the manual quality inspection result also includes:

Compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to obtain the number of falsely detected trigger segments. The falsely detected trigger segments include trigger segments detected by the computer equipment but not detected by the quality inspector;

The non-triggered rules detected by computer equipment and the non-triggered rules detected by quality inspectors are compared to obtain the number of correct non-triggered rules. Correct non-triggered rules include non-triggered rules detected by both computer equipment and quality inspectors.

In an embodiment of the present application, determining the effect of the computer device's quality inspection on the content object according to the comparison result includes:

Obtain the second sum of the number of correctly triggered fragments and the number of correct untriggered rules;

Obtain the third sum of the number of correctly triggered segments, the number of missed trigger segments, the number of falsely detected trigger segments, and the number of correct untriggered rules;

Obtain the second ratio of the second sum to the third sum;

The second ratio is determined as the accuracy rate, and the accuracy rate is used to indicate the probability that the computer device can correctly detect the triggered segment and the untriggered rule when performing quality inspection on the content object.

In the second aspect, a method for determining the effect of quality inspection is provided. The method includes:

Obtain the manual quality inspection result of the target content object by the quality inspector. The manual quality inspection result includes the trigger segment and the untriggered rule checked out by the quality inspector. The trigger segment is the content segment in the target content object that meets the checkout rule, and the rule is not triggered It is the checkout rule that the target content object does not satisfy;

The manual quality inspection result is sent to the server. The manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine the quality inspection effect of the computer equipment on the content object according to the comparison result. Among them, the machine The quality inspection results include the triggered segments and non-triggered rules detected by the computer equipment.

In an embodiment of the present application, obtaining the manual quality inspection result of the target content object by the quality inspector includes:

Receive quality inspection instructions triggered by quality inspectors for target content objects;

Obtain the manual quality inspection result of the target content object by the quality inspector according to the quality inspection instruction.

In an embodiment of the present application, the quality inspection instruction includes a segment checkout instruction, and receiving the quality inspection instruction triggered by the quality inspector for the target content object includes:

Receiving a segment detection instruction triggered by a quality inspector for the first trigger segment in the target content object;

The segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, receiving a segment detection instruction triggered by a quality inspector for the first trigger segment in the target content object includes:

Display the quality inspection interface for the target content object, the quality inspection interface includes a segment confirmation option corresponding to the first trigger segment;

When a trigger operation on the segment confirmation option is detected, the segment detection instruction is received.

In an embodiment of the present application, the quality inspection instruction includes a misinspection instruction, and receiving the quality inspection instruction triggered by the quality inspector for the target content object includes:

Receiving the mischeck instruction triggered by the quality inspector for the second trigger segment in the target content object;

Wherein, the mischeck instruction is used to indicate that the quality inspector has not detected the second trigger segment, and the second trigger segment is a trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, receiving the mischeck instruction triggered by the quality inspector for the second trigger segment in the target content object includes:

The quality inspection interface for the target content object is displayed, and the quality inspection interface includes a false inspection option corresponding to the second trigger segment;

When a trigger operation on the false detection option is detected, a false detection instruction is received.

In an embodiment of the present application, the method further includes:

After receiving the misdetection instruction, send the detection cancellation information to the server;

Among them, the cancellation checkout information is used to instruct the server to delete the second trigger segment from the checkout segment database after determining the effect of the computer equipment on the content object's quality inspection. The checkout segment database stores the content checkout of the target object by the computer equipment. Trigger fragments out.

In an embodiment of the present application, the quality inspection instruction includes a missed inspection instruction, and receiving the quality inspection instruction triggered by the quality inspector for the target content object includes:

Receive missed inspection instructions triggered by quality inspectors for target content objects;

Among them, the missed inspection instruction is used to instruct the quality inspector to detect the third trigger segment that is not detected by the computer equipment based on the target detection rule.

In an embodiment of the present application, receiving a missed inspection instruction triggered by a quality inspector for a target content object includes:

Display the quality inspection interface for the target content object, the quality inspection interface includes missed inspection options;

After detecting the trigger operation of the missed inspection option, obtain the third trigger segment marked by the quality inspector in the target content object, and obtain the target detection rule corresponding to the third trigger segment entered by the quality inspector;

The missed detection instruction is generated according to the third trigger segment and the target detection rule.

In a third aspect, a device for determining the effect of quality inspection is provided. The device includes:

The first acquisition module is used to acquire the machine quality inspection result of the target content object by the computer equipment. The machine quality inspection result includes the trigger segment and the untriggered rule detected by the computer equipment. The trigger segment is the content in the target content object that meets the checkout rule Fragment, the untriggered rule is the checkout rule that the target content object does not satisfy;

The second acquisition module is used to acquire the manual quality inspection result of the target content object by the quality inspector, and the manual quality inspection result includes the trigger segment and the untriggered rule detected by the quality inspector;

The comparison module is used to compare the machine quality inspection results and manual quality inspection results, and determine the effect of the computer equipment on the content object based on the comparison results.

In an embodiment of the present application, the comparison module is specifically used for:

Compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to get the number of correct trigger segments and the number of missed trigger segments;

Among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes the trigger segment not detected by the computer device but detected by the quality inspector.

In an embodiment of the present application, the comparison module is specifically used for:

Obtain the first sum of the number of correctly triggered segments and the number of missed trigger segments;

Obtain the first ratio of the number of correctly triggered segments to the first sum;

The first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.

In an embodiment of the present application, the comparison module is also used for:

Compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to obtain the number of falsely detected trigger segments. The falsely detected trigger segments include trigger segments detected by the computer equipment but not detected by the quality inspector;

The non-triggered rules detected by computer equipment and the non-triggered rules detected by quality inspectors are compared to obtain the number of correct non-triggered rules. Correct non-triggered rules include non-triggered rules detected by both computer equipment and quality inspectors.

In an embodiment of the present application, the comparison module is specifically used for:

Obtain the second sum of the number of correctly triggered fragments and the number of correct untriggered rules;

Obtain the third sum of the number of correctly triggered segments, the number of missed trigger segments, the number of falsely detected trigger segments, and the number of correct untriggered rules;

Obtain the second ratio of the second sum to the third sum;

The second ratio is determined as the accuracy rate, and the accuracy rate is used to indicate the probability that the computer device can correctly detect the triggered segment and the untriggered rule when performing quality inspection on the content object.

In a fourth aspect, a device for determining the effect of quality inspection is provided, and the device includes:

The acquisition module is used to obtain the manual quality inspection result of the quality inspector on the target content object. The manual quality inspection result includes the trigger segment and the untriggered rule checked out by the quality inspector. The trigger segment is the content in the target content object that meets the checkout rule Fragment, the untriggered rule is the checkout rule that the target content object does not satisfy;

The first sending module is used to send the manual quality inspection result to the server. The manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine that the computer equipment performs quality inspection on the content object according to the comparison result. The effect of inspection, where the machine quality inspection result includes the triggered segments and untriggered rules detected by the computer equipment.

In an embodiment of the present application, the acquisition module includes:

The receiving sub-module is used to receive the quality inspection instruction triggered by the quality inspector for the target content object;

The acquiring sub-module is used to acquire the manual quality inspection result of the quality inspector on the target content object according to the quality inspection instruction.

In an embodiment of the present application, the quality inspection instruction includes a fragment detection instruction, and the receiving sub-module is specifically used for:

Receiving a segment detection instruction triggered by a quality inspector for the first trigger segment in the target content object;

The segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the receiving submodule is specifically used for:

Display the quality inspection interface for the target content object, the quality inspection interface includes a segment confirmation option corresponding to the first trigger segment;

When a trigger operation on the segment confirmation option is detected, the segment detection instruction is received.

In an embodiment of the present application, the quality inspection instruction includes an error inspection instruction, and the receiving sub-module is specifically used for:

Receiving the mischeck instruction triggered by the quality inspector for the second trigger segment in the target content object;

Wherein, the mischeck instruction is used to indicate that the quality inspector has not detected the second trigger segment, and the second trigger segment is a trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the receiving submodule is specifically used for:

The quality inspection interface for the target content object is displayed, and the quality inspection interface includes a false inspection option corresponding to the second trigger segment;

When a trigger operation on the false detection option is detected, a false detection instruction is received.

In an embodiment of the present application, the device further includes:

The second sending module is used to send the detection cancellation information to the server after receiving the misdetection instruction;

Among them, the cancellation checkout information is used to instruct the server to delete the second trigger segment from the checkout segment database after determining the effect of the computer equipment on the content object's quality inspection. The checkout segment database stores the content checkout of the target object by the computer equipment. Trigger fragments out.

In an embodiment of the present application, the quality inspection instruction includes a missed inspection instruction, and the receiving sub-module is specifically used for:

Receive missed inspection instructions triggered by quality inspectors for target content objects;

Among them, the missed inspection instruction is used to instruct the quality inspector to detect the third trigger segment that is not detected by the computer equipment based on the target detection rule.

In an embodiment of the present application, the receiving submodule is specifically used for:

Display the quality inspection interface for the target content object, the quality inspection interface includes missed inspection options;

After detecting the trigger operation of the missed inspection option, obtain the third trigger segment marked by the quality inspector in the target content object, and obtain the target detection rule corresponding to the third trigger segment entered by the quality inspector;

The missed detection instruction is generated according to the third trigger segment and the target detection rule.

In a fifth aspect, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of any one of the methods in the first aspect when the processor executes the computer program; or,

The processor implements the steps of any one of the methods in the second aspect when the processor executes the computer program.

In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned methods of the first aspect are implemented; or, the computer program is executed by the processor When implementing the steps of any one of the above-mentioned second aspects.

The beneficial effects brought by the technical solutions provided by the embodiments of the present application include at least:

Obtain the machine quality inspection results and manual quality inspection results of the computer equipment on the target content object. The machine quality inspection results include the trigger segments and non-triggered rules detected by the computer equipment, and the manual quality inspection results include the trigger segments detected by the quality inspector. After the rule is not triggered, the machine quality inspection result is compared with the manual quality inspection result, and the effect of the computer equipment quality inspection on the content object is determined according to the comparison result, thereby providing an evaluation of the quality inspection effect of the computer equipment The method solves the problem that the effect of quality inspection of computer equipment cannot be evaluated.

Description of the drawings

FIG. 1 is a schematic diagram of an implementation environment of a method for determining a quality inspection effect provided by an embodiment of the application;

2 is a flowchart of another method for determining the quality inspection effect provided by an embodiment of the application;

FIG. 3 is a flowchart of another method for determining the quality inspection effect provided by an embodiment of the application;

4 is a flowchart of another method for determining the quality inspection effect provided by an embodiment of the application;

FIG. 5 is a flowchart of another method for determining the quality inspection effect provided by an embodiment of the application;

FIG. 6 is a flowchart of another method for determining the quality inspection effect provided by an embodiment of the application;

FIG. 7 is a flowchart of another method for determining a quality inspection effect provided by an embodiment of the application;

FIG. 8 is a block diagram of a device for determining a quality inspection effect provided by an embodiment of the application;

FIG. 9 is a block diagram of another device for determining a quality inspection effect provided by an embodiment of the application;

FIG. 10 is a block diagram of another device for determining a quality inspection effect provided by an embodiment of the application;

FIG. 11 is a block diagram of a computer device provided by an embodiment of this application.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

At present, many companies are equipped with customer service personnel who directly provide services to users. Normally, customer service personnel can provide services to users by answering hotline calls, or provide services to users through network instant messaging, or through Other ways to provide services to users. In order to ensure the quality of service provided by customer service personnel to users, it is usually necessary to perform quality inspection on the content of the conversation between the customer service personnel and the user (hereinafter referred to as the content of the customer service conversation).

In the related technology, in order to improve the efficiency of quality inspection, computer equipment can perform quality inspection on the content of the customer service dialogue according to preset checkout rules. That is, in related technologies, the checkout rules can be preset by the technicians. Then, the computer equipment performs quality inspection on the content of the customer service dialogue according to the preset checkout rule.

For example, the check-out rule preset by the technician can be the rule that the conversation content is too fast. If a certain conversation content segment in the conversation content between the customer service staff and the user is found to meet the "conversation content too fast" ”Rule, the computer equipment can check out the conversation content segment in the conversation content based on the above checkout rule, thereby realizing the quality inspection of the customer service conversation content.

However, how to evaluate the effectiveness of computer equipment quality inspection has become an urgent problem to be solved. In view of this, the embodiment of the present application provides a method for determining the quality inspection effect. In the method for determining the quality inspection effect, the server can obtain the machine quality inspection result of the target content object (which may be the customer service dialogue content) of the computer equipment And manual quality inspection results, where the machine quality inspection results include the trigger segments and non-triggered rules detected by computer equipment, and the manual quality inspection results include the trigger segments and non-triggered rules detected by the quality inspector. Then, the server can check the machine The quality inspection result is compared with the manual quality inspection result, and according to the comparison result, the quality inspection effect of the computer equipment on the content object is determined, thereby providing a method for evaluating the quality inspection effect of the computer equipment, and solving the problem of the computer equipment. The effect of quality inspection is evaluated.

In the following, the implementation environment involved in the method for determining the quality inspection effect provided by the embodiments of the present application will be briefly described.

Please refer to FIG. 1, the implementation environment may include a server 101 and at least one quality inspection terminal 102 (only one quality inspection terminal 102 is exemplarily shown in FIG. 1). The server 101 and each quality inspection terminal 102 can be wired Or establish a communication connection wirelessly.

The quality inspection terminal 102 is a terminal operated by the quality inspector. The quality inspector can input his own quality inspection results of the content object (which can be the content of the customer service dialogue) into the quality inspection terminal 102. The quality inspection terminal 102 receives the quality inspection personnel After the input quality inspection result, the quality inspection result can be sent to the server 101 as a manual quality inspection result.

The server 101 stores the quality inspection results of the content objects by the computer equipment (that is, the machine quality inspection results). The server 101 can receive the manual quality inspection results sent by the quality inspection terminal 102, and the server 101 can check the machine quality inspection results and artificial quality inspection results. The inspection results are compared, and the effects of the computer equipment quality inspection are evaluated based on the comparison results.

In the embodiment of the present application, the quality inspection terminal 102 may be a desktop computer, a tablet computer, a smart phone, a wearable device, an MP3 device, or an MP4 device, etc. The server 101 may be one server, or may be composed of multiple servers Server cluster.

Please refer to FIG. 2, which shows a flowchart of a method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect may be applied to the server 101 in the implementation environment shown in FIG. 1. As shown in Figure 2, the method for determining the quality inspection effect may include the following steps:

Step 201: The server obtains the machine quality inspection result of the target content object by the computer equipment.

In the embodiments of this application, the "content object" refers to the object that the computer device performs quality inspection. For example, the "content object" can be the content of a customer service conversation. Optionally, the content of the customer service conversation can be audio, video, or text. Wait. By executing the method for determining the quality inspection effect provided by the embodiments of the present application, the server can determine the quality inspection effect of the computer equipment on the "content object".

As mentioned above, the server may store the machine quality inspection result. In step 201, the server may obtain the machine quality inspection result of the target content object from the machine quality inspection result stored in itself, where the target content object may be a specific Of multiple content objects. For example, the server may store the machine quality inspection results of 10,000 customer service conversation contents. In step 201, the server may select 100 customer service conversation contents from the 10,000 customer service conversation contents, and obtain the machines for the 100 customer service conversation contents Quality inspection results, among which, the content of the 100 customer service dialogues is the "target content object" mentioned above.

The machine quality inspection result of the target content object may include trigger segments and non-triggered rules detected by the computer equipment, where the trigger segment refers to the content segment in the target content object that meets the checkout rule, and the non-triggered rule refers to the target content object Unsatisfied check-out rules.

Taking the target content object as the customer service dialogue content DD, the check-out rules include the rules of excessive speaking speed and the rules of using uncivilized language as an example, if the customer service dialogue content DD includes the dialogue content that meets the "rules of excessive speaking speed" Fragment A. At the same time, the customer service dialogue content DD does not include any dialogue content fragments that meet the "rules of using uncivilized terms", then this dialogue content fragment A can be called a trigger fragment, and the rules that use uncivilized terms are called The rule is not triggered.

Step 202: The server obtains the manual quality inspection result of the target content object by the quality inspector.

In the embodiment of the present application, the server may receive the manual quality inspection result of the quality inspection personnel on the target content object sent by the quality inspection terminal, where the manual quality inspection result includes the trigger segment and the untriggered rule detected by the quality inspection personnel.

Step 203: The server compares the machine quality inspection result of the target object with the manual quality inspection result of the target object, and determines the effect of the computer equipment on the content object according to the comparison result.

In the embodiments of the present application, at least one of the detection rate and the accuracy rate may be used to characterize the effect of the computer equipment on the quality inspection of the content object. Among them, the detection rate is used to indicate the probability that the computer equipment can correctly detect the trigger segment when the content object is quality inspected, and the accuracy rate is used to indicate that the computer device can correctly detect the trigger segment and the untriggered rule when the content object is quality inspected. The probability.

In the method for determining the quality inspection effect provided by the embodiments of the present application, the machine quality inspection results and manual quality inspection results of the target content object are obtained by the computer equipment, where the machine quality inspection results include the trigger fragments and the failures detected by the computer equipment. Trigger rules. The manual quality inspection results include the triggered segments and non-triggered rules detected by the quality inspector. Then, the machine quality inspection results are compared with the manual quality inspection results, and the computer equipment determines the quality inspection of the content object based on the comparison results Therefore, it provides a method for evaluating the effect of computer equipment quality inspection, and solves the problem that the effect of computer equipment quality inspection cannot be evaluated.

Please refer to FIG. 3, which shows a flowchart of another method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect may be applied to the server 101 in the implementation environment shown in FIG. As shown in FIG. 3, based on the above-mentioned embodiment, the technical process of "the server compares the machine quality inspection result of the target content object with the manual quality inspection result of the target content object" in step 203 may include the following steps:

Step 301: The server compares the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to obtain the number of correct trigger segments and the number of missed trigger segments.

Among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector. Normally, the correct trigger segment can be called the true class (English: True Positive; abbreviation: TP), and the missed trigger segment includes the computer equipment Trigger segments that are not detected but are detected by quality inspectors, usually, trigger segments that are not detected can be called false negative (English: False Negative; abbreviation: FN).

Taking the target content object as the customer service dialogue content EE and the check-out rule as the rule of excessive speech rate as an example, if the computer equipment detects the dialogue content fragments (also That is, the trigger fragment) is the dialogue content fragment A and the dialogue content fragment B. The dialogue content fragment (that is, the trigger fragment) that meets the "rule of excessive speaking rate" detected by the quality inspector from the customer service dialogue content EE is the dialogue The content fragment A, the dialogue content fragment B, and the dialogue content fragment C, since the dialogue content fragment A and the dialogue content fragment B are trigger fragments detected by both the computer equipment and the quality inspector, the dialogue content fragment A and the dialogue content fragment B is the correct trigger segment. Since the dialog content segment C is a trigger segment detected by the quality inspector but not detected by the computer equipment, the dialog content segment C is the missed trigger segment.

Step 302: The server compares the trigger segment detected by the computer equipment with the trigger segment detected by the quality inspector to obtain the number of falsely detected trigger segments.

Among them, the false positive trigger segment includes the trigger segment detected by the computer equipment but not detected by the quality inspector. Generally, the false positive trigger segment can be called the false positive type (English: False Positive; abbreviated as FP).

Taking the target content object as the customer service dialogue content FF and the check-out rule as the rule of excessive speech rate as an example, if the computer equipment detects the dialogue content fragments (also That is the trigger fragment) is the dialogue content fragment A and the dialogue content fragment B. The dialogue content fragment (that is, the trigger fragment) that meets the "rule of excessive speaking rate" detected by the quality inspector from the customer service dialogue content FF is the dialogue The content segment A, because the dialogue content segment B is a trigger segment detected by the computer equipment but not detected by the quality inspector, therefore, the dialog content segment B is the false detection trigger segment.

Step 303: Compare the non-triggered rules detected by the computer equipment with the non-triggered rules detected by the quality inspector to obtain the number of correct non-triggered rules.

Among them, the correct non-triggered rules include non-triggered rules detected by both computer equipment and quality inspectors. Normally, the correct non-triggered rules can be referred to as true negative (English: True Negative; abbreviated: TN).

Taking the target content object as the customer service conversation content GG and the check-out rule as the rule of excessive speech rate as an example, if the computer device does not check out the conversation content fragments that satisfy the "rule of excessive speech rate" from the customer service conversation content GG, The quality inspector also did not check out the conversation content fragments that meet the "rule of excessive speech rate" from the customer service conversation content GG, and the "rule of excessive speech rate" is the correct and not triggered rule.

It should be pointed out that in one possible implementation manner, the server may only execute the technical process of step 301, and in another possible implementation manner, the server may execute the technical process of step 301 to step 303.

In the case where the server executes the technical process from step 301 to step 303, step 301 to step 303 can be executed one after another or simultaneously. In the case of successive execution, the embodiment of this application does not limit the execution of step 301 to step 303. Priority.

Please refer to FIG. 4, which shows a flowchart of another method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect may be applied to the server 101 in the implementation environment shown in FIG. As shown in Figure 4, on the basis of the above-mentioned embodiment, the technical process of "the server determines the quality inspection effect of the computer equipment on the content object" in step 203 may include the following steps:

Step 401: The server obtains the first sum of the number of correct trigger segments and the number of missed trigger segments.

Step 402: The server obtains the first ratio of the number of correctly triggered segments to the first sum value.

Step 403: The server determines the first ratio as the detection rate.

Among them, the technical process from step 401 to step 403 can be expressed in the following mathematical language:

Among them, δ is the detection rate, a is the number of correct trigger segments, and b is the number of missed trigger segments.

Please refer to FIG. 5, which shows a flowchart of another method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect may be applied to the server 101 in the implementation environment shown in FIG. As shown in FIG. 5, on the basis of the above-mentioned embodiment, the technical process of "the server determines the effect of the computer equipment on the content object according to the comparison result" in step 203 may include the following steps:

Step 501: The server obtains the second sum of the number of correctly triggered segments and the number of correct untriggered rules.

Step 502: The server obtains the third sum of the number of correctly triggered segments, the number of missed trigger segments, the number of falsely detected trigger segments, and the number of correct untriggered rules.

Step 503: The server obtains a second ratio between the second sum value and the third sum value.

Step 504: The server determines the second ratio as the accuracy rate.

Among them, the technical process from step 501 to step 504 can be expressed in the following mathematical language:

Among them, γ is the accuracy rate, a is the number of correctly triggered segments, b is the number of missed trigger segments, c is the number of correct untriggered rules, and d is the number of falsely detected trigger segments.

Please refer to FIG. 6, which shows a flowchart of a method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect can be applied to the quality inspection terminal 102 in the implementation environment shown in FIG. . As shown in Figure 6, the method for determining the quality inspection effect may include the following steps:

Step 601: The quality inspection terminal obtains the manual quality inspection result of the target content object by the quality inspector.

As mentioned above, the quality inspection terminal can receive the quality inspection result of the quality inspector on the target content object input by the quality inspector, and obtain the received quality inspection result as the manual quality inspection result of the quality inspector on the target content object .

Step 602: The quality inspection terminal sends the manual quality inspection result of the quality inspector on the target content object to the server.

Among them, the manual quality inspection result sent by the quality inspection terminal to the server can trigger the server to compare the manual quality inspection result of the target content object with the machine quality inspection result of the target content object, and determine the effect of the computer equipment on the content object according to the comparison result. .

Please refer to FIG. 7, which shows a flowchart of another method for determining a quality inspection effect provided by an embodiment of the present application. The method for determining a quality inspection effect can be applied to the quality inspection terminal 102 in the implementation environment shown in FIG. in. As shown in FIG. 7, based on the above-mentioned embodiment, the technical process of step 601 may include the following steps:

Step 701: The quality inspection terminal receives a quality inspection instruction triggered by the quality inspector for the target content object.

In an embodiment of the present application, the quality inspection instruction may include at least one of a segment detection instruction, a false inspection instruction, and a missed inspection instruction.

In the following, the embodiments of the present application will respectively briefly describe the technical processes of the quality inspection terminal receiving the segment detection instruction, the mis-inspection instruction, and the miss-inspection instruction.

First, the technical process of the quality inspection terminal receiving the fragment detection instruction.

In the embodiment of the present application, the quality inspection terminal may receive a segment checkout instruction triggered by the quality inspector for the first trigger segment in the target content object, where the first trigger segment refers to the computer device checking out the target content object For the trigger segment of the target content object, the segment detection instruction triggered by the first trigger segment is used to instruct the quality inspector to detect the first trigger segment.

Optionally, the quality inspection terminal may receive a segment detection instruction triggered by the quality inspector for the first trigger segment in the target content object through voice input, text input, and option triggering.

Taking the QC terminal as an example to receive the fragment checkout instruction triggered by the QC personnel for the first triggering fragment in the target content object by means of option triggering, the QC terminal can display the QC interface for the target content object, where the The quality inspection interface may include a segment confirmation option corresponding to the first trigger segment. When a trigger operation on the segment confirmation option is detected, the quality inspection terminal can receive the segment detection triggered by the first trigger segment in the target content object instruction.

For example, for the customer service conversation content TT, the computer device detects 3 trigger fragments from the customer service conversation content TT based on the checkout rule of "speaking too fast", and the 3 trigger fragments are the conversation content fragment A and the conversation content fragment respectively. B and conversation content fragment C. In the process of obtaining the manual quality inspection result of the customer service dialogue content TT, the quality inspection terminal can display the quality inspection interface of the customer service dialogue content TT. The quality inspection interface may include dialogue content fragment A, dialogue content fragment B, and dialogue content fragment C. At the same time, the quality inspection interface may also include checkout rules corresponding to dialogue content fragment A, dialogue content fragment B, and dialogue content fragment C. In addition, the quality inspection interface may also include dialogue content fragment A and dialogue content fragment B, respectively. Three segment confirmation options corresponding to the conversation content segment C. The quality inspector can listen to the customer service dialogue content TT, and in the process of listening, judge whether the dialogue content segment A, the dialogue content segment B, and the dialogue content segment C meet the checkout rule of "speaking too fast", if the dialogue content segment A If the checkout rule of "speaking too fast" is satisfied, the quality inspector can trigger the segment confirmation option corresponding to the conversation content fragment A. After the trigger operation of the quality inspector is detected, the quality inspection terminal can receive the content of the conversation The segment checkout instruction of segment A.

Second, the technical process of receiving misinspection instructions by the quality inspection terminal.

In the embodiment of the present application, the quality inspection terminal may receive the false detection instruction triggered by the quality inspector for the second trigger segment in the target content object, where the second trigger segment is the trigger segment detected by the computer device for the target content object , The false detection instruction is used to indicate that the quality inspector has not detected the second trigger segment.

In the same way as described above, the quality inspection terminal can also receive a false inspection instruction triggered by the quality inspector for the second trigger segment in the target content object through voice input, text input, and option triggering.

Taking the quality inspection terminal to receive the false inspection instruction triggered by the quality inspector for the second trigger segment in the target content object through the option trigger as an example, the quality inspection terminal can display the quality inspection interface for the target content object, where the quality inspection The inspection interface may include a false detection option corresponding to the second trigger segment. After detecting the trigger operation of the false inspection option, the quality inspection terminal can receive the false detection instruction triggered for the second trigger segment in the target content object .

As the above example, the quality inspection interface of the customer service dialogue content TT may also include 3 misdetection options corresponding to the dialogue content fragment A, the dialogue content fragment B, and the dialogue content fragment C respectively. When obtaining the manual quality inspection result of the customer service dialogue content TT During the process, the quality inspector can listen to the customer service dialogue content TT, and in the listening process, judge whether the dialogue content segment A, the dialogue content segment B, and the dialogue content segment C meet the check-out rule of “speaking too fast”. If the content segment B does not meet the detection rule of "speaking too fast", the quality inspector can trigger the false detection option corresponding to the conversation content segment B. After the trigger operation of the quality inspector is detected, the quality inspection terminal can receive A false detection instruction for the conversation content segment B.

In an embodiment of the present application, after receiving the false detection instruction triggered for the second trigger segment in the target content object, the quality inspection terminal may also send the checkout cancellation information to the server, where the checkout cancellation information It is used to instruct the server to delete the second trigger segment from the detected segment database after determining the effect of the computer equipment on the content object's quality inspection. The detected segment database stores the trigger segment detected by the computer device for the content of the target object. In this way, the accuracy of the machine quality inspection results stored in the server can be guaranteed.

Third, the technical process of the quality inspection terminal receiving missed inspection instructions.

In the embodiment of the present application, the quality inspection terminal may receive the missed inspection instruction triggered by the quality inspector for the target content object, where the missed inspection instruction is used to instruct the quality inspector to detect that the computer equipment is not detected based on the target detection rule The third trigger segment.

In the same way as described above, the quality inspection terminal can also receive the missed inspection instruction triggered by the quality inspector for the target content object through voice input, text input, and option triggering.

For example, the quality inspection terminal receives the missed inspection instruction triggered by the quality inspector for the target content object through the option trigger. The quality inspection terminal can display the quality inspection interface for the target content object, where the quality inspection interface includes the missed inspection option After detecting the trigger operation of the missed inspection option, the quality inspection terminal can obtain the third trigger segment marked by the quality inspector in the target content object, and obtain the target inspection input corresponding to the third trigger segment by the quality inspector. Then, the quality inspection terminal can generate a missed inspection instruction according to the third trigger segment and the target detection rule.

As the above example, the quality inspection interface of the customer service dialogue content TT may also include the missed inspection option. In the process of obtaining the manual quality inspection result of the customer service dialogue content TT, the quality inspector can listen to the customer service dialogue content TT, and in the process of listening Determine whether there are dialogue content fragments in the customer service dialogue content TT other than dialogue content fragment A, dialogue content fragment B, and dialogue content fragment C that meet the "speaking rate too fast" checkout rule. If they exist, the quality inspector can The missed inspection option is triggered. After the missed inspection option is triggered, the quality inspector can mark the dialogue content fragments (hereinafter referred to as dialogue content fragment D) that meet the "speaking rate too fast" detection rule in the customer service dialogue content TT, At the same time, the quality inspector can input the target detection rule corresponding to the dialogue content segment D. In this example, the target detection rule is the "speaking too fast" detection rule, and then the quality inspection terminal can be based on The dialogue content segment D and the detection rule of "speaking too fast" generate missed detection instructions.

Step 702: The quality inspection terminal obtains the manual quality inspection result of the target content object by the quality inspector according to the quality inspection instruction.

The quality inspection terminal can obtain the triggered segments and non-triggered rules detected by the quality inspector from the target content object based on the quality inspection instruction, thereby obtaining the manual quality inspection result.

Please refer to FIG. 8, which shows a block diagram of an apparatus 800 for determining a quality inspection effect provided by an embodiment of the present application. The apparatus 800 for determining a quality inspection effect may be configured in a server. As shown in FIG. 8, the device 800 for determining the quality inspection effect may include: a first acquiring module 801, a second acquiring module 802, and a comparing module 803.

Wherein, the first obtaining module 801 is used to obtain the machine quality inspection result of the target content object by the computer equipment. The machine quality inspection result includes the trigger segment and non-trigger rule detected by the computer equipment. The trigger segment is the target content object that meets the inspection result. The content fragment of the rule out, the untriggered rule is the check out rule that the target content object does not satisfy.

The second acquisition module 802 is used to acquire the manual quality inspection result of the target content object by the quality inspector, and the manual quality inspection result includes the trigger segment and the untriggered rule detected by the quality inspector.

The comparison module 803 is used to compare the machine quality inspection result and the manual quality inspection result, and determine the effect of the computer equipment on the content object according to the comparison result.

In an embodiment of the present application, the comparison module 803 is specifically used to: compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to obtain the number of correct trigger segments and the number of missed trigger segments Number; among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes the trigger segment that is not detected by the computer equipment but detected by the quality inspector.

In an embodiment of the present application, the comparison module 803 is specifically configured to: obtain the first sum value of the number of correct trigger segments and the number of missed trigger segments; obtain the number of correct trigger segments and the first sum value The first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.

In an embodiment of the present application, the comparison module 803 is also used to: compare the trigger segment detected by the computer equipment with the trigger segment detected by the quality inspector to obtain the number of falsely detected trigger segments, and the falsely detected trigger segments Including the trigger fragments detected by the computer equipment but not by the quality inspector; compare the non-triggered rules detected by the computer equipment and the non-triggered rules detected by the quality inspector to get the number of correct non-triggered rules, which are correct and not triggered The rules include untriggered rules detected by both computer equipment and quality inspectors.

In an embodiment of the present application, the comparison module 803 is specifically configured to: obtain the second sum of the number of correctly triggered segments and the number of correct untriggered rules; obtain the number of correctly triggered segments and missed trigger segments The third sum of the number of, the number of falsely detected trigger fragments, and the number of correct but not triggered rules; obtain the second ratio of the second sum to the third sum; determine the second ratio as the accuracy rate, accurate The rate is used to indicate the probability that the computer equipment can correctly detect the triggered segment and the non-triggered rule when performing quality inspection on the content object.

The device for determining the quality inspection effect provided by the embodiment of the present application can implement the foregoing method embodiment, and its implementation principles and technical effects are similar, and will not be repeated here.

For the specific definition of the device for determining the quality inspection effect, please refer to the above definition of the method for determining the quality inspection effect, which will not be repeated here. Each module in the above-mentioned device for determining the quality inspection effect can be implemented in whole or in part by software, hardware and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

Please refer to FIG. 9, which shows a block diagram of an apparatus 900 for determining a quality inspection effect provided by an embodiment of the present application. The apparatus 900 for determining a quality inspection effect may be configured in a quality inspection terminal. As shown in FIG. 9, the device 900 for determining the quality inspection effect may include: an acquiring module 901 and a first sending module 902.

Wherein, the acquisition module 901 is used to acquire the manual quality inspection result of the quality inspector on the target content object. The manual quality inspection result includes the trigger segment and the untriggered rule detected by the quality inspector. The trigger segment is the target content object that meets the inspection result. The content fragment of the rule out, the untriggered rule is the check out rule that the target content object does not satisfy.

The first sending module 902 is used to send the manual quality inspection result to the server. The manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine the computer equipment versus the content object according to the comparison result. The effect of the quality inspection, where the machine quality inspection result includes the triggered segments and untriggered rules detected by the computer equipment.

In an embodiment of the present application, the acquiring module 901 includes a receiving sub-module and an acquiring sub-module.

Wherein, the receiving sub-module is used to receive the quality inspection instruction triggered by the quality inspector for the target content object.

The acquiring sub-module is used to acquire the manual quality inspection result of the quality inspector on the target content object according to the quality inspection instruction.

In an embodiment of the present application, the quality inspection instruction includes a segment detection instruction, and the receiving sub-module is specifically configured to: receive the segment detection instruction triggered by the quality inspector for the first trigger segment in the target content object; wherein , The segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the receiving sub-module is specifically configured to: display a quality inspection interface for the target content object, the quality inspection interface including a segment confirmation option corresponding to the first trigger segment; when the segment confirmation option is detected After the trigger operation, receive the segment detection instruction.

In an embodiment of the present application, the quality inspection instruction includes a false inspection instruction, and the receiving sub-module is specifically configured to: receive the false inspection instruction triggered by the quality inspector for the second trigger segment in the target content object; The inspection instruction is used to indicate that the quality inspector has not detected the second trigger segment, and the second trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the receiving submodule is specifically configured to: display a quality inspection interface for the target content object, the quality inspection interface including a false detection option corresponding to the second trigger segment; when the false detection option is detected After the trigger operation, the error detection instruction is received.

In an embodiment of the present application, the quality inspection instruction includes a missed inspection instruction, and the receiving sub-module is specifically used to: receive the missed inspection instruction triggered by the quality inspector for the target content object; wherein the missed inspection instruction is used to instruct the quality inspection The personnel detects the third trigger segment that is not detected by the computer equipment based on the target detection rule.

In an embodiment of the present application, the receiving submodule is specifically used to: display a quality inspection interface for the target content object, the quality inspection interface includes a missed inspection option; when a trigger operation on the missed inspection option is detected, obtain the quality The inspector marks the third trigger segment in the target content object, and obtains the target detection rule corresponding to the third trigger segment input by the quality inspector; and generates a missed inspection instruction according to the third trigger segment and the target detection rule.

Please refer to FIG. 10, the embodiment of the present application also provides another device 1000 for determining the quality inspection effect. In addition to the modules included in the device for determining quality inspection effect 900, the device 1000 for determining the quality inspection effect is optional, The device 1000 for determining the quality inspection effect further includes a second sending module 903.

The second sending module 903 is configured to send the checkout cancellation information to the server after receiving the mischeck instruction; wherein the checkout cancellation information is used to instruct the server to check the effect of the computer equipment on the content object, The second trigger segment is deleted from the detected segment database, and the detected segment database stores the trigger segment detected by the computer device for the content of the target object.

The device for determining the quality inspection effect provided by the embodiment of the present application can implement the foregoing method embodiment, and its implementation principles and technical effects are similar, and will not be repeated here.

For the specific definition of the device for determining the quality inspection effect, please refer to the above definition of the method for determining the quality inspection effect, which will not be repeated here. Each module in the above-mentioned device for determining the quality inspection effect can be implemented in whole or in part by software, hardware and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

In an embodiment of the present application, a computer device is provided. The computer device may be a server or a quality inspection terminal, and its internal structure diagram may be as shown in FIG. 11. The computer device includes a processor and a memory connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The computer program is executed by the processor to realize a method for determining the quality inspection effect.

Those skilled in the art can understand that the structure shown in FIG. 11 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

In an embodiment of the present application, a computer device is provided. The computer device may be a server. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

Obtain the machine quality inspection result of the target content object by the computer equipment. The machine quality inspection result includes the trigger segment and the untriggered rule detected by the computer equipment. The trigger segment is the content segment in the target content object that meets the checkout rule, and the untriggered rule is the target Check out rules that the content object does not meet; obtain the manual quality inspection results of the target content object by the quality inspector. The manual quality inspection results include the trigger fragments and non-triggered rules detected by the quality inspector; compare the machine quality inspection results and manual quality inspections As a result, according to the comparison result, the effect of the quality inspection of the computer equipment on the content object is determined.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to obtain the number of correct trigger segments and missed detections The number of trigger segments; among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes the trigger segment not detected by the computer equipment but detected by the quality inspector.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: acquiring the first sum of the number of correctly triggered segments and the number of missed trigger segments; acquiring the number of correct trigger segments and the first A first ratio of the sum value; the first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: compare the trigger segment detected by the computer equipment with the trigger segment detected by the quality inspector to obtain the number of falsely detected trigger segments. The detection trigger segment includes the trigger segment detected by the computer equipment but not by the quality inspector; compare the non-triggered rules detected by the computer equipment and the non-triggered rules detected by the quality inspector to obtain the correct number of non-triggered rules. Correct non-triggered rules include non-triggered rules detected by both computer equipment and quality inspectors.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: acquiring the second sum of the number of correctly triggered segments and the number of correct untriggered rules; acquiring the number of correctly triggered segments and the missing The third sum of the number of detected trigger segments, the number of falsely detected trigger segments, and the number of correct and untriggered rules; obtain the second ratio of the second sum to the third sum; determine the second ratio as accurate The accuracy rate is used to indicate the probability that the computer equipment can correctly detect the triggered segment and the untriggered rule when the content object is inspected.

The implementation principles and technical effects of the computer equipment provided in the embodiments of the present application are similar to those of the foregoing method embodiments, and will not be repeated here.

In an embodiment of the present application, a computer device is provided. The computer device may be a quality inspection terminal. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following step:

Obtain the manual quality inspection result of the target content object by the quality inspector. The manual quality inspection result includes the trigger segment and the untriggered rule checked out by the quality inspector. The trigger segment is the content segment in the target content object that meets the checkout rule, and the rule is not triggered It is the detection rule that the target content object does not meet; the manual quality inspection result is sent to the server, and the manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine the computer equipment pair according to the comparison result. The effect of the content object's quality inspection, where the machine quality inspection result includes the triggered segments and untriggered rules detected by the computer equipment.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: receiving the quality inspection instruction triggered by the quality inspector for the target content object; obtaining the manual quality inspection of the quality inspector on the target content object according to the quality inspection instruction result.

In an embodiment of the present application, the quality inspection instruction includes a segment checkout instruction, and the processor further implements the following steps when executing the computer program: receiving the segment checkout instruction triggered by the quality inspector for the first trigger segment in the target content object ; Among them, the segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: displaying a quality inspection interface for the target content object, the quality inspection interface including a segment confirmation option corresponding to the first trigger segment; After confirming the trigger operation of the option, receive the fragment detection instruction.

In an embodiment of the present application, the quality inspection instruction includes a false inspection instruction, and the processor further implements the following steps when executing the computer program: receiving the false inspection instruction triggered by the quality inspector for the second trigger segment in the target content object; wherein , The false detection instruction is used to indicate that the quality inspector does not detect the second trigger segment, and the second trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: displaying a quality inspection interface for the target content object, the quality inspection interface includes a false inspection option corresponding to the second trigger segment; After the trigger operation of the check option, the mischeck instruction is received.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: after receiving the misdetection instruction, it sends the cancellation checkout information to the server; wherein the cancellation checkout information is used to instruct the server to determine the computer equipment After the effect of quality inspection on the content object, the second trigger segment is deleted from the detected segment database, and the detected segment database stores the trigger segment detected by the computer device for the content of the target object.

In an embodiment of the present application, the quality inspection instruction includes a missed inspection instruction, and the processor further implements the following steps when executing the computer program: receiving the missed inspection instruction triggered by the quality inspector for the target content object; wherein the missed inspection instruction is used to indicate The quality inspector detects the third trigger segment that is not detected by the computer equipment based on the target detection rule.

In an embodiment of the present application, the processor further implements the following steps when executing the computer program: displaying a quality inspection interface for the target content object, the quality inspection interface includes a missed inspection option; when a trigger operation on the missed inspection option is detected, The third trigger segment marked by the quality inspector in the target content object is acquired, and the target detection rule corresponding to the third trigger segment input by the quality inspector is acquired; a missed inspection instruction is generated according to the third trigger segment and the target detection rule.

The implementation principles and technical effects of the computer equipment provided in the embodiments of the present application are similar to those of the foregoing method embodiments, and will not be repeated here.

In an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtain the machine quality inspection result of the target content object by the computer equipment. The machine quality inspection result includes the trigger segment and the untriggered rule detected by the computer equipment. The trigger segment is the content segment in the target content object that meets the checkout rule, and the untriggered rule is the target Check out rules that the content object does not meet; obtain the manual quality inspection results of the target content object by the quality inspector. The manual quality inspection results include the trigger fragments and non-triggered rules detected by the quality inspector; compare the machine quality inspection results and manual quality inspections As a result, according to the comparison result, the effect of the quality inspection of the computer equipment on the content object is determined.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: the trigger segment detected by the computer equipment is compared with the trigger segment detected by the quality inspector, and the number of correct trigger segments and the misses are obtained. The number of detection trigger segments; among them, the correct trigger segment includes the trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes the trigger segment not detected by the computer equipment but detected by the quality inspector.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: obtaining the first sum of the number of correct trigger segments and the number of missed trigger segments; and obtaining the number of correct trigger segments and The first ratio of the first sum value; the first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are further implemented: the trigger segment detected by the computer equipment is compared with the trigger segment detected by the quality inspector to obtain the number of false trigger segments, The false detection trigger segment includes the trigger segment detected by the computer equipment but not by the quality inspector; compare the untriggered rule detected by the computer equipment and the untriggered rule detected by the quality inspector to get the number of correct untriggered rules , The correct non-triggered rules include non-triggered rules detected by both computer equipment and quality inspectors.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: obtaining the second sum of the number of correctly triggered segments and the number of correct non-triggered rules; obtaining the number of correctly triggered segments, The third sum of the number of missed trigger fragments, the number of false trigger fragments, and the number of correct rules not triggered; obtain the second ratio of the second sum to the third sum; determine the second ratio as Accuracy: The accuracy is used to indicate the probability that the computer equipment can correctly detect the triggered segment and the untriggered rule when the content object is inspected.

The implementation principles and technical effects of the computer-readable storage medium provided in this embodiment are similar to those of the foregoing method embodiments, and will not be repeated here.

In an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Obtain the manual quality inspection result of the target content object by the quality inspector. The manual quality inspection result includes the trigger segment and the untriggered rule checked out by the quality inspector. The trigger segment is the content segment in the target content object that meets the checkout rule, and the rule is not triggered It is the detection rule that the target content object does not meet; the manual quality inspection result is sent to the server, and the manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine the computer equipment pair according to the comparison result. The effect of the content object's quality inspection, where the machine quality inspection result includes the triggered segments and untriggered rules detected by the computer equipment.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: receiving the quality inspection instruction triggered by the quality inspector for the target content object; obtaining the quality inspector’s artificial quality of the target content object according to the quality inspection instruction Inspection result.

In an embodiment of the present application, the quality inspection instruction includes a segment checkout instruction, and the computer program is executed by the processor to further implement the following steps: receive the segment checkout triggered by the quality inspector for the first trigger segment in the target content object Instruction; where the segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is the trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: display the quality inspection interface for the target content object, the quality inspection interface includes a segment confirmation option corresponding to the first trigger segment; After the trigger operation of the segment confirmation option, the segment detection instruction is received.

In an embodiment of the present application, the quality inspection instruction includes a false inspection instruction, and when the computer program is executed by the processor, the following steps are further implemented: receiving the false inspection instruction triggered by the quality inspector for the second trigger segment in the target content object; Wherein, the mischeck instruction is used to indicate that the quality inspector has not detected the second trigger segment, and the second trigger segment is a trigger segment detected by the computer device for the target content object.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: display the quality inspection interface for the target content object, the quality inspection interface includes a false detection option corresponding to the second trigger segment; After the false detection option is triggered, the false detection command is received.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: after receiving the misdetection instruction, send the cancellation checkout information to the server; wherein the cancellation checkout information is used to instruct the server to determine the computer After the device performs quality inspection on the content object, the second trigger segment is deleted from the detected segment database, and the detected segment database stores the trigger segment detected by the computer device for the content of the target object.

In an embodiment of the present application, the quality inspection instruction includes a missed inspection instruction, and when the computer program is executed by the processor, the following steps are also implemented: receiving the missed inspection instruction triggered by the quality inspector for the target content object; wherein the missed inspection instruction is used for Instruct the quality inspector to detect the third trigger segment that is not detected by the computer equipment based on the target detection rule.

In an embodiment of the present application, when the computer program is executed by the processor, the following steps are also implemented: display a quality inspection interface for the target content object, the quality inspection interface includes a missed inspection option; when a trigger operation on the missed inspection option is detected , Obtain the third trigger segment marked by the quality inspector in the target content object, and acquire the target detection rule corresponding to the third trigger segment input by the quality inspector; generate a missed inspection instruction according to the third trigger segment and the target detection rule .

The implementation principles and technical effects of the computer-readable storage medium provided in this embodiment are similar to those of the foregoing method embodiments, and will not be repeated here.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage. In the medium, when the computer program is executed, it may include the procedures of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, All should be considered as the scope of this specification.

The above-mentioned embodiments only express a few implementation modes of the present application, and their description is relatively specific and detailed, but they should not be understood as a limitation on the scope of the patent application. It should be pointed out that for ordinary technical quality inspectors in this field, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims (18)

1. A method for determining the effect of quality inspection, characterized in that the method includes:

   Obtain a machine quality inspection result of a target content object by a computer device, the machine quality inspection result including a trigger segment and an untriggered rule detected by the computer device, and the trigger segment is a content segment in the target content object that meets the checkout rule , The untriggered rule is a check-out rule that the target content object does not satisfy;

   Acquiring a manual quality inspection result of the target content object by a quality inspector, where the manual quality inspection result includes a trigger segment and an untriggered rule detected by the quality inspector;

   The machine quality inspection result is compared with the manual quality inspection result, and the effect of the computer equipment on the content object is determined according to the comparison result.
2. The method according to claim 1, wherein the comparing the machine quality inspection result with the manual quality inspection result comprises:

   Compare the trigger segments detected by the computer equipment with the trigger segments detected by the quality inspector to get the number of correct trigger segments and the number of missed trigger segments;

   Wherein, the correct trigger segment includes a trigger segment detected by both the computer equipment and the quality inspector, and the missed trigger segment includes a trigger segment not detected by the computer device but detected by the quality inspector.
3. The method according to claim 2, wherein the determining the effect of the computer equipment on the content object according to the comparison result comprises:

   Acquiring the first sum of the number of correct trigger segments and the number of missed trigger segments;

   Obtaining a first ratio of the number of correctly triggered segments to the first sum;

   The first ratio is determined as the detection rate, and the detection rate is used to indicate the probability that the computer device can correctly detect the trigger segment when performing quality inspection on the content object.
4. The method according to claim 2, wherein the comparing the machine quality inspection result with the manual quality inspection result further comprises:

   The trigger segment detected by the computer equipment is compared with the trigger segment detected by the quality inspector to obtain the number of falsely detected trigger segments. The falsely detected trigger segment includes the trigger segments detected by the computer equipment but not detected by the quality inspector. ；

   Comparing the untriggered rules detected by the computer equipment and the untriggered rules detected by the quality inspector, the number of correct untriggered rules is obtained. The correct untriggered rules include the untriggered rules detected by both the computer equipment and the quality inspector rule.
5. The method according to claim 4, wherein the determining the effect of the computer device on the content object according to the comparison result comprises:

   Acquiring the second sum value of the number of correctly triggered segments and the number of correct untriggered rules;

   Acquiring the third sum of the number of correct trigger segments, the number of missed trigger segments, the number of false trigger segments, and the number of correct untriggered rules;

   Acquiring a second ratio of the second sum value and the third sum value;

   The second ratio is determined as an accuracy rate, and the accuracy rate is used to indicate the probability that the computer device can correctly detect the triggered segment and the untriggered rule when performing quality inspection on the content object.
6. A method for determining the effect of quality inspection, characterized in that the method includes:

   Acquire the manual quality inspection result of the target content object by the quality inspector, the manual quality inspection result includes the trigger segment and the non-triggered rule checked out by the quality inspector, and the trigger segment is the target content object that meets the checkout rule Content fragments, the untriggered rule is a checkout rule that the target content object does not satisfy;

   The manual quality inspection result is sent to the server, and the manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and determine the computer equipment's response to the content according to the comparison result The effect of the subject's quality inspection, where the machine quality inspection result includes the trigger segment and the untriggered rule detected by the computer device.
7. The method according to claim 6, wherein said obtaining the manual quality inspection result of the target content object by the quality inspector comprises:

   Receiving a quality inspection instruction triggered by a quality inspector for the target content object;

   Obtain the manual quality inspection result of the target content object by the quality inspector according to the quality inspection instruction.
8. The method according to claim 7, wherein the quality inspection instruction comprises a segment checkout instruction, and the receiving the quality inspection instruction triggered by the quality inspector for the target content object comprises:

   Receiving a segment detection instruction triggered by a quality inspector for the first trigger segment in the target content object;

   The segment detection instruction is used to instruct the quality inspector to detect the first trigger segment, and the first trigger segment is a trigger segment detected by a computer device for the target content object.
9. 8. The method according to claim 8, wherein the receiving the segment checkout instruction triggered by the quality inspector for the first trigger segment in the target content object comprises:

   Displaying a quality inspection interface for the target content object, the quality inspection interface including a segment confirmation option corresponding to the first trigger segment;

   After detecting a trigger operation on the segment confirmation option, receiving the segment detection instruction.
10. 8. The method according to claim 7, wherein the quality inspection instruction comprises a false inspection instruction, and the receiving the quality inspection instruction triggered by the quality inspector for the target content object comprises:

    Receiving a mischeck instruction triggered by a quality inspector for the second trigger segment in the target content object;

    Wherein, the mischeck instruction is used to indicate that the quality inspector does not detect the second trigger segment, and the second trigger segment is a trigger segment detected by a computer device for the target content object.
11. The method according to claim 10, wherein the receiving the mischeck instruction triggered by the quality inspector for the second trigger segment in the target content object comprises:

    Displaying a quality inspection interface for the target content object, the quality inspection interface including a false inspection option corresponding to the second trigger segment;

    After detecting the triggering operation of the false detection option, the false detection instruction is received.
12. The method according to claim 10 or 11, wherein the method further comprises:

    After receiving the misdetection instruction, send detection cancellation information to the server;

    Wherein, the checkout cancellation information is used to instruct the server to delete the second trigger segment from the checkout segment database after determining the effect of the computer equipment on the content object's quality inspection, and the checkout segment database stores There is a trigger segment that a computer device detects for the content of the target object.
13. The method according to claim 7, wherein the quality inspection instruction comprises a missed inspection instruction, and the receiving the quality inspection instruction triggered by the quality inspector for the target content object comprises:

    Receiving a missed inspection instruction triggered by a quality inspector for the target content object;

    Wherein, the missed inspection instruction is used to instruct the quality inspector to detect a third trigger segment that is not detected by the computer equipment based on the target detection rule.
14. The method according to claim 13, wherein the receiving a missed inspection instruction triggered by a quality inspector for the target content object comprises:

    Displaying a quality inspection interface for the target content object, the quality inspection interface including missed inspection options;

    When the trigger operation on the missed inspection option is detected, the third trigger segment marked by the quality inspector in the target content object is acquired, and the third trigger segment input by the quality inspector corresponding to the third trigger segment is acquired The target detection rule;

    The missed detection instruction is generated according to the third trigger segment and the target detection rule.
15. A device for determining the effect of quality inspection, characterized in that the device comprises:

    The first acquisition module is used to acquire the machine quality inspection result of the target content object by the computer equipment, the machine quality inspection result includes the trigger segment and the non-trigger rule detected by the computer equipment, and the trigger segment is in the target content object Content fragments that meet the check-out rule, where the untriggered rule is a check-out rule that the target content object does not satisfy;

    The second acquiring module is configured to acquire the manual quality inspection result of the target content object by the quality inspector, and the manual quality inspection result includes the trigger segment and the untriggered rule detected by the quality inspector;

    The comparison module is used to compare the machine quality inspection result and the manual quality inspection result, and determine the effect of the computer equipment on the content object according to the comparison result.
16. A device for determining the effect of quality inspection, characterized in that the device comprises:

    The acquiring module is used to acquire the manual quality inspection result of the target content object by the quality inspector. The manual quality inspection result includes the trigger segment and the non-triggered rule detected by the quality inspector, and the trigger segment is in the target content object. Content fragments that meet the check-out rule, where the untriggered rule is a check-out rule that the target content object does not satisfy;

    The first sending module is configured to send the manual quality inspection result to the server, and the manual quality inspection result is used to trigger the server to compare the manual quality inspection result with the machine quality inspection result of the target content object, and according to The comparison result determines the effect of the computer equipment on the quality inspection of the content object, wherein the machine quality inspection result includes the trigger segment and the non-trigger rule detected by the computer equipment.
17. A computer device, comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when the computer program is executed; or ,

    The processor implements the steps of the method in any one of claims 6 to 14 when the computer program is executed.
18. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method according to any one of claims 1 to 5 when executed by a processor; or,

    When the computer program is executed by a processor, the steps of the method according to any one of claims 6 to 14 are realized.

Images