CN110610195A

CN110610195A - Answer obtaining method and device, electronic equipment and readable storage medium

Info

Publication number: CN110610195A
Application number: CN201910745422.4A
Authority: CN
Inventors: 方铭; 赵文倩
Original assignee: Beijing Sankuai Online Technology Co Ltd
Current assignee: Beijing Sankuai Online Technology Co Ltd
Priority date: 2019-08-13
Filing date: 2019-08-13
Publication date: 2019-12-24

Abstract

The embodiment of the disclosure provides an answer obtaining method and device, electronic equipment and a storage medium. The method comprises the following steps: receiving answers of a first number of questions and answers fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions; screening at least one target user meeting preset conditions from the multiple users according to answers of the detection questions fed back by the multiple users; determining the confidence of each option in each question to be verified according to the answer of the question to be verified fed back by the target user; and determining the target answer of each question to be verified according to the confidence. The method and the device can effectively identify the cheating user, and the obtained final adoption result is more real and accurate, flexible and understandable.

Description

Answer obtaining method and device, electronic equipment and readable storage medium

Technical Field

The embodiment of the disclosure relates to the technical field of internet, and in particular relates to an answer obtaining method, an answer obtaining device, an electronic device and a readable storage medium.

Background

In order to collect the use habits of some users, websites or merchants and the like usually provide a series of questions for the users to answer online to collect the information of the users, at present, most of the question providers convert the questions into choice questions for the users to answer online, for example, the question, "the merchant type of the merchant is?", the corresponding answer options are "yes", "no", "uncertain", next question "and the like.

However, in the above solution, the situation of user cheating is not considered, for example, some users are answers selected randomly, and the more cheating users, the lower the accuracy of the final adopted result.

Disclosure of Invention

The embodiment of the disclosure provides an answer obtaining method and device, an electronic device and a readable storage medium, which are used for identifying cheating users who answer and improving the accuracy of an adoption result corresponding to a question.

According to a first aspect of embodiments of the present disclosure, there is provided an answer obtaining method, including:

receiving answers of a first number of questions and answers fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions;

screening at least one target user meeting preset conditions from the multiple users according to answers of the detection questions fed back by the multiple users;

determining the confidence of each option in each question to be verified according to the answer of the question to be verified fed back by the target user; and

and determining the target answer of each question to be verified according to the confidence.

In a specific implementation of the embodiment of the present invention, before the step of receiving answers to the first number of questions and answers fed back by the plurality of users, the method further includes:

acquiring the problem to be verified and the detection problem;

mixing the detection questions into the questions to be verified to generate the question-answering questions; and

and pushing the question and answer questions to the plurality of users.

In an implementation of the embodiment of the present invention, the step of screening at least one target user meeting a preset condition from the plurality of users according to the answers of the detection questions fed back by the plurality of users includes:

aiming at a single user in the plurality of users, acquiring a fourth quantity of correct answers of the single user to the detection questions according to the answers of the detection questions fed back by the single user; and

and in the case that the fourth number is greater than a preset number threshold, determining that the single user is one of the target users.

In a specific implementation of the embodiment of the present invention, the step of determining a confidence level of each option in each question to be verified according to the answer to the question to be verified fed back by the target user includes:

acquiring a first number of people corresponding to the target user and a second number of people corresponding to the cheating user from the plurality of users; the cheating user refers to a user who does not meet the preset condition;

aiming at each question to be verified, acquiring a first option and at least one second option except the first option from a plurality of options corresponding to the question to be verified; and

determining the confidence level of the first option according to the second number, the third number, the first number, the second number, the first option, and the second option.

In a specific implementation of the embodiment of the present invention, the step of determining the confidence level of the first option according to the second number, the third number, the first number, the second number, the first option, and at least one of the second options includes:

calculating to obtain a first probability of the cheating user selecting the first option according to the number of options contained in the problem to be verified;

calculating to obtain a second probability that the cheating user passes the cheating prevention strategy according to the third quantity and the number of all options contained in the detection questions;

calculating the answer accuracy rate of the target user according to the second probability, the first number of people and the second number of people;

acquiring a third number of people selecting the first option and a fourth number of people selecting the second option from the target users; and

and calculating the confidence coefficient of the first option according to the first probability, the answer accuracy, the third number of people and the fourth number of people.

In a specific implementation of the embodiment of the present invention, the step of determining a target answer to each of the questions to be verified according to the confidence level includes:

comparing the confidence of the each option in the question to be verified to a confidence threshold; and

and determining the target answer of each question to be verified according to the comparison result.

In a specific implementation of the embodiment of the present invention, the step of determining the target answer of each question to be verified according to the comparison result includes:

acquiring a target option with the confidence degree larger than the confidence degree threshold value from each option of the problem to be verified; and

when the number of the target options is at least one, taking the target option with the maximum confidence as the target answer of the question to be verified; or when the number of the target options is zero, determining that the target answer of the question to be verified is a null value.

According to a second aspect of embodiments of the present disclosure, there is provided an answer obtaining apparatus including:

the system comprises an answer receiving module for receiving answers of a first number of questions fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions;

the target user screening module is used for screening at least one target user meeting preset conditions from the multiple users according to answers of the detection questions fed back by the multiple users;

the confidence coefficient determining module is used for determining the confidence coefficient of each option in each question to be verified according to the answer of the question to be verified fed back by the target user; and

and the target answer determining module is used for determining the target answer of each question to be verified according to the confidence.

In a specific implementation of the embodiment of the present invention, the method further includes:

the problem acquisition module is used for acquiring the problem to be verified and the detection problem;

the question and answer generating module is used for mixing the detection questions into the questions to be verified to generate the question and answer; and

and the question and answer pushing module is used for pushing the question and answer to the plurality of users.

In a specific implementation of the embodiment of the present invention, the target user filtering module includes:

a fourth quantity obtaining sub-module, configured to, for a single user of the multiple users, obtain, according to the answer of the detection question fed back by the single user, a fourth quantity of correct answers to the detection question by the single user; and

and the target user judgment sub-module is used for judging that the single user is one of the target users under the condition that the fourth number is greater than a preset number threshold.

In a specific implementation of the embodiment of the present invention, the confidence determining module includes:

the first and second people number obtaining submodule is used for obtaining a first people number corresponding to the target user and a second people number corresponding to the cheating user from the plurality of users; the cheating user refers to a user who does not meet the preset condition;

the option obtaining sub-module is used for obtaining a first option and at least one second option except the first option in a plurality of options corresponding to the problem to be verified aiming at each problem to be verified; and

a confidence determination submodule configured to determine the confidence of the first option according to the second number, the third number, the first number, the second number, the first option, and the second option.

In a specific implementation of the embodiment of the present invention, the confidence level determining sub-module includes:

the first probability calculation submodule is used for calculating and obtaining a first probability of the cheating user selecting the first option according to the number of options contained in the problem to be verified;

the second probability calculation submodule is used for calculating and obtaining a second probability of the cheating user passing the cheating prevention strategy according to the third quantity and the quantity of all options contained in the detection questions;

the answer accuracy rate calculation submodule is used for calculating the answer accuracy rate of the target user according to the second probability, the first number of people and the second number of people;

a third fourth person number obtaining sub-module, configured to obtain a third person number for selecting the first option and a fourth person number for selecting the second option from the target users; and

and the confidence coefficient calculation submodule is used for calculating the confidence coefficient of the first option according to the first probability, the answer accuracy, the third number of people and the fourth number of people.

In a specific implementation of the embodiment of the present invention, the target answer determining module includes:

a confidence comparison submodule for comparing the confidence of each option in the problem to be verified with a confidence threshold; and

and the target answer determining submodule is used for determining the target answer of each question to be verified according to the comparison result.

In a specific implementation of the embodiment of the present invention, the target answer determining sub-module includes:

the target option obtaining sub-module is used for obtaining the target option of which the confidence coefficient is greater than the confidence coefficient threshold value from each option of the problem to be verified; and

a target answer obtaining sub-module, configured to, when the number of the target options is at least one, take the target option with the largest confidence as the target answer to the question to be verified; or when the number of the target options is zero, determining that the target answer of the question to be verified is a null value.

According to a third aspect of embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor, a memory, and a computer program stored on the memory and operable on the processor, wherein the processor implements one or more of the answer acquisition methods described above when executing the program.

According to a fourth aspect of embodiments of the present disclosure, there is provided a readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform one or more of the answer obtaining methods described above.

The embodiment of the disclosure provides a question and answer obtaining scheme, which includes receiving answers of a first number of question and answer questions fed back by a plurality of users, wherein the question and answer questions include a second number of questions to be verified and a third number of detection questions, screening at least one target user meeting preset conditions from the plurality of users according to the answers of the detection questions fed back by the plurality of users, determining a confidence level of each option in each question to be verified according to the answers of the question to be verified fed back by the target user, and determining a target answer of each question to be verified according to the confidence level. According to the method and the device, the cheating users can be effectively identified according to the detection questions, the confidence coefficient of each option of each problem to be verified is used as the final adoption basis, the adoption result is not determined according to the number of the selected people of each option, and therefore the obtained final adoption result is more real and accurate, flexible and understandable.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments of the present disclosure will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a flowchart illustrating steps of an answer obtaining method according to an embodiment of the present disclosure;

fig. 2 is a flowchart illustrating steps of an answer obtaining method according to an embodiment of the disclosure;

fig. 3 is a schematic structural diagram of an answer obtaining apparatus according to an embodiment of the disclosure;

fig. 4 is a schematic structural diagram of an answer obtaining device according to an embodiment of the disclosure.

Detailed Description

Technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be obtained by a person skilled in the art without making creative efforts based on the embodiments of the present disclosure, belong to the protection scope of the embodiments of the present disclosure.

Example one

Referring to fig. 1, a flowchart illustrating steps of an answer obtaining method provided in an embodiment of the present disclosure is shown, and as shown in fig. 1, the method may specifically include the following steps:

step 101: receiving answers of a first number of questions and answers fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions.

In the embodiment of the present disclosure, the question-answer questions may include questions to be verified and detection questions, and the questions to be verified may be questions issued by the internet platform or a certain merchant, that is, questions that the internet platform or the certain merchant needs to adopt corresponding results, for example, a question issued by the certain merchant, "what type of clothing you like?," what color you like shoes?, "or the like," or social investigation questions issued by the internet platform, such as "do you buy endowment insurance for oneself?," or the like.

The detection questions are questions mixed in the question to be verified and used for detecting cheating users and non-cheating users, the detection questions are relatively simple questions and are used for detecting whether the user answers the questions carefully and carefully or randomly, and the detection questions can be simple and easy-to-answer questions such as '1 + 1-?' and '5-8-?'.

The first number refers to the number of questions answered by the answering user, i.e., the first number is the same as the number of questions, e.g., when the number of questions is 20, the first number is 20.

The second number is the number of questions to be verified included in the question-answer, and the third number is the number of detection questions included in the question-answer, for example, the total number of question-answer is 50, where the number of questions to be verified is 30, the second number is 30, and the third number is 20 (that is, 20 detection questions are mixed in the question-answer).

I.e. the first number is equal to the sum of the second number and the third number.

It is to be understood that the above examples are only examples set forth for a better understanding of the technical solutions of the embodiments of the present disclosure, and are not to be taken as the only limitations on the embodiments of the present disclosure.

After the first number of questions is sent to the plurality of users, answers to the first number of questions fed back by the plurality of users may be received.

Of course, the number of answers to the question and answer questions fed back by each user may be the first number, or may be greater than the first number, mainly considering that there may be a process of selecting multiple options by the user in a certain question and answer question, which is not the point of the embodiment of the present disclosure, but is only used to briefly describe this form.

After the first number of questions and answers are sent to the plurality of users, there may be a case where the individual user does not answer and directly ignores the answers.

After receiving the answers to the first number of questions fed back by the plurality of users, step 102 is performed.

Step 102: and screening at least one target user meeting preset conditions from the plurality of users according to the answers of the detection questions fed back by the plurality of users.

The target user refers to a non-cheating user among a plurality of users who feed back answers to the question and answer questions.

The preset condition is a condition for determining whether the user is a cheating user according to the accuracy of the answer of the detection question fed back by each user, for example, when the detection question mixed in the question-answer question is 20 tracks, the preset condition can be set to 12 tracks, that is, when the user answers 12 tracks or more in the correct detection question, the user is determined to be a non-cheating user, otherwise, the user is determined to be a cheating user.

Of course, the detailed process for determining whether the user is the target user according to the preset condition will be described in the following second embodiment, and the embodiment of the present disclosure is not described herein again.

After receiving answers of a first number of question and answer questions fed back by a plurality of users, judging whether the user is a target user according to the answer of the detection question fed back by each user, and further finishing screening at least one target user meeting preset conditions from the plurality of users.

After at least one target user is selected from the multiple users according to the answers to the detection questions fed back by the multiple users, step 103 is executed.

Step 103: and determining the confidence of each option in each question to be verified according to the answer of the question to be verified fed back by the target user.

The confidence level refers to a credibility that each option corresponding to each to-be-verified problem can be used as an adoption result, and the confidence level may be a numerical value, such as 0.5, 0.8, and the like, and specifically, may be determined according to a service requirement, which is not limited in this embodiment of the present disclosure.

After at least one target user meeting preset conditions is screened out from the multiple users, the confidence level of each option in each question to be verified can be determined according to the answer of the question to be verified fed back by the target user.

That is, in the present disclosure, the answer fed back by the non-target user (i.e., the cheating user) of the plurality of users is not considered, but the answer of the question to be verified fed back by the non-cheating user is directly used to determine the adoption result (i.e., the target answer) of each question to be verified.

Of course, for the above-mentioned manner of determining the confidence level of each option of the problem to be verified, the confidence level of each option of the problem to be verified may be calculated by using a bayesian probability model, and a specific calculation process will be described in detail in the following second embodiment, which is not described herein again in this disclosure.

After determining the confidence of each option in each question to be verified according to the answer of the question to be verified fed back by the target user, step 104 is executed.

Step 104: and determining the target answer of each question to be verified according to the confidence.

After the confidence of each option in each question to be verified is determined according to the answer of the question to be verified fed back by the target user, the target answer of the question to be verified can be determined according to the confidence of each option in each question to be verified.

In the present disclosure, a confidence threshold may be preset, and when the confidence of each option in the to-be-verified question is less than or equal to the confidence threshold, the target answer of the to-be-verified question is determined to be a null value, that is, there is no target answer. And when the confidence of one or more options exists in the confidence of each option in the question to be verified, the option corresponding to the confidence with the maximum confidence value is selected from the one or more options with the confidence values larger than the confidence threshold, and the option corresponding to the maximum confidence is used as the target answer of the question to be verified.

For the above process, the following embodiment two will be described in detail, and the embodiments of the present disclosure are not described herein again.

According to the method and the device, the cheating users can be effectively identified according to the detection questions, the confidence coefficient of each option of each problem to be verified is used as the final adoption basis, the adoption result is not determined according to the number of the selected people of each option, and therefore the obtained final adoption result is more real and accurate.

The question answer obtaining method provided by the embodiment of the disclosure includes receiving answers of a first number of question answers fed back by a plurality of users, wherein the question answers include a second number of questions to be verified and a third number of detection questions, screening at least one target user meeting preset conditions from the plurality of users according to the answers of the detection questions fed back by the plurality of users, determining confidence of each option in each question to be verified according to the answers of the questions to be verified fed back by the target user, and determining the target answer of each question to be verified according to the confidence. According to the method and the device, the cheating users can be effectively identified according to the detection questions, the confidence coefficient of each option of each problem to be verified is used as the final adoption basis, the adoption result is not determined according to the number of the selected people of each option, and therefore the obtained final adoption result is more real and accurate, flexible and understandable.

Example two

Referring to fig. 2, a flowchart illustrating steps of an answer obtaining method provided in the embodiment of the present disclosure is shown, and as shown in fig. 2, the method may specifically include the following steps:

step 201: and acquiring the problem to be verified and the detection problem.

In the embodiment of the present disclosure, the question to be verified may be a question issued by the internet platform or a certain merchant, that is, a question that the internet platform or a certain merchant needs to adopt the corresponding result, for example, a question issued by a certain merchant, "what type of clothing? you like," what color of shoes? you like, "or a social investigation question issued by the internet platform," how you will buy endowment insurance for oneself?, "or the like.

The number of the questions to be verified may be determined by the internet platform or the merchant, and the number of the detection questions may be determined by the number of the questions to be verified, for example, if it is preset that one detection question is mixed every 2 questions when the number of the questions to be verified is 30 questions, the number of the mixed detection questions is 15; if it is preset that one detection question is mixed every 5 questions, the number of the mixed detection questions is 6.

After the problem to be verified, which is released by an internet platform or a certain merchant, is obtained, the detection problems of the matching number can be obtained according to a preset rule.

After the questions to be verified and the detected questions are obtained, step 202 is performed.

Step 202: and mixing the detection questions into the questions to be verified to generate the question-answering questions.

After the problem to be verified and the detection questions are obtained, the detection questions can be mixed into the problem to be verified, the mixing rule can be a random rule, namely, a specified number of detection questions are randomly mixed into the problem to be verified, and the mixing interval is random.

Of course, the interval of the detection questions may also be preset, for example, every 3 or 5 questions to be verified are mixed with 1 detection question, and so on.

In a specific implementation, a person skilled in the art may set a mixing manner of the detection questions according to business requirements, which is not limited in the embodiment of the present disclosure.

After the detection questions are mixed into the questions to be verified, question-and-answer questions can be generated based on the questions to be verified and the mixed detection questions, namely, the question-and-answer questions are a combination of the questions to be verified and the detection questions.

After the detected questions are mixed into the questions to be verified to generate the question-and-answer questions, step 203 is executed.

Step 203: and pushing the question and answer questions to the plurality of users.

After the question and answer questions are generated, the question and answer questions can be pushed to a plurality of users, and understandably, the pushing process can be that when the users access the internet platform, the question and answer questions pushed out by the internet are pushed to the users who are currently accessing, so as to be displayed on a screen of the user terminal; for the merchant, when the user accesses the merchant through the internet, the question and answer questions pushed by the merchant can be pushed to the user.

Of course, in practical applications, other pushing manners may also be adopted, and in particular, may be determined according to practical situations, and the embodiment of the present disclosure is not limited thereto.

After the questions are pushed to the plurality of users, step 204 is performed.

Step 204: receiving answers of a first number of questions and answers fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions.

Of course, after the first number of questions and answers are sent to the plurality of users, there may be a case where the individual user does not directly ignore the answers, and the embodiment of the present disclosure does not consider such a case, and only uses the user who has fed back the answers as a reference for selecting the adopted result.

After receiving the answers to the first number of questions fed back by the plurality of users, step 205 is performed.

Step 205: and aiming at a single user in the plurality of users, acquiring a fourth quantity of correct answers of the single user to the detection questions according to the answers of the detection questions fed back by the single user.

A single user refers to any one of a plurality of users.

The fourth number is the number of the detection questions answered by a single user, for example, the total number of the detection questions is 20, wherein if a certain user answers 15 questions correctly, the fourth number is 15 for the user; when the number of correct answers to the detection questions of a certain user is 18, the fourth number is 18 for the user.

After receiving the answers of the first number of question-answer questions fed back by the plurality of users, a fourth number of correct answers of the single user to the detection questions can be obtained according to the answers of the detection questions fed back by the single user.

After obtaining a fourth quantity of answers to the detection questions, which are correctly answered by the single user, according to the answers to the detection questions fed back by the single user, for the single user, step 206 is performed.

Step 206: and in the case that the fourth number is greater than a preset number threshold, determining that the single user is one of the target users.

The preset number threshold is a threshold preset by service personnel and used for judging whether a single user is a target user. The preset number threshold may be 15, 16, etc., and specifically, may be determined according to practical situations, and the embodiment of the present disclosure is not limited thereto.

After a fourth quantity of correct answers to the detection questions by each of the multiple users is obtained, each fourth quantity can be compared with a preset quantity threshold value, and under the condition that the fourth quantity corresponding to a single user is less than or equal to the preset quantity threshold value, the single user can be judged to be a non-target user (namely a cheating user), namely the user possibly has the condition that questions and answers issued by an internet platform or a merchant and the like are randomly answered and not seriously examined; and in the case that the fourth quantity corresponding to the single user is greater than the preset quantity threshold value, the single user can be determined to be the target user (i.e. the non-cheating user).

After the target user is obtained from the multiple users according to the fourth quantity of answers to the detection questions correctly answered by each of the multiple users, step 207 is executed.

Step 207: acquiring a first number of people corresponding to the target user and a second number of people corresponding to the cheating user from the plurality of users; the cheating user refers to a user who does not meet the preset condition.

The cheating users are users who do not meet the preset conditions, namely the fourth quantity of the cheating users who answer the detection questions correctly is smaller than or equal to the preset quantity threshold value.

The first number of people refers to the number of people among the plurality of users who are target users (i.e., non-cheating users), and the second number of people refers to the number of people among the plurality of users who are cheating users.

It is understood that the sum of the first number of users and the second number of users is the total number of the users who feed back answers to the question and answer, for example, the total number of the users is 38, and when the first number of users corresponding to the target user is 20, the second number of users is 38-20 — 18, that is, the number of cheating users is 18.

After the first number of people corresponding to the target user and the second number of people corresponding to the cheating user are obtained from the plurality of users, step 208 is executed.

Step 208: and aiming at each question to be verified, acquiring a first option in a plurality of options corresponding to the question to be verified and at least one second option except the first option.

The first option is one of a plurality of options corresponding to the problem to be verified, and the second option is the other options except the first option in the plurality of options corresponding to the problem to be verified. For example, the question to be verified includes options a, b, and c, and when option a is identified as the first option, options b and c are the second options; and when the option b is identified as the first option, the options a and c are the second option.

For each question to be verified, after determining the first option and the at least one second option of each question to be verified according to a preset rule (for example, according to the ordering of the questions to be verified, the first option is regarded as the first option), step 209 is performed.

Step 209: determining the confidence level of the first option according to the second number, the third number, the first number, the second number, the first option, and the second option.

In the above step, after the second number (number of questions to be verified), the third number (number of detected questions), the first number (number of target users), the second number (number of cheating users), the first option (one option in each question to be verified), and the second option (other options except the first option in each question to be verified) are obtained, the confidence of the first option may be determined by combining the second number, the third number, the first number, the second number, the first option, and the second option.

In the present disclosure, the confidence of the first option of each to-be-verified problem may be calculated by using the second number, the third number, the first number, the second number, the first option, and the second option as parameters and referring to a bayesian probability model, and for a specific calculation process, the following specific implementation manner may be combined for detailed description.

In a specific implementation of the embodiment of the present disclosure, the step 209 may include:

substep S1: and calculating to obtain a first probability of the cheating user selecting the first option according to the number of options contained in the problem to be verified.

The embodiment of the present disclosure may calculate the confidence of each option in each to-be-verified problem by combining with a bayesian probability model, that is, the confidence of one option (i.e., the first option) for one to-be-verified problem in the following implementation process.

After selecting a question to be verified from the question and answer, the number of options of the question to be verified can be determined, for example, when the options of the question to be verified include options a and B, the number of the options of the question to be verified is 2; and when the options of the question to be verified include A, B, C and D, the number of the options of the question to be verified is 4.

After the number of options included in the problem to be verified is obtained, a first probability that the cheating user selects the first option may be calculated according to the number of options included in the problem to be verified, where understandably, the first option refers to one of the multiple options included in the problem to be verified, and understandably, the embodiment of the present disclosure is a process of calculating the confidence level of each option of the problem to be verified, respectively, any one of the multiple options of the problem to be verified may be used as the first option, for example, the problem to be verified includes options a and B, the option a may be used as the first option, and after the confidence level of the option a is calculated, the option B may be used as the first option, so as to calculate the confidence level of the option B.

After the number of options included in the problem to be verified is obtained, a first probability that the cheating user selects the first option may be obtained by calculation according to the number of options included in the problem to be verified, and specifically, the calculation process may be as shown in the following formula (1).

In the above formula (1), t represents the number of options of the question to be verified, P (a | a) represents a first probability, a represents a first option, and a represents an event that the user clicks the first option.

It can be understood that, since the cheating user randomly selects an option for each question to be verified, the probability of selecting each option is the same for each cheating user, that is, when the number of options is t, the probability of selecting the first option by the cheating user is t

After calculating a first probability that the cheating user selects the first option according to the number of options included in the question to be verified, sub-step S2 is performed.

Substep S2: and calculating to obtain a second probability of the cheating user passing the cheating prevention strategy according to the third quantity and the number of all options contained in the detection questions.

The anti-cheating policy is a policy for shielding cheating users by mixing detection questions into the problems to be verified in the embodiment of the disclosure.

Of course, in practical applications, it is not possible to shield all cheating users by mixing detection questions, and there may be some cheating users who pass the anti-cheating policy.

The second probability is the second probability that the cheating user passes the anti-cheating strategy, i.e. the second probability that a certain cheating user passes the anti-cheating strategy.

The total number of options is the sum of the numbers of options corresponding to all the detection questions that are mixed, for example, the number of the detection questions that are mixed is 10, and the number of options of each detection question is 4, so that the number of all the options included in the detection questions is 10 × 4 — 40.

After the third number of the mixed detection questions and the number of all the options included in the mixed detection questions are obtained, a second probability that the cheating user passes through the cheating prevention strategy can be calculated, and specifically, the calculation process can be shown by referring to the following formula (2):

in the above formula (2), h represents the second probability, e represents the third number, and L represents the number of all the options included in the mixed detection question.

And substituting the third quantity and the number of all options contained in the detection questions into the formula (2) to calculate a second probability that the cheating user passes the cheating prevention strategy.

Substep S3: and calculating the answer accuracy rate of the target user according to the second probability, the first number of people and the second number of people.

The answer accuracy rate refers to the accuracy rate of the target user (non-cheating user) to answer the problem to be verified.

In the above process, when the second probability, the first number of people and the second number of people are obtained, the answer accuracy of the target user can be calculated according to the second probability, the first number of people and the second number of people, and specifically, the detailed description can be given with reference to the calculation process of the following formula.

Suppose that in a certain answer, z people (i.e. target users) pass through the anti-cheating strategy, w people are identified as cheating users, the number of normal users (i.e. non-cheating users) is x, and the number of the cheating users is y.

Then:

x+y＝z+w (3)

z＝x+hy (4)

in the above equations (3) and (4), h is the second probability.

The following equations (5) and (6) can be obtained by integrating the above equations (3) and (4):

in the above process, after the answer result of z people is subjected to spot inspection, the overall accuracy is set as q, and then the calculation process of q can be shown in the following formula (7):

in the above equation (7), r represents the answer accuracy of the target user.

By calculating the above equation (7), the following equation (8) can be obtained:

the answer accuracy of the target user can be calculated through the formula (8).

After the answer accuracy of the target user is calculated according to the second probability, the first number of people and the second number of people, sub-step S4 is executed.

Substep S4: and acquiring a third number of people selecting the first option and a fourth number of people selecting the second option from the target users.

The third number of people is the number of people who select the first option in the target user, and the fourth number of people is the number of people who select the second option in the target user.

After receiving answers of the questions fed back by the users, the third number of people selecting the first option and the fourth number of people selecting the second option can be obtained according to answers fed back by the target users and the questions to be verified.

After the third number of persons and the fourth number of persons are acquired, substep S5 is performed.

Substep S5: and calculating the confidence coefficient of the first option according to the first probability, the answer accuracy, the third number of people and the fourth number of people.

Based on the first probability, the answer accuracy, the third number of people and the fourth number of people obtained in the process, the confidence coefficient of the first option can be calculated.

In this step, the confidence of the first option may be calculated by combining a bayesian formula, which may be shown as the following formula (9):

in the above formula (9), a is one option in the question (i.e. the first option in the present disclosure), b is a non-a option (i.e. the second option in the present disclosure), a is an event that the user clicks option a, and P (a | a) is the probability that a is the correct option when one user selects a; p (a) is the probability that a is the correct choice (i.e., the target answer); p (b) is the probability that the correct option is not a, P (a | a) is the probability that the user answers correctly, and P (a | b) is the probability that the user answers incorrectly.

In the embodiment of the present disclosure, a bayesian formula (the above formula (9)) may be referred to, and the following formula (10) may be obtained by substituting the first probability, the answer accuracy, the third person number, and the fourth person number obtained in the above process into the above formula (9):

in the above formula (10), a, b, …, t represent the options of a certain question to be verified, p (a) represents the probability that a is the correct option, N_aThe number of people who select a is indicated,N_bindicating the number of people who pick b, …, N_tIndicates the number of people who pick t, P (N)_aN_b...N_tA) represents the probability that the head will pick a, P (aN)_aN_b...N_t) Representing the confidence of the question to be verified, P (t) representing the probability of correct option of t, P (N)_aN_b...N_tI t) represents the probability of total people choosing t.

In the case of the above equation (10):

wherein, in the above formula (11) and formula (12):

the confidence of the first option can be calculated by the above equations (10), (11), (12), (13) and (14).

After determining the confidence level of the first option based on the second number, the third number, the first number, the second number, the first option, and the second option, step 210 is performed.

Step 210: comparing the confidence of the each option in the question to be verified to a confidence threshold.

The confidence threshold refers to a threshold preset by a service person and associated with a confidence, and the confidence threshold may be 0.5, 0.6, 0.9, and the like, and specifically, may be determined according to a service requirement, which is not limited in this embodiment of the disclosure.

After the confidence of each option in the problem to be verified is obtained, the confidence of each option can be compared with a confidence threshold respectively, and then a comparison result can be obtained.

After comparing the confidence of each option in the question to be verified with the confidence threshold to obtain a comparison result, step 211 is executed.

Step 211: and determining the target answer of each question to be verified according to the comparison result.

After the comparison result is obtained, the target answer of each question to be verified may be determined according to the comparison result, and the process of determining the target answer may be described in detail with reference to the following specific implementation manner.

In a specific implementation of the present disclosure, the step 211 may include:

substep N1: and acquiring the target option with the confidence degree larger than the confidence degree threshold value from each option of the problem to be verified.

In the embodiment of the present disclosure, the target option refers to an option in each to-be-verified question, where the confidence of the option in each to-be-verified question is greater than the confidence threshold, for example, the to-be-verified question a includes options a, b, and c, the confidence threshold is 0.5, the confidence of the option a is 0.4, the confidence of the option b is 0.6, and the confidence of the option c is 0.8, and then the target option in the to-be-verified question a is the option b and the option c.

Of course, for each question to be verified, the target options of the question to be verified may be 0, or 1, or 2 or more, and specifically, may be determined according to the actual situation.

After the target option with the confidence level larger than the confidence level threshold value is obtained from each option of the problem to be verified, the substep N2 is executed.

Substep N2: when the number of the target options is at least one, taking the target option with the maximum confidence as the target answer of the question to be verified; or when the number of the target options is zero, determining that the target answer of the question to be verified is a null value.

For each question to be verified, the way of determining the target answer according to the target option can be divided into the following two cases:

1. case where the number of target options of the question to be verified is at least one

a. When the number of the target options of the question to be verified is 1, directly taking the target options as target answer options of the question to be verified;

b. when the number of the target options of the to-be-verified question is 2 or more, obtaining the confidence degrees corresponding to the plurality of target options respectively, and taking the target option with the highest confidence degree as the target answer option of the to-be-verified question, for example, the target option of the to-be-verified question a includes an option a and an option b, the confidence degree of the option a is 0.6, and the confidence degree of the option b is 0.8, and then taking the option b as the target answer option of the to-be-verified question a.

2. Case where the number of target options of the question to be verified is 0

And when the target option does not exist in the multiple options of the problem to be verified, determining the target answer of the problem to be verified as a null value, namely that the problem to be verified does not have the target answer option.

The accuracy of the adoption result of each question and answer can be obtained by setting the confidence threshold value.

The answer obtaining method provided by the embodiment of the disclosure, in addition to the beneficial effects of the answer obtaining method provided by the first embodiment, may further preset a confidence threshold, and select the option with the largest confidence as the adoption result under the condition that the confidence of the question and answer options is greater than the confidence threshold, thereby further improving the accuracy of obtaining the adoption result of each question and answer.

EXAMPLE III

Referring to fig. 3, a schematic structural diagram of an answer obtaining apparatus provided in the embodiment of the present disclosure is shown, and as shown in fig. 3, the apparatus may specifically include the following modules:

an answer receiving module 310, configured to receive answers of a first number of questions fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions;

the target user screening module 320 is configured to screen at least one target user meeting a preset condition from the multiple users according to the answers of the detection questions fed back by the multiple users;

the confidence level determining module 330 is configured to determine a confidence level of each option in each question to be verified according to the answer of the question to be verified fed back by the target user; and

and the target answer determining module 340 is configured to determine a target answer for each question to be verified according to the confidence.

The problem answer obtaining device provided by the embodiment of the disclosure is configured to, by receiving answers of a first number of questions and answers fed back by a plurality of users, select at least one target user meeting preset conditions from the plurality of users according to the answers of the first number of questions to be verified and a third number of detection questions fed back by the plurality of users, determine a confidence level of each option in each question to be verified according to the answers of the questions to be verified fed back by the target user, and determine a target answer of each question to be verified according to the confidence level. According to the method and the device, the cheating users can be effectively identified according to the detection questions, the confidence coefficient of each option of each problem to be verified is used as the final adoption basis, the adoption result is not determined according to the number of the selected people of each option, and therefore the obtained final adoption result is more real and accurate, flexible and understandable.

Example four

Referring to fig. 4, a schematic structural diagram of an answer obtaining apparatus provided in the embodiment of the present disclosure is shown, and as shown in fig. 4, the apparatus may specifically include the following modules:

a problem obtaining module 410, configured to obtain the problem to be verified and the detection problem;

the question-answer generating module 420 is configured to mix the detection questions into the questions to be verified to generate the question-answer questions;

a question and answer pushing module 430, configured to push the question and answer to the multiple users;

an answer receiving module 440, configured to receive answers of a first number of questions fed back by a plurality of users; the question-answering questions comprise a second number of questions to be verified and a third number of detection questions;

the target user screening module 450 is configured to screen at least one target user meeting a preset condition from the multiple users according to the answers of the detection questions fed back by the multiple users;

a confidence level determining module 460, configured to determine a confidence level of each option in each question to be verified according to the answer of the question to be verified fed back by the target user; and

and a target answer determining module 470, configured to determine a target answer for each question to be verified according to the confidence.

In a specific implementation of the present disclosure, the target user filtering module 450 includes:

a fourth quantity obtaining sub-module 4501, configured to, for a single user of the multiple users, obtain, according to the answer of the detection question fed back by the single user, a fourth quantity that the single user answers the detection question correctly; and

a target user determination sub-module 4502, configured to determine that the single user is one of the target users when the fourth number is greater than a preset number threshold.

In one specific implementation of the present disclosure, the confidence determining module 460 includes:

a first and second number of people obtaining sub-module 4601, configured to obtain, from the plurality of users, a first number of people corresponding to the target user and a second number of people corresponding to the cheating user; the cheating user refers to a user who does not meet the preset condition;

an option obtaining sub-module 4602, configured to, for each problem to be verified, obtain a first option in a plurality of options corresponding to the problem to be verified and at least one second option except for the first option; and

a confidence level determination submodule 4603, configured to determine the confidence level of the first option according to the second number, the third number, the first number, the second number, the first option, and the second option.

In a specific implementation of the present disclosure, the confidence determination sub-module 4603 includes:

In one specific implementation of the present disclosure, the target answer determining module 470 includes:

a confidence comparison sub-module 4701 for comparing the confidence of each option in the question to be verified with a confidence threshold; and

and a target answer determining sub-module 4702 for determining the target answer for each question to be verified according to the comparison result.

In one specific implementation of the present disclosure, the target answer determination sub-module 4702 includes:

The answer obtaining device provided by the embodiment of the disclosure, in addition to the beneficial effects of the answer obtaining device provided by the third embodiment, may further preset a confidence threshold, and select the option with the maximum confidence as the adoption result under the condition that the confidence of the question and answer options is greater than the confidence threshold, thereby further improving the accuracy of obtaining the adoption result of each question and answer.

An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor, the processor implementing the answer obtaining method of the foregoing embodiment when executing the program.

Embodiments of the present disclosure also provide a readable storage medium, in which instructions are executed by a processor of an electronic device to enable the electronic device to perform the answer obtaining method of the foregoing embodiments.

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present disclosure are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the embodiments of the present disclosure as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the embodiments of the present disclosure.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the disclosure may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the disclosure, various features of the embodiments of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, claimed embodiments of the disclosure require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of an embodiment of this disclosure.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

The various component embodiments of the disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be understood by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a motion picture generating device according to an embodiment of the present disclosure. Embodiments of the present disclosure may also be implemented as an apparatus or device program for performing a portion or all of the methods described herein. Such programs implementing embodiments of the present disclosure may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit embodiments of the disclosure, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the disclosure may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The above description is only for the purpose of illustrating the preferred embodiments of the present disclosure and is not to be construed as limiting the embodiments of the present disclosure, and any modifications, equivalents, improvements and the like that are made within the spirit and principle of the embodiments of the present disclosure are intended to be included within the scope of the embodiments of the present disclosure.

The above description is only a specific implementation of the embodiments of the present disclosure, but the scope of the embodiments of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present disclosure, and all the changes or substitutions should be covered by the scope of the embodiments of the present disclosure. Therefore, the protection scope of the embodiments of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An answer obtaining method, comprising:

2. The method of claim 1, wherein prior to the step of receiving answers to the first number of questions fed back by the plurality of users, further comprising:

acquiring the problem to be verified and the detection problem;

and pushing the question and answer questions to the plurality of users.

3. The method according to claim 1, wherein the step of selecting at least one target user meeting a predetermined condition from the plurality of users according to the answer to the detection question fed back by the plurality of users comprises:

4. The method according to claim 1, wherein the step of determining the confidence level of each option in each question to be verified according to the answer of the question to be verified fed back by the target user comprises:

5. The method of claim 4, wherein said step of determining said confidence level for said first item based on said second number, said third number, said first number, said second number, said first item, and at least one of said second items comprises:

6. The method according to claim 1, wherein the step of determining the target answer of each question to be verified according to the confidence level comprises:

7. The method according to claim 6, wherein the step of determining the target answer for each of the questions to be verified according to the comparison result comprises:

8. An answer obtaining apparatus, comprising:

9. An electronic device, comprising:

processor, memory and computer program stored on the memory and executable on the processor, characterized in that the processor implements the answer acquisition method according to one or more of claims 1 to 7 when executing the program.

10. A readable storage medium, characterized in that instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the answer acquisition method as recited in one or more of method claims 1 to 7.