CA3166342A1 - Automatic question setting method, apparatus and system - Google Patents
Automatic question setting method, apparatus and systemInfo
- Publication number
- CA3166342A1 CA3166342A1 CA3166342A CA3166342A CA3166342A1 CA 3166342 A1 CA3166342 A1 CA 3166342A1 CA 3166342 A CA3166342 A CA 3166342A CA 3166342 A CA3166342 A CA 3166342A CA 3166342 A1 CA3166342 A1 CA 3166342A1
- Authority
- CA
- Canada
- Prior art keywords
- question
- matching degree
- examination question
- answer
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000009434 installation Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
Abstract
Provided are an automatic question setting method, apparatus and system. The method comprises: acquiring question and answer content of the current examination question (S11); calculating a first matching degree between the question and answer content of each historical examination question and the question and answer content of the current examination question (S12); selecting the next question of the historical examination question with the highest first matching degree to be a candidate examination question (S13); calculating a second matching degree between the candidate examination question and the next question in a question library (S14); and if the second matching degree satisfies a preset condition, taking the candidate examination question as a target examination question for question setting (S15). According to the method, apparatus and system, an actual scenario can be simulated in order to automatically set questions.
Description
AUTOMATIC QUESTION SETTING METHOD, APPARATUS AND SYSTEM
BACKGROUND OF THE INVENTION
Technical Field [0001] The present application relates to the field of questioning and answering, and more particularly to a method of automatically setting a question, and corresponding device and system.
Description of Related Art
BACKGROUND OF THE INVENTION
Technical Field [0001] The present application relates to the field of questioning and answering, and more particularly to a method of automatically setting a question, and corresponding device and system.
Description of Related Art
[0002] Q&A scenes often appear in many products and platforms. For instance, the interaction between intelligent customer service and a user in an e-commerce platform is a Q&A scene.
In order to enhance the precision of response by the intelligent customer service, it is required to raise questions in simulation of users, to appraise responses by the intelligent customer service, so as to subsequently make directional improvements.
In order to enhance the precision of response by the intelligent customer service, it is required to raise questions in simulation of users, to appraise responses by the intelligent customer service, so as to subsequently make directional improvements.
[0003] In the state of the art, a question bank is preset to select examination questions therefrom to be raised to the intelligent customer service in simulation of users according to a certain rule such as sequence, to acquire answers from the intelligent customer service, so as to detect the answering precision of the intelligent customer service.
[0004] In the state of the art, the rule and sequence for setting questions from the question bank were previously well set, no matter what an answer to the current question might be, the next question will be continually selected and set from the question bank according to the preset rule. Such a question setting mode is detached from the actual Q&A
scene, as the questions are contextually not necessarily interrelated.
scene, as the questions are contextually not necessarily interrelated.
[0005] The same problem also exists under other Q&A scenes.
SUMMARY OF THE INVENTION
SUMMARY OF THE INVENTION
[0006] The present application provides a method of automatically setting a question, and corresponding device and system to address the problem pending in the state of the art in which it is impossible to simulate the actual scene while simulating questions and answers, so that questions as set are contextually not associated.
[0007] According to the first aspect, the present application provides a method of automatically setting a question, and the method comprises:
Date Regue/Date Received 2022-06-29
Date Regue/Date Received 2022-06-29
[0008] obtaining Q&A content with respect to a current examination question;
[0009] calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question;
[0010] selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question;
[0011] calculating a second matching degree between the candidate examination question and the next question in a question bank; and
[0012] taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
[0013] Preferably, the method further comprises:
[0014] taking the next question in the question bank to serve as a target examination question to set the question, if the second matching degree does not satisfy the preset condition.
[0015] Preferably, the step of calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question includes:
[0016] coding the Q&A content of the current examination question according to a preset Q&A
model, and obtaining a question code and an answer code of the current examination question;
model, and obtaining a question code and an answer code of the current examination question;
[0017] coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
[0018] calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
[0019] calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question;
and
and
[0020] calculating the first matching degree according to the question matching degree and the answer matching degree.
[0021] Preferably, the step of selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question includes:
Date Regue/Date Received 2022-06-29
Date Regue/Date Received 2022-06-29
[0022] selecting any historical examination question whose question matching degree is greater than a first preset threshold and whose answer matching degree is greater than a second preset threshold from Q&A contents of all historical examination questions to serve as a historical examination question to be selected; and
[0023] selecting the next question of the historical examination question with the highest first matching degree from the historical examination question(s) to be selected to serve as a candidate examination question.
[0024] Preferably, the method further comprises:
[0025] coding the Q&A content of the current examination question according to a preset Q&A
model, and obtaining a question code and an answer code of the current examination question;
model, and obtaining a question code and an answer code of the current examination question;
[0026] calculating a third matching degree between the question code and the answer code of the current examination question;
[0027] calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm; and
[0028] calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
[0029] According to the second aspect, the present application provides a device for automatically setting a question, and the device comprises:
[0030] a current examination question obtaining unit, for obtaining Q&A
content with respect to a current examination question;
content with respect to a current examination question;
[0031] a first matching degree calculating unit, for calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question;
[0032] a candidate examination question unit, for selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question;
[0033] a second matching degree calculating unit, for calculating a second matching degree between the candidate examination question and the next question in a question bank; and Date Regue/Date Received 2022-06-29
[0034] a question setting unit, for taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
[0035] Preferably, the question setting unit is further employed for taking the next question in the question bank to serve as a target examination question to set the question, if the second matching degree does not satisfy the preset condition.
[0036] Preferably, the first matching degree calculating unit includes a first coding unit, a question matching degree calculating unit, an answer matching degree calculating unit and a first matching degree calculating subunit, of which:
[0037] the first coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
[0038] the first coding unit is further employed for coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
[0039] the question matching degree calculating unit is employed for calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
[0040] the answer matching degree calculating unit is employed for calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and
[0041] the first matching degree calculating subunit is employed for calculating the first matching degree according to the question matching degree and the answer matching degree.
[0042] Preferably, the device further comprises:
[0043] a second coding unit, a third matching degree calculating unit, a fourth matching degree calculating unit and an answer score calculating unit, of which:
[0044] the second coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
[0045] the third matching degree calculating unit is employed for calculating a third matching Date Regue/Date Received 2022-06-29 degree between the question code and the answer code of the current examination question;
[0046] the fourth matching degree calculating unit is employed for calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm; and
[0047] the answer score calculating unit is employed for calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
[0048] According to the third aspect, the present application provides a computer system that comprises:
[0049] one or more processor(s); and
[0050] a memory, associated with the one or more processor(s), for storing a program instruction that executes the aforementioned method when it is read and executed by the one or more processor(s).
[0051] According to specific embodiments provided by the present application, the present application has made public the following technical effects.
[0052] The technical solution of the present application bases on the Q&A
circumstance of the current examination question to select close Q&As from historical Q&As, and selects the next sentence of the closest historical Q&A to serve as the next examination question when the next sentence of the closest historical Q&A is highly similar to the next question in the question bank, whereby is realized automatic determination of an associated question from historical Q&As for continued question setting according to the answering circumstance of the user to each question, scene-based simulated training is constructed, and the degree of association among examination questions is enhanced.
circumstance of the current examination question to select close Q&As from historical Q&As, and selects the next sentence of the closest historical Q&A to serve as the next examination question when the next sentence of the closest historical Q&A is highly similar to the next question in the question bank, whereby is realized automatic determination of an associated question from historical Q&As for continued question setting according to the answering circumstance of the user to each question, scene-based simulated training is constructed, and the degree of association among examination questions is enhanced.
[0053] Moreover, in the present application are synthesized the matching degree between the standard answer and the answer of the examinee with the matching degree between the examination question and the answer of the examinee to score the answer of the examinee, so that scoring of the answer of the examinee reaches actual scene response and approaches standard answer specification.
BRIEF DESCRIPTION OF THE DRAWINGS
BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly describe the technical solutions in the embodiments of the present Date Regue/Date Received 2022-06-29 application or in prior-art technology, drawings required for the illustration of the embodiments will be briefly introduced below. Apparently, the drawings described below are merely directed to some embodiments of the present application, and it is possible for persons ordinarily skilled in the art to base on these drawings to acquire other drawings without spending creative effort in the process.
[0055] Fig. 1 is a flowchart schematically illustrating the method according to the present application;
[0056] Figs. 2-4 are views schematically illustrating specific embodiments of the present application;
[0057] Fig. 5 is a view illustrating the structure of the device according to an embodiment of the present application; and
[0058] Fig. 6 is a view illustrating the structure of the computer system.
DETAILED DESCRIPTION OF THE INVENTION
DETAILED DESCRIPTION OF THE INVENTION
[0059] The technical solutions in the embodiments of the present application will be clearly and comprehensively described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the embodiments as described are merely partial embodiments, rather than the entire embodiments, of the present application.
All other embodiments obtainable by persons ordinarily skilled in the art on the basis of the embodiments in the present application without spending any creative effort shall all be covered by the protection scope of the present application.
All other embodiments obtainable by persons ordinarily skilled in the art on the basis of the embodiments in the present application without spending any creative effort shall all be covered by the protection scope of the present application.
[0060] As noted in the Description of Related Art, the mode of setting questions from a question bank according to a preset sequence ignores the association between the Q&A
circumstance of the previous examination question and the next examination question, so that the next examination question is detached from the Q&A of the previous examination question, whereby the degree of association between the examination questions is rendered low.
circumstance of the previous examination question and the next examination question, so that the next examination question is detached from the Q&A of the previous examination question, whereby the degree of association between the examination questions is rendered low.
[0061] The present application aims to provide a method of automatically setting a question, in which the Q&A circumstance of a user to the current examination question is based on to select the closest historical Q&A from historical Q&As to determine an actually associated scene, the next sentence of the closest historical Q&A is taken as a candidate of the next question, and the final next question is determined in conjunction with the similarity Date Regue/Date Received 2022-06-29 between the candidate question and the next question in the question bank.
Since similar historical Q&As are combined to determine the actual scene, question is raised with the next question in the actual scene, the degree of association between the next question and the current question must be enhanced.
Since similar historical Q&As are combined to determine the actual scene, question is raised with the next question in the actual scene, the degree of association between the next question and the current question must be enhanced.
[0062] Embodiment 1
[0063] As shown in Fig. 1, embodiment 1 of the present application provides a method of automatically setting a question, and the method comprises the following steps.
[0064] Sll - obtaining Q&A content with respect to a current examination question.
[0065] The Q&A content with respect to a current examination question is the question and the answer of the current examination question.
[0066] In order to determine the associated scene, it is firstly needed to obtain the question and the answer of the user with respect to the current examination question.
[0067] In a specific embodiment, the question and the answer can be taken as a question pair <Q, A> to be stored in association, where Q represents the question, and A
represents the corresponding answer.
represents the corresponding answer.
[0068] S12 - calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question.
[0069] Q&A contents of historical examination questions can be obtained by the collection of data in actual scenes, for instance, Q&A data between users and customer service on an e-commerce platform is collected to serve as Q&A contents of historical examination questions. The Q&A content of each historical examination question can also be stored in association in the form of a question pair <Q, A>.
[0070] S13 - selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question.
[0071] Through this step is just determined the actual scene associated with the current question.
The selection of the next question in the actual scene as a candidate is to base on the actual scene to select questions.
The selection of the next question in the actual scene as a candidate is to base on the actual scene to select questions.
[0072] S14 ¨ calculating a second matching degree between the candidate examination question and the next question in a question bank.
[0073] S15 ¨ taking the candidate examination question to serve as a target examination question Date Regue/Date Received 2022-06-29 to set the question, if the second matching degree satisfies a preset condition.
[0074] Steps S14 and S15 take consideration that the actual scene should not be deviated too much from examination questions in the question bank, accordingly, it is calculated to match the selected candidate examination question with the next question in the question bank, when the matching degree satisfies the requirement, we consider that the candidate examination question also satisfies the basic requirement of question setting, and that it is more associated with the current examination question relative to the next question in the question bank, so this candidate examination question is taken to serve as the target examination question to set the question.
[0075] As shown in Fig. 1, the method further comprises:
[0076] S16 - taking the next question in the question bank to serve as a target examination question to set the question, if the second matching degree does not satisfy the preset condition. This step ensures that a historical examination question that is much deviated from the question bank is taken as the next question.
[0077] When the first matching degree is calculated in step S12, the following mode can be employed to proceed:
[0078] coding the Q&A content of the current examination question according to a preset Q&A
model, and obtaining a question code and an answer code of the current examination question;
model, and obtaining a question code and an answer code of the current examination question;
[0079] coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
[0080] calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
[0081] calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question;
and
and
[0082] calculating the first matching degree according to the question matching degree and the answer matching degree. The two can specifically be summated to calculate the first matching degree.
[0083] Respectively coding the Q&A contents can be effected by respectively inputting the Date Regue/Date Received 2022-06-29 contents in the Q&A model, and vector sequences are obtained through coding, the Q&A
model here can be embodied as a currently available one.
model here can be embodied as a currently available one.
[0084] In a preferred embodiment, when the historical examination question with the highest matching degree is selected, the relation amongst the question matching degree, the answer matching degree and the respective threshold of each historical examination question can be firstly judged, and any historical examination question whose question matching degree is greater than a first preset threshold and whose answer matching degree is greater than a second preset threshold is selected from the Q&A contents of all historical examination questions to serve as a historical examination question to be selected.
[0085] The next question of the historical examination question with the highest first matching degree is selected from the historical examination question(s) to be selected to serve as a candidate examination question.
[0086] Through the above mode it is made possible to eliminate the portion from historical Q&As deviating relatively greatly from the current examination question or answer, to thereby further ensure the degree of overall association between the selected candidate examination question and the current examination question.
[0087] In a preferred embodiment of the present application, the answer of the user can be scored, and the method further comprises:
[0088] coding the Q&A content of the current examination question according to a preset Q&A
model, and obtaining a question code and an answer code of the current examination question;
model, and obtaining a question code and an answer code of the current examination question;
[0089] calculating a third matching degree between the question code and the answer code of the current examination question;
[0090] calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm ¨ the fourth matching degree can be specifically calculated according to such algorithms as word segmentation comparison and keyword weighting, etc.; and
[0091] calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
[0092] Through the above method, the final score is incorporated with the matching degrees between the question and the answer and between the answer and the standard answer, Date Regue/Date Received 2022-06-29 whereby precision in score appraisal is enhanced.
[0093] In practice, it is further possible to take the score as a feature value in conjunction with a difference in lengths between the answer of the user and the standard answer, to be calculated together with a keyword matching degree of the standard answer to obtain the final score.
[0094] What follows is an example of a specific scene of the present application:
[0095] Examination question: I have received a short message to the effect that installation has been arranged for my TV set, but I have never applied for such installation.
[0096] Standard answer: Hello! It is seen here you have an installation of a Samsung TV set in Shenzhen City, with the time of installation as appointed being the seventh May, haven't anyone of your family member made the appointment?
[0097] Answer of the examinee: A query has been made here on your behalf, excuse me, was a Samsung TV set model No. QA82Q900RBJXXZ bought?
[0098] Automatic scoring and automatic question setting are performed according to the above scene:
[0099] The answer of the examinee is scored:
[0100] the model shown in Fig. 2 is firstly utilized to score the examination question and the standard answer for the first time, and a score 1->9 is obtained;
[0101] the model shown in Fig. 3 is then utilized to score the standard answer and the answer of the examinee for the second time, and a score 2->5 is obtained;
[0102] the model shown in Fig. 4 is thereafter utilized to score the two scores assigned with other feature values,
[0103] for instance, [score 1, score 2, difference in lengths, keyword matching rate, difference in numbers of sentences, difference in numbers of short sentences, jaccard similarity]
[0104] corresponding feature scores, e.g., [9, 5, 8, 0.5, 2, 1, 0.381 are sent to a scoring model to obtain the final score ->7.
[0105] Automatic question setting:
[0106] retrieval of similar scenes:
[0107] matching degrees are calculated for historical Q&As in a historical Q&A
database, and lo Date Regue/Date Received 2022-06-29 most matching historical Q&As are selected and determined as follows:
database, and lo Date Regue/Date Received 2022-06-29 most matching historical Q&As are selected and determined as follows:
[0108] user: Why I was told that installation had been arranged for my TV set, but I have never applied for such installation.
[0109] customer service: A query has been made on your behalf, excuse me, was a Samsung TV
set of this model bought?
set of this model bought?
[0110] thereafter the next sentence under the scene:
[0111] user: Yes, could you help me to see how cancellation can be made?
[0112] Comparison is made with the next standard question well set in the question bank, if the similarity is greater than a set domain value, the next sentence in the scene is taken to set the question to provide diversity to the examination questions, if it is lower than the domain value, the next standard examination question in the question bank is taken to set the question to prevent the examinee from deviating from the question.
[0113] Embodiment 2
[0114] Corresponding to the above method, embodiment 2 of the present application provides a device for automatically setting a question, as shown in Fig. 5, the device comprises:
[0115] a current examination question obtaining unit 51, for obtaining Q&A
content with respect to a current examination question;
content with respect to a current examination question;
[0116] a first matching degree calculating unit 52, for calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question;
[0117] a candidate examination question unit 53, for selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question;
[0118] a second matching degree calculating unit 54, for calculating a second matching degree between the candidate examination question and the next question in a question bank; and
[0119] a question setting unit 55, for taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
[0120] Preferably, the question setting unit 55 is further employed for taking the next question in Date Regue/Date Received 2022-06-29 the question bank to serve as a target examination question to set the question, if the second matching degree does not satisfy the preset condition.
[0121] Preferably, the first matching degree calculating unit includes a first coding unit, a question matching degree calculating unit, an answer matching degree calculating unit and a first matching degree calculating subunit, of which:
[0122] the first coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
[0123] the first coding unit is further employed for coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
[0124] the question matching degree calculating unit is employed for calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
[0125] the answer matching degree calculating unit is employed for calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and
[0126] the first matching degree calculating subunit is employed for calculating the first matching degree according to the question matching degree and the answer matching degree.
[0127] The candidate examination question unit is specifically employed for selecting any historical examination question whose question matching degree is greater than a first preset threshold and whose answer matching degree is greater than a second preset threshold from all Q&A contents of all historical examination questions to serve as a historical examination question to be selected, and selecting the next question of the historical examination question with the highest first matching degree from the historical examination question(s) to be selected to serve as a candidate examination question.
[0128] Preferably, the device further comprises:
[0129] a second coding unit, a third matching degree calculating unit, a fourth matching degree calculating unit and an answer score calculating unit, of which:
[0130] the second coding unit is employed for coding the Q&A content of the current examination Date Regue/Date Received 2022-06-29 question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
[0131] the third matching degree calculating unit is employed for calculating a third matching degree between the question code and the answer code of the current examination question;
[0132] the fourth matching degree calculating unit is employed for calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm; and
[0133] the answer score calculating unit is employed for calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
[0134] Embodiment 3
[0135] Corresponding to the above method and device, Embodiment 3 of the present application provides a computer system that comprises:
[0136] one or more processor(s); and
[0137] a memory, associated with the one or more processor(s), for storing a program instruction that executes the method as recited in Embodiment 1 when it is read and executed by the one or more processor(s).
[0138] Fig. 6 exemplarily illustrates the framework of a computer system that can specifically include a processor 1510, a video display adapter 1511, a magnetic disk driver 1512, an input/output interface 1513, a network interface 1514, and a memory 1520. The processor 1510, the video display adapter 1511, the magnetic disk driver 1512, the input/output interface 1513, the network interface 1514, and the memory 1520 can be communicably connected with one another via a communication bus 1530.
[0139] The processor 1510 can be embodied as a general CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuit(s) for executing relevant program(s) to realize the technical solutions provided by the present application.
[0140] The memory 1520 can be embodied in such a form as an ROM (Read Only Memory), an RAM (Random Access Memory), a static storage device, or a dynamic storage device. The memory 1520 can store an operating system 1521 for controlling the running of a computer Date Regue/Date Received 2022-06-29 system 1500, and a basic input/output system (BIOS) for controlling lower-level operations of the computer system 1500. In addition, the memory 1520 can also store a web browser 1523, a data storage administration system 1524, and an icon font processing system 1525, etc. The icon font processing system 1525 can be an application program that specifically realizes the aforementioned various step operations in the embodiments of the present application. To sum it up, when the technical solutions provided by the present application are to be realized via software or firmware, the relevant program codes are stored in the memory 1520, and invoked and executed by the processor 1510.
[0141] The input/output interface 1513 is employed to connect with an input/output module to realize input and output of information. The input/output module can be equipped in the device as a component part (not shown in the drawings), and can also be externally connected with the device to provide corresponding functions. The input means can include a keyboard, a mouse, a touch screen, a microphone, and various sensors etc., and the output means can include a display screen, a loudspeaker, a vibrator, an indicator light etc.
[0142] The network interface 1514 is employed to connect to a communication module (not shown in the drawings) to realize intercommunication between the current device and other devices. The communication module can realize communication in a wired mode (via USB, network cable, for example) or in a wireless mode (via mobile network, WIFI, Bluetooth, etc.).
[0143] The bus 1530 includes a passageway transmitting information between various component parts of the device (such as the processor 1510, the video display adapter 1511, the magnetic disk driver 1512, the input/output interface 1513, the network interface 1514, and the memory 1520).
[0144] Additionally, the computer system 1500 may further obtain information of specific collection conditions from a virtual resource object collection condition information database 1541 for judgment on conditions, and so on.
[0145] As should be noted, although merely the processor 1510, the video display adapter 1511, the magnetic disk driver 1512, the input/output interface 1513, the network interface 1514, the memory 1520, and the bus 1530 are illustrated for the aforementioned device, the device may further include other component parts prerequisite for realizing normal running during specific implementation. In addition, as can be understood by persons skilled in the art, the aforementioned device may as well only include component parts necessary for Date Regue/Date Received 2022-06-29 realizing the solutions of the present application, without including the entire component parts as illustrated.
[0146] As can be known through the description to the aforementioned embodiments, it is clearly learnt by person skilled in the art that the present application can be realized through software plus a general hardware platform. Based on such understanding, the technical solutions of the present application, or the contributions made thereby over the state of the art, can be essentially embodied in the form of a software product, and such a computer software product can be stored in a storage medium, such as an ROM/RAM, a magnetic disk, an optical disk etc., and includes plural instructions enabling a computer equipment (such as a personal computer, a server, or a network device etc.) to execute the methods described in various embodiments or some sections of the embodiments of the present application.
[0147] The various embodiments are progressively described in the Description, identical or similar sections among the various embodiments can be inferred from one another, and each embodiment stresses what is different from other embodiments.
Particularly, with respect to the system or system embodiment, since it is essentially similar to the method embodiment, its description is relatively simple, and the relevant sections thereof can be inferred from the corresponding sections of the method embodiment. The system or system embodiment as described above is merely exemplary in nature, units therein described as separate parts can be or may not be physically separate, parts displayed as units can be or may not be physical units, that is to say, they can be located in a single site, or distributed over a plurality of network units. It is possible to base on practical requirements to select partial modules or the entire modules to realize the objectives of the embodied solutions.
It is understandable and implementable by persons ordinarily skilled in the art without spending creative effort in the process.
Particularly, with respect to the system or system embodiment, since it is essentially similar to the method embodiment, its description is relatively simple, and the relevant sections thereof can be inferred from the corresponding sections of the method embodiment. The system or system embodiment as described above is merely exemplary in nature, units therein described as separate parts can be or may not be physically separate, parts displayed as units can be or may not be physical units, that is to say, they can be located in a single site, or distributed over a plurality of network units. It is possible to base on practical requirements to select partial modules or the entire modules to realize the objectives of the embodied solutions.
It is understandable and implementable by persons ordinarily skilled in the art without spending creative effort in the process.
[0148] The method of automatically setting a question, and the corresponding device and system provided by the present application have been explained in detailed above, specific examples are used in this paper to have enunciated the principle of the present application and the modes of execution, the descriptions to the above embodiments are merely meant to help understand the method and kernel conception of the present application; at the same time, persons ordinarily skilled in the art may make variations in both the specific modes of execution and the range of application on the basis of the spirits of the present application. In summary, the content of this Description shall not be understood as Date Regue/Date Received 2022-06-29 restrictive to the present application.
Date Regue/Date Received 2022-06-29
Date Regue/Date Received 2022-06-29
Claims (10)
1. A method of automatically setting a question, characterized in that the method comprises:
obtaining question and answer content with respect to a current examination question;
calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question;
selecting the next question of the historical examination question with the highest first matching degree to serve as a candidate examination question;
calculating a second matching degree between the candidate examination question and the next question in a question bank; and taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
obtaining question and answer content with respect to a current examination question;
calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question;
selecting the next question of the historical examination question with the highest first matching degree to serve as a candidate examination question;
calculating a second matching degree between the candidate examination question and the next question in a question bank; and taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
2. The method according to Claim 1, characterized in that the method further comprises:
taking the next question in the question bank to serve as the target examination question to set the question, if the second matching degree does not satisfy the preset condition.
taking the next question in the question bank to serve as the target examination question to set the question, if the second matching degree does not satisfy the preset condition.
3. The method according to Claim 1, characterized in that the step of calculating a first matching degree between the Q&A content of each historical examination question and the Q&A content of the current examination question includes:
coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
coding the Q&A content of each historical examination question according to the preset Q&A
model, and obtaining a question code and an answer code of each historical examination question;
calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and calculating the first matching degree according to the question matching degree and the answer matching degree.
coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
coding the Q&A content of each historical examination question according to the preset Q&A
model, and obtaining a question code and an answer code of each historical examination question;
calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and calculating the first matching degree according to the question matching degree and the answer matching degree.
4. The method according to Claim 3, characterized in that the step of selecting the next question of a historical examination question with the highest first matching degree to serve as a candidate examination question includes:
selecting any historical examination question whose question matching degree is greater than a first preset threshold and whose answer matching degree is greater than a second preset threshold from Q&A contents of all historical examination questions to serve as a historical examination question to be selected; and selecting the next question of the historical examination question with the highest first matching degree from the historical examination question(s) to be selected to serve as a candidate examination question.
selecting any historical examination question whose question matching degree is greater than a first preset threshold and whose answer matching degree is greater than a second preset threshold from Q&A contents of all historical examination questions to serve as a historical examination question to be selected; and selecting the next question of the historical examination question with the highest first matching degree from the historical examination question(s) to be selected to serve as a candidate examination question.
5. The method according to anyone of Claims 1 to 4, characterized in that the method further comprises:
coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
calculating a third matching degree between the question code and the answer code of the current examination question;
calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm;
and calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
calculating a third matching degree between the question code and the answer code of the current examination question;
calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm;
and calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
6. A device for automatically setting a question, characterized in that the device comprises:
a current examination question obtaining unit, for obtaining Q&A content with respect to a current examination question;
a first matching degree calculating unit, for calculating a first matching degree between the Q&A
content of each historical examination question and the Q&A content of the current examination questi on;
a candidate examination question unit, for selecting the next question of the historical examination question with the highest first matching degree to serve as a candidate examination questi on;
a second matching degree calculating unit, for calculating a second matching degree between the candidate examination question and the next question in a question bank; and a question setting unit, for taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
a current examination question obtaining unit, for obtaining Q&A content with respect to a current examination question;
a first matching degree calculating unit, for calculating a first matching degree between the Q&A
content of each historical examination question and the Q&A content of the current examination questi on;
a candidate examination question unit, for selecting the next question of the historical examination question with the highest first matching degree to serve as a candidate examination questi on;
a second matching degree calculating unit, for calculating a second matching degree between the candidate examination question and the next question in a question bank; and a question setting unit, for taking the candidate examination question to serve as a target examination question to set the question, if the second matching degree satisfies a preset condition.
7. The device according to Claim 6, characterized in that the question setting unit is further employed for taking the next question in the question bank to serve as the target examination question to set the question, if the second matching degree does not satisfy the preset condition.
8. The device according to Claim 6, characterized in that the first matching degree calculating unit includes a first coding unit, a question matching degree calculating unit, an answer matching degree calculating unit and a first matching degree calculating subunit, of which:
the first coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
the first coding unit is further employed for coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
the question matching degree calculating unit is employed for calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
the answer matching degree calculating unit is employed for calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and the first matching degree calculating subunit is employed for calculating the first matching degree according to the question matching degree and the answer matching degree.
the first coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
the first coding unit is further employed for coding the Q&A content of each historical examination question according to the preset Q&A model, and obtaining a question code and an answer code of each historical examination question;
the question matching degree calculating unit is employed for calculating a question matching degree between the question code of each historical examination question and the question code of the current examination question;
the answer matching degree calculating unit is employed for calculating an answer matching degree between the answer code of each historical examination question and the answer code of the current examination question; and the first matching degree calculating subunit is employed for calculating the first matching degree according to the question matching degree and the answer matching degree.
9. The device according to Claims 6 to 8, characterized in that the device further comprises:
a second coding unit, a third matching degree calculating unit, a fourth matching degree calculating unit and an answer score calculating unit, of which:
the second coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
the third matching degree calculating unit is employed for calculating a third matching degree between the question code and the answer code of the current examination question;
the fourth matching degree calculating unit is employed for calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm; and the answer score calculating unit is employed for calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
a second coding unit, a third matching degree calculating unit, a fourth matching degree calculating unit and an answer score calculating unit, of which:
the second coding unit is employed for coding the Q&A content of the current examination question according to a preset Q&A model, and obtaining a question code and an answer code of the current examination question;
the third matching degree calculating unit is employed for calculating a third matching degree between the question code and the answer code of the current examination question;
the fourth matching degree calculating unit is employed for calculating a fourth matching degree between an answer to the current examination question and a standard answer to the current examination question according to a text similarity algorithm; and the answer score calculating unit is employed for calculating a score of the answer to the current examination question according to the third matching degree and the fourth matching degree.
10. A computer system, characterized in comprising:
one or more processor(s); and a memory, associated with the one or more processor(s), for storing a program instruction that performs operations as recited in anyone of Claims 1 to 5 when it is read and executed by the one or more processor(s).
one or more processor(s); and a memory, associated with the one or more processor(s), for storing a program instruction that performs operations as recited in anyone of Claims 1 to 5 when it is read and executed by the one or more processor(s).
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911400187.3A CN111159379B (en) | 2019-12-30 | 2019-12-30 | Automatic question setting method, device and system |
| CN201911400187.3 | 2019-12-30 | ||
| PCT/CN2020/111954 WO2021135322A1 (en) | 2019-12-30 | 2020-08-28 | Automatic question setting method, apparatus and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA3166342A1 true CA3166342A1 (en) | 2021-07-08 |
Family
ID=70559482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3166342A Pending CA3166342A1 (en) | 2019-12-30 | 2020-08-28 | Automatic question setting method, apparatus and system |
Country Status (3)
| Country | Link |
|---|---|
| CN (1) | CN111159379B (en) |
| CA (1) | CA3166342A1 (en) |
| WO (1) | WO2021135322A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111159379B (en) * | 2019-12-30 | 2022-12-20 | 苏宁云计算有限公司 | Automatic question setting method, device and system |
| CN113761832A (en) * | 2021-08-05 | 2021-12-07 | 联想(北京)有限公司 | Information processing method, device and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7606714B2 (en) * | 2003-02-11 | 2009-10-20 | Microsoft Corporation | Natural language classification within an automated response system |
| US9471872B2 (en) * | 2012-06-29 | 2016-10-18 | International Business Machines Corporation | Extension to the expert conversation builder |
| CN107220380A (en) * | 2017-06-27 | 2017-09-29 | 北京百度网讯科技有限公司 | Question and answer based on artificial intelligence recommend method, device and computer equipment |
| CN109255697A (en) * | 2018-08-15 | 2019-01-22 | 普信恒业科技发展（北京）有限公司 | A kind of automatic credit estimation method and system based on artificial intelligence |
| CN109697228A (en) * | 2018-12-13 | 2019-04-30 | 平安科技（深圳）有限公司 | Intelligent answer method, apparatus, computer equipment and storage medium |
| CN109977208B (en) * | 2019-03-22 | 2021-04-09 | 北京中科汇联科技股份有限公司 | Dialogue system integrating FAQ (failure-based query language) and task and active guidance |
| CN110110066B (en) * | 2019-05-09 | 2023-01-06 | 腾讯科技（深圳）有限公司 | Interactive data processing method and device and computer readable storage medium |
| CN111159379B (en) * | 2019-12-30 | 2022-12-20 | 苏宁云计算有限公司 | Automatic question setting method, device and system |
-
2019
- 2019-12-30 CN CN201911400187.3A patent/CN111159379B/en active Active
-
2020
- 2020-08-28 CA CA3166342A patent/CA3166342A1/en active Pending
- 2020-08-28 WO PCT/CN2020/111954 patent/WO2021135322A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021135322A1 (en) | 2021-07-08 |
| CN111159379B (en) | 2022-12-20 |
| CN111159379A (en) | 2020-05-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230274205A1 (en) | Multi computational agent performance of tasks | |
| US10311856B2 (en) | Synthesized voice selection for computational agents | |
| US10853747B2 (en) | Selection of computational agent for task performance | |
| JP7316453B2 (en) | Object recommendation method and device, computer equipment and medium | |
| US10489838B2 (en) | Method for using customer attributes to select a service representative | |
| JP2020521376A (en) | Agent decisions to perform actions based at least in part on image data | |
| US20160026643A1 (en) | Presenting suggested facets | |
| CN110070076B (en) | Method and device for selecting training samples | |
| CA3166342A1 (en) | Automatic question setting method, apparatus and system | |
| CN113407850B (en) | Method and device for determining and acquiring virtual image and electronic equipment | |
| US11635883B2 (en) | Indication of content linked to text | |
| KR20190075310A (en) | Electronic device and method for providing information related to phone number | |
| US20180096284A1 (en) | Multi computational agent performance of tasks | |
| US10623578B2 (en) | Computer system, method for providing API, and program | |
| WO2021104274A1 (en) | Image and text joint representation search method and system, and server and storage medium | |
| CN109829117A (en) | Method and apparatus for pushed information | |
| JP7362996B2 (en) | Speech recognition method and device | |
| CN114186039A (en) | Visual question answering method and device and electronic equipment | |
| CN109190048B (en) | Wearing object recommendation method and device, electronic equipment and storage medium | |
| US10657692B2 (en) | Determining image description specificity in presenting digital content | |
| JP2021162997A (en) | Information processing device and information processing method | |
| CN112308642A (en) | House source searching method, terminal and server | |
| JP6813880B2 (en) | Computer systems, methods, and computer programs for determining the suitability of users and content | |
| KR101699478B1 (en) | Server for analyzing naming and method for analyzing the same | |
| US20170169483A1 (en) | Method for pushing and displaying information and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20220629 |
|
| EEER | Examination request |
Effective date: 20220629 |
|
| EEER | Examination request |
Effective date: 20220629 |
|
| EEER | Examination request |
Effective date: 20220629 |
|
| EEER | Examination request |
Effective date: 20220629 |