CN107705231B - Computer-aided paper marking method and device and computer-readable storage medium - Google Patents
Computer-aided paper marking method and device and computer-readable storage medium Download PDFInfo
- Publication number
- CN107705231B CN107705231B CN201711081247.0A CN201711081247A CN107705231B CN 107705231 B CN107705231 B CN 107705231B CN 201711081247 A CN201711081247 A CN 201711081247A CN 107705231 B CN107705231 B CN 107705231B
- Authority
- CN
- China
- Prior art keywords
- test paper
- test
- paper
- answers
- question
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
Abstract
The invention provides a computer-aided marking method, a computer-aided marking system and a computer-readable storage medium. By utilizing the marking method and the marking system, a computer objective judgment mechanism can be introduced in the marking process of the subjective questions, so that the marking process of the subjective questions can be carried out with the assistance of a computer; in the present invention, the score of the test paper of the testee is not only dependent on the answer submitted by the testee, but also dynamically changes according to the answers of the test papers submitted by all other testees. In addition, the invention also provides a feedback mechanism, thereby avoiding the occurrence of misjudgment in the case of computer-aided scoring, improving the scoring speed and ensuring the accuracy of scoring results.
Description
Technical Field
The invention belongs to the technical field of computer assistance, and particularly relates to a computer-assisted scoring method and device and a computer-readable storage medium.
Background
Currently, one's written translation ability is evaluated, primarily by scoring it through various test means. For example, an examination scoring method using a combination of subjective questions and objective questions is used. For objective questions, the candidate can score only by selecting a correct candidate item, and the scoring work can be completed through machine statistics testing; for subjective questions, different translators give different translations for the same item to be translated, and the reference answers given by a question maker are usually limited, so that misjudgment is easy to generate if machine statistics is adopted, because the different translation results given by different translators are possibly different from the reference answers, but are all correct answers. At this time, means of manual review are typically introduced. However, when the number of the test paper is large, if each test paper is manually evaluated, the test workload is obviously increased, and the test cost is increased.
Even in the case of manual scoring, there is a possibility that erroneous judgment may occur due to subjective tendency and carelessness of the reader. Further, in the subjective question test, since the answers are not unified, one test question usually gives a plurality of reference answers; it is also possible for the examinee to give correct answers in addition to the plurality of reference answers when actually answering the question. The accuracy of the answers is very different, and if the scores are judged only by mistake or correctness, the ability of distinguishing different testees cannot be further realized; similarly, if the overall answer sheet condition of the tested population on the same subjective question is not considered, the real level of the tested person cannot be reflected only by grading each test sheet in an isolated manner.
Disclosure of Invention
The invention provides a computer-aided paper marking method, a computer-aided paper marking system and a computer-readable storage medium, aiming at solving the problems that the manual paper marking efficiency is low and the capability of a tested person cannot be accurately evaluated by a static isolated marking mechanism in the prior art.
By utilizing the marking method and the marking system, a computer objective judgment mechanism can be introduced in the marking process of the subjective questions, so that the marking process of the subjective questions can be carried out with the assistance of a computer; further, in the technical solution of the present invention, the score of each test paper is not only judged by the answer submitted by the test paper, but is dynamically changed according to the overall level of all test papers, specifically, is measured according to the unity or weight of the answers given by all test papers to the same tested item, so that the level of the tested person can be more accurately distinguished.
In other words, unlike the existing paper evaluation method and technology, when a test paper is submitted by a certain tested person, the score of the test paper is actually determined, in the present invention, when the test paper is completed by a certain test person, the score of the test paper is actually in a pending state, and only after the answer of the test questions of all other test persons are collected and summarized simultaneously, the score of the test paper of each individual test person can be finally determined, i.e. the method of the present invention is a dynamic paper evaluation mechanism based on all the test paper data, and is not a traditional static mechanism.
In a first aspect of the invention, a computer-aided examination paper marking method is provided, which comprises an examination paper pre-evaluation stage, an examination paper re-evaluation stage and a feedback rechecking stage; in the test paper pre-evaluation stage, evaluating answers of a first type of test questions of the current test paper to obtain a first score in real time; if the first score meets a first preset standard, entering a test paper reevaluation stage;
and in the test paper reevaluation stage, analyzing the answers of the second type of test questions of the current test paper, and obtaining a second score of the current test paper by combining the analysis results of the answers of the second type of test questions of all other test papers entering the test paper reevaluation stage in a preset time period and/or a preset number.
That is, after the current testee submits his own test paper, the review system cannot immediately obtain the score of the test paper submitted by the testee, because in the present invention, the score of the test paper of the testee not only depends on the answer submitted by the testee, but also dynamically changes according to the answers submitted by all other testees.
As a feedback mechanism, if the second score meets a second predetermined criterion, a feedback review phase is entered.
The first type of test questions are test questions with only one kind of reference answers and comprise single-item selection questions, multiple-item selection questions, indefinite-item selection questions and judgment questions; the second type of test questions is test questions with at least one kind of reference answer. Those skilled in the art often refer to objective and subjective questions.
Preferably, the second type of test questions are subjective translation questions, the subjective translation questions comprise a plurality of items to be translated, a plurality of different reference answer sets are given in advance for each item to be translated, and each reference answer in the reference answer sets comprises a plurality of keywords; defining a first flag value for each reference answer; for each item to be translated, a second flag value is defined for each test paper to be read.
In the specific paper evaluation process, in the second paper evaluation stage, the second score of the current test paper is obtained according to the following steps:
(1) acquiring a translation result submitted by the current examination paper for an item to be translated, and matching the submitted translation result with each reference answer in the reference answer set;
(2) if at least one keyword is successfully matched, taking the reference answer with the highest matching degree with the translation result submitted by the current test paper in the reference answer set as a benchmark item of the translation result submitted by the current test paper, and taking the number of the successfully matched keywords as a benchmark score of the current test paper for the item to be translated; meanwhile, updating the first mark value of the reference answer which is the highest in matching degree with the translation result submitted by the current test paper in the reference answer set;
(3) if the relevant keyword of the translation result given by the current test paper is not successfully matched with any keyword of any reference answer in the reference answer set, updating the second mark value;
(4) obtaining the translation result submitted by the next test paper aiming at the item to be translated, and repeating the steps (1) - (3) until all the translation results submitted by the test paper aiming at the item to be translated have executed the steps (1) - (3);
(5) acquiring the current first mark value of each reference answer in the reference answer set, the second mark values of all test papers for the current item to be translated, and the reference items of all test papers for the item to be translated;
(6) and calculating second scores of all the test papers for the item to be translated.
Further, the step of entering the feedback review includes performing manual review on answers of the second type of test questions of the current test paper.
Although subjective question marking can be realized through the invention, the processes of keyword extraction/analysis, matching and the like are not as accurate as objective question comparison after all, and in order to avoid misjudgment, if the objective question score of the tested person is higher, the subjective question score is lower, the answer of the tested person is possibly misjudged, or the tested person gives a non-uniform translation result, and a manual marking mechanism is introduced at the moment.
It will be appreciated that the introduction of this mechanism does not increase the manual scoring workload, since this should be a small number, but the answers given by some of the testees should be within the keyword range of the reference answers, since the test designer typically gives enough reference answers for each subjective question.
In a second aspect of the present invention, there is provided a computer-aided paper marking device, comprising an objective question and paper judging system, a test paper screening system, a subjective question and paper judging system, a review screening system and a manual review system:
the objective question appraising system is used for judging answers of objective question parts of the current test paper and obtaining objective question scores of the current test paper in real time;
the test paper screening system is used for screening the test paper with objective question score meeting a first condition and sending the test paper into the subjective question appraising system;
the subjective question judging system is used for obtaining the subjective question scores of all the test papers by combining the analysis results of the subjective question answers of all other test papers entering the subjective question judging system in a preset time period and/or a preset number;
the review screening system is used for screening the test paper with the subjective question score meeting the second condition and sending the test paper into the manual review system;
the manual review system is used for taking manual intervention measures for subjective question parts of test papers entering the manual review system.
Specifically, the subjective question judging system collects all other input answers for each subjective question in test papers entering the subjective question paper judging system, extracts keywords from the input answers, compares the extracted keywords with a plurality of reference answers corresponding to each subjective question, and obtains a subjective question score of each test paper according to a comparison result.
In a concrete implementation, the marking device further comprises at least one database, and the database stores objective question reference answers and subjective question reference answers; wherein each objective question only has one corresponding reference answer; each subjective question corresponds to a plurality of reference answers.
Further, the review screening system is configured to screen out test paper with a subjective question score meeting a second condition, and send the test paper to the manual review system, and further includes: and if the subjective question score of the current test paper is lower than the second standard threshold value and the objective question score is higher than the first standard threshold value, the test paper is sent to a manual review system.
In a third aspect of the present invention, there is also provided a computer-readable storage medium having stored thereon computer-executable instructions; the executable instructions are executed by a processor and a memory, so as to realize the scoring method.
Drawings
FIG. 1 is a flow chart of the scoring method according to the present invention.
Fig. 2 is a frame diagram of the scoring device according to the present invention.
DETAILED DESCRIPTION OF EMBODIMENT (S) OF INVENTION
In this embodiment, the examination papers to be read are all the examination papers submitted by the translation workers after taking a certain examination. The test paper comprises a subjective question part and an objective question part. The purpose of the exam is to screen out those candidates with higher translation levels.
For the objective question part, the prior art can realize the online examination paper reading of a computer and obtain scores in real time; however, for subjective questions, since different translators give different translations for the same item to be translated, and the reference answers given by the questioner are usually limited, if machine statistics is used, misjudgment is easily generated, because it is likely that different translation results given by different translators may be different from the reference answers, but the translation results are all correct answers. At this time, means of manual review are typically introduced.
However, when the number of the test paper is huge, if each test paper is manually evaluated, the test workload is obviously increased, and the test cost is increased; even if a small number of test papers are individually tested, due to the difference in level of the examiners and the cognitive problem, it is possible to give a wrong judgment to the test paper on the translation result beyond the conventional reference answer but actually a sufficiently skillful translation result, thereby failing to correctly evaluate the test paper's ability.
The inventors consider that the evaluation criteria for translation work are "confidence", "reach", and "elegance" in order from low to high. So-called "letter", i.e. translation is accurate; so-called "to", require a thorough idea; the "elegant" is required to be rich in culture.
On one hand, for a qualified translation job, the capability of ensuring the accuracy of the translated text should be firstly ensured, namely the most basic 'confidence' standard is reached; on the other hand, for a good translation worker, the standard of "reach" and "ya" is achieved, and the stage of "confidence" is necessarily passed, so that the test paper which can achieve the standard of "reach" and "ya" should be selected, and should also be selected from the test paper which passes the objective problem test and achieves a certain score.
Therefore, as shown in fig. 1, in the review method of the present embodiment, firstly, in a test paper pre-review stage, the answers of the first type of test questions of the current test paper are reviewed, and a first score is obtained in real time; if the first score meets a first preset standard, entering a test paper reevaluation stage;
in the test paper reevaluation stage, analyzing the answers of the second type of test questions of the current test paper, and obtaining a second score of the current test paper by combining the analysis results of the answers of the second type of test questions of all other test papers entering the test paper reevaluation stage in a preset time period and/or a preset number;
and entering a feedback review stage if the second score meets a second predetermined criterion.
Fig. 2 shows a frame diagram of an examination paper reading device according to the present invention.
The following scenarios and examples can better illustrate the specific implementation of the present invention in conjunction with fig. 1 and 2.
In the conventional test, various reference answers are given to subjective questions, which basically only pursue the 'confidence' standard, but less answers are given to the 'reach' and 'ya' standards. Of course, in practice, no answer to the "reach" or "ya" criteria is given in advance, since the linguistic expressions vary from person to person.
On this premise, a plurality of keywords included in each translation result in the translation result set all represent "belief", and cannot represent "dada" or "ya".
For example, the item to be translated is
(T1)I look for what I miss, I know not what it is, I feel so sad, so drear, so lonely, without cheer.
The pre-given reference answers may be:
(A1) i look for my thoughts, I do not know what it is, I feel so sad, so cloudy, so solitary, and do not get cheerful.
(A2) I find what I think of, I do not know what, I feel very sick, so it is bored, so alone, do not cheer.
Both are the standard translations of "belief" in the traditional sense, and the keywords are not like "sadness", "love", "thinking", "lonely", "cheering", etc.;
however, the following translations meet the criteria of Dayan:
(A3) searching for foraging, cold and clear, and extracting bitter chiffon.
For another example, a certain line of the movie "Flipped" is used as the item to be translated:
(T2)Some of us get dipped in flat、some in satin、some in gloss. But every once in a while you find someone who's iridescent、and when you do、nothing will ever compare.
translation of its reference answers, although accurate, is trivial:
(A1) some of us bubble water in apartments. Some are satin and some are glossy. But at intervals you find that one is irish and when you do so, nothing is more important than this.
However, a certain test paper may give a better answer as follows:
(A2) people live in high buildings, have deep ditches at places, have all the people, have rust as a whole, have thousands of species of old and useless people, and have the rainbow which is known to the people.
However, the more elegant answers described above do not usually appear in a plurality of different sets of reference answers that are given in advance; nor does it contain any of the keywords of the standard translations described previously. If the subjective question marking judgment is carried out by only adopting the technology of computer key word matching, the misjudgment is likely to be generated.
For example, for the aforementioned item to be translated T1, the best answer is A3, but none of the keywords in A3 match the reference (A1-A2). At this time, if the score is given by the matching degree, the score of a3 is likely to be 0. Obviously, such test paper needs manual examination, but how to accurately and effectively identify the test paper is one of the important problems solved by the invention.
The following examples specifically show the corresponding processing modes in the subjective question review stage.
First, in the database of the marking device, for the item T to be translated1={t1,t2,t3,……，tnPrestoring to-be-translated item T1Corresponding k translation result sets A1，A2，……Ak。
At the same time, for each given translation result A1，A2，……AkDefining a first flag value, said first flag values each initialized to 0;
for each item to be translated, defining a second mark value for each examination paper to be read, wherein the second mark value is initialized to 0; after all the test papers complete the subjective question test, the test paper of each tester is scored according to the following steps:
(1) acquiring a translation result submitted by the current test paper aiming at the item to be translated, and comparing the submitted translation result with each translation result in the translation result set to obtain a reference item and a reference score of the submitted translation result;
specifically, the relevant keywords of the submitted translation result are extracted and matched with the keywords of each translation result in the translation result set, so as to obtain a reference entry; the number of the successfully matched keywords is used as a benchmark score;
for example, the current test paper is for the entry to be translated T1={t1,t2,t3,……，tnGive the translation result E1={e1,e2,e3,……，erIn which e1,e2,e3,……，er isComposition sentence E1The related keywords of (1); then e will be1,e2,e3,……，erRespectively corresponding to k translation results A1，A2，……AkComparing the keywords of each; statistics E1And k translation results A1，A2，……AkThe matching number of the keywords is sorted;
(2) if at least one keyword is successfully matched, taking the translation result with the highest matching degree with the translation result submitted by the current test paper in the translation result set as a reference item of the translation result submitted by the current test paper, and taking the number of the keywords successfully matched as a reference score of the current test paper for the item to be translated;
meanwhile, updating the first mark value of the translation result which is the highest in matching degree with the translation result submitted by the current test paper in the translation result set;
for example, if AjAnd E1The number of matched related keywords is the maximum, Y, AjPlus 1, the first flag value of; marking the reference score of the current test paper for the item to be translated as Y;
if the relevant keyword of the translation result given by the current test paper is not successfully matched with any keyword of any translation result in the translation result set, for example, the test paper gives other more elegant answers different from the traditional standard answers, and the answer does not contain any traditional keyword, updating the second mark value; specifically, the second flag value of the current test paper for the item to be translated is increased by 1;
(3) obtaining the translation result submitted by the next test paper aiming at the item to be translated, and repeating the steps (1) - (2) until all the translation results submitted by the test paper aiming at the item to be translated have executed the steps (1) - (2);
(4) acquiring the current first marker value of each translation result in the translation result set, the second marker values of all test papers for the current item to be translated, and reference items of all test papers for the item to be translated;
(5) and calculating the final scores of all the test papers for the item to be translated.
Specifically, when the m-bit test paper is aimed at the item to be translated T1After the submitted translation results are all subjected to the steps, the translation result A given for each item to be translated can be obtained1，A2，……AkFirst flag value X of1,X2,……，Xk(ii) a m-bit test paper for item to be translated T1Of said second flag value Z1，Z2，……ZmAnd m-bit test paper is aimed at item to be translated T1The resulting base is compared to base entries (Yr, As), where YrIs the base score determined in the aforementioned step (2), AsIs the reference item determined according to the step (2);
at this time, the m-th test paper has a final score of Yr × Xs for the item to be translated T1, where Xs is the first label value of the reference item As.
To better illustrate the above scoring process, the following table is given as an example of the calculation process:
as just one example, in the embodiment, the test paper is provided by 10 testees, and a translation result set given by a certain item to be translated includes four translation results (A1-A4) as reference answers. The four translation results are all the correct translation results of the item to be translated. In general, to ensure the universality of the reference answers, different sentence patterns are usually adopted for the four translation results, different translation words are adopted for the same word to be translated, proper expansion is performed, and the like, so that the four translation results express the same meaning but are different in word expression.
Taking the above table as an example, four translation results a1-a4 are given for a certain item to be translated.
(1) And acquiring a translation result of the first test paper. Through matching, the answer given by the first test paper has the highest matching degree with A4, and the number of matched keywords is 6. At this time, A4 is selected as a reference item of the first test paper, and the reference of the first test paper is divided into 6 points; then, the first flag value is updated, and since a4 is selected as the reference entry, only the first flag value of a4 is updated to 1, and the first flag values of the other translation results a1-A3 remain 0; at this time, the second flag value is not updated;
(2) and acquiring the translation result of the second test paper. Through matching, the answer given by the second test paper is still the highest in matching degree with A4, and the number of matched keywords is 6. At this time, A4 is selected as a reference item of the second test paper, and the reference of the second test paper is divided into 6 points; then, the first flag value is updated, and since a4 is selected as the reference entry again, the first flag value of a4 is updated to 2 again, and the first flag values of the other translation results a1-A3 remain 0; at this time, the second flag value is not updated;
(3) and acquiring a translation result of the third test paper. Through matching, the answer of the third test paper and the keywords of A1-A4 cannot be matched, i.e. the number of matched keywords is zero. At this time, updating the second flag value to be 1 also means that the third test paper has no reference entry and no reference score;
(4) and acquiring a translation result of the fourth test paper. Through matching, the answer given by the fourth test paper has the highest matching degree with A1, and the number of matched keywords is 5. At this time, A1 is selected as a reference item of the fourth test paper, and the reference of the fourth test paper is divided into 5 points; then, the first flag value is updated, and since a1 is selected as the reference entry, the first flag value of a1 is updated to 1. At this time, the first marker value of A4 is still 2, and the first marker values of the other translation results A2-A3 are still 0; at this time, the second flag value is not updated;
by analogy, the translation results of the remaining fifth to tenth test papers are obtained one by one, and the specific number of the matched keywords, the reference items, the reference scores, the first mark values and the second mark values are updated as shown in the remaining part of the table.
After all the above steps are completed, at this time, according to the last row of the table, the first flag values corresponding to the four translation results a1-a4 are 2, 4, 0 and 3, respectively;
meanwhile, the reference item, the reference score, the second marking value and the final total score of the current item to be translated of the 10-bit test paper are respectively as follows:
as can be seen from the above process, in the scoring scheme of the present invention, unlike the conventional scoring method: the achievement of each test paper is not only determined by the answer submitted by the test paper, but can not be known until the achievements of all test papers are counted.
In addition, since this stage is directed to subjective question testing, in the conventional method, manual examination of all test papers is required; however, the above process of the present invention can be performed automatically by a computer to output the final score directly.
This is, of course, not the full purpose of the invention. Another purpose of this scheme is to alleviate manual paper marking work load.
Since there is an objective problem testing stage before the subjective problem testing. The person who passes the objective test can enter the subjective test stage. Therefore, the test paper in the evaluation stage of the subjective questions is submitted by the personnel with certain translation capability, and the objective questions can be improved to pass the standard and the like for better screening.
Normally, for an item to be translated, a person with certain translation capability can give a certain translation result (without the answer submitted), and the translation result at least can contain one or more keywords and can be matched with at least one of the preset reference answers A1-A4.
If the person who passes the objective question test does not match any of the keywords of the predetermined reference answers a1-a4 with the translation result given by the person for one or more items to be translated in the subjective question stage, it is likely that: the answer given by the paper is not the traditional answer that only stays in the 'letter' standard, but has reached the 'da', 'ya' standard.
At this point, we have reason to believe that the test paper has translation capabilities beyond that of the average person. However, if a machine matching method is employed, the translation score of the test paper entry would be 0, e.g., the third test paper in the previous example.
When this occurs, we need to do the manual scoring phase. In the above example, with our scheme, only the third test paper of 10 test papers needs to be manually reviewed, and the screening rate reaches 90%.
Of course, the above example is only a simple illustration, and the sample size of 10 samples is actually too small to show the advantages of the present invention.
In an actual scenario, the sample size is of course more than that. This is also a statistically valid embodiment of the present invention. The larger the sample size, the more rational the above protocol. Because according to the theorem of large numbers, if the number of test papers exceeds a certain degree, the more random the distribution of answers submitted by the test papers is, the more stable the number of test papers reaching the standard of elegance is, the certain proportion.
In addition, when the sample size is larger, the statistical result is more and more consistent with the normal distribution, that is, the two ends are small, the middle is large, and the statistical result is embodied in the invention, which means that the test paper with the second mark value of 1 should only occupy a small part.
Also, the above embodiment only illustrates the case where 10 test papers submit one question (actually one sentence translation) of answer. In practice, the subjective question test has to be performed on more than one question, the reference answer has to be performed on more than one sentence, and the keyword data of the answer to be submitted is increased, which increases the sample size. The larger the sample size is, the more obvious the statistical rule of the invention is, the more advantages are achieved, because according to the traditional testing method, the larger the sample size is, and the workload of (manual examination paper reading) is larger. In the invention, all the steps are automatically completed by a computer statistical module, and a large sample size is exactly needed to ensure the accuracy. The above is merely a simple illustration and example, for example, it is not necessary to enter manual scoring in the case of 0 matching, and a ratio may be set according to the sample size.
Claims (8)
1. A computer-aided examination paper marking method comprises an examination paper pre-evaluation stage, an examination paper re-evaluation stage and a feedback rechecking stage;
in the test paper pre-evaluation stage, evaluating answers of a first type of test questions of the current test paper to obtain a first score in real time; if the first score meets a first preset standard, entering a test paper reevaluation stage;
the method is characterized in that:
in the test paper reevaluation stage, analyzing the answers of the second type of test questions of the current test paper, and obtaining a second score of the current test paper by combining the analysis results of the answers of the second type of test questions of all other test papers entering the test paper reevaluation stage in a preset time period and/or a preset number;
if the second score meets a second preset standard, entering a feedback review stage;
the second type of test questions are subjective translation questions containing a plurality of items to be translated, and a plurality of different reference answer sets are given in advance for each item to be translated; each reference answer in the reference answer set comprises a plurality of key words; defining a first flag value for each reference answer; for each item to be translated, defining a second mark value for each test paper to be read;
in the examination paper reevaluation stage, a second score of the current examination paper is obtained according to the following steps:
(1) acquiring a translation result submitted by the current examination paper for an item to be translated, and matching the submitted translation result with each reference answer in the reference answer set;
(2) if at least one keyword is successfully matched, taking the reference answer with the highest matching degree with the translation result submitted by the current test paper in the reference answer set as a benchmark item of the translation result submitted by the current test paper, and taking the number of the successfully matched keywords as a benchmark score of the current test paper for the item to be translated; meanwhile, updating the first mark value of the reference answer which is the highest in matching degree with the translation result submitted by the current test paper in the reference answer set;
(3) if the relevant keyword of the translation result given by the current test paper is not successfully matched with any keyword of any reference answer in the reference answer set, updating the second mark value;
(4) obtaining the translation result submitted by the next test paper aiming at the item to be translated, and repeating the steps (1) - (3) until all the translation results submitted by the test paper aiming at the item to be translated have executed the steps (1) - (3);
(5) acquiring the current first mark value of each reference answer in the reference answer set, the second mark values of all test papers for the current item to be translated, and the reference items of all test papers for the item to be translated;
(6) and calculating second scores of all the test papers for the item to be translated.
2. The method of claim 1, wherein: the first type of test questions are test questions with only one kind of reference answers and comprise single-item selection questions, multiple-item selection questions, indefinite-item selection questions and judgment questions; the second type of test questions is test questions with at least one reference answer.
3. The method of claim 1, wherein: and the stage of entering the feedback rechecking comprises the step of manually marking the answers of the second type of test questions of the current test paper.
4. A computer-aided paper marking device for implementing the paper marking method according to any one of claims 1 to 3, comprising an objective question paper judging system, a test paper screening system, a subjective question paper judging system, a review screening system and a manual review system;
the objective question appraising system is used for judging answers of objective question parts of the current test paper and obtaining objective question scores of the current test paper in real time;
the test paper screening system is used for screening the test paper with objective question score meeting a first condition and sending the test paper into the subjective question appraising system;
the method is characterized in that:
the subjective question judging system is used for obtaining the subjective question scores of all the test papers by combining the analysis results of the subjective question answers of all other test papers entering the subjective question judging system in a preset time period and/or a preset number;
the review screening system is used for screening the test paper with the subjective question score meeting the second condition and sending the test paper into the manual review system;
the manual review system is used for taking manual intervention measures for subjective question parts of test papers entering the manual review system.
5. The apparatus of claim 4, wherein:
the subjective question judging system collects all other input answers aiming at each subjective question in test papers entering the subjective question judging system, key word extraction is carried out on the input answers, the extracted key words are compared with a plurality of reference answers corresponding to each subjective question, and the subjective question score of each test paper is obtained according to the comparison result.
6. The apparatus of claim 4, wherein:
the system also comprises at least one database, wherein the database stores objective question reference answers and subjective question reference answers; wherein each objective question only has one corresponding reference answer; each subjective question corresponds to a plurality of reference answers.
7. The apparatus of any of claims 4-6, wherein:
the review screening system is used for screening the test paper with the subjective question score meeting the second condition and sending the test paper into the manual review system, and further comprises: and if the subjective question score of the current test paper is lower than the second standard threshold value and the objective question score is higher than the first standard threshold value, the test paper is sent to a manual review system.
8. A computer-readable storage medium having stored thereon computer-executable instructions; executing the executable instructions by a processor and a memory for implementing the scoring method of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711081247.0A CN107705231B (en) | 2017-11-07 | 2017-11-07 | Computer-aided paper marking method and device and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711081247.0A CN107705231B (en) | 2017-11-07 | 2017-11-07 | Computer-aided paper marking method and device and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107705231A CN107705231A (en) | 2018-02-16 |
| CN107705231B true CN107705231B (en) | 2021-08-20 |
Family
ID=61176647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711081247.0A Active CN107705231B (en) | 2017-11-07 | 2017-11-07 | Computer-aided paper marking method and device and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107705231B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108446277B (en) * | 2018-03-27 | 2021-08-17 | 北京大前科技有限责任公司 | Method and device for simulating learning |
| CN110705278A (en) * | 2018-07-09 | 2020-01-17 | 北大方正集团有限公司 | Subjective question marking method and subjective question marking device |
| CN109359849A (en) * | 2018-10-09 | 2019-02-19 | 上海起作业信息科技有限公司 | Information processing method, device, medium and electronic equipment |
| CN113256252A (en) * | 2021-05-25 | 2021-08-13 | 上海金程教育培训有限公司 | B/S architecture-based test system |
| CN116610774B (en) * | 2023-07-20 | 2023-09-26 | 河北鑫考科技股份有限公司 | High-efficiency intelligent online paper reading auxiliary method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942994A (en) * | 2014-04-22 | 2014-07-23 | 济南大学 | Computer assessment method for subjective questions |
| CN106940788A (en) * | 2017-03-07 | 2017-07-11 | 百度在线网络技术（北京）有限公司 | Intelligent scoring method and device, computer equipment and computer-readable medium |
| CN107240394A (en) * | 2017-06-14 | 2017-10-10 | 北京策腾教育科技有限公司 | A kind of dynamic self-adapting speech analysis techniques for man-machine SET method and system |
| CN107273861A (en) * | 2017-06-20 | 2017-10-20 | 广东小天才科技有限公司 | A kind of subjective question marking methods of marking, device and terminal device |
-
2017
- 2017-11-07 CN CN201711081247.0A patent/CN107705231B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942994A (en) * | 2014-04-22 | 2014-07-23 | 济南大学 | Computer assessment method for subjective questions |
| CN106940788A (en) * | 2017-03-07 | 2017-07-11 | 百度在线网络技术（北京）有限公司 | Intelligent scoring method and device, computer equipment and computer-readable medium |
| CN107240394A (en) * | 2017-06-14 | 2017-10-10 | 北京策腾教育科技有限公司 | A kind of dynamic self-adapting speech analysis techniques for man-machine SET method and system |
| CN107273861A (en) * | 2017-06-20 | 2017-10-20 | 广东小天才科技有限公司 | A kind of subjective question marking methods of marking, device and terminal device |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于短文本相似度计算的主观题自动阅卷方法;张均胜 等;《图书情报工作》;20141031;第31-37页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107705231A (en) | 2018-02-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107705231B (en) | Computer-aided paper marking method and device and computer-readable storage medium | |
| CN106570109B (en) | Method for automatically generating question bank knowledge points through text analysis | |
| Alderman et al. | Item performance across native language groups on the Test of English as a Foreign Language | |
| T. Proyer et al. | Gender differences in vocational interests and their stability across different assessment methods | |
| Tack et al. | Human and automated CEFR-based grading of short answers | |
| Fidalgo et al. | Utility of the Mantel-Haenszel procedure for detecting differential item functioning in small samples | |
| Różycka‐Tran et al. | Measurement invariance of the Belief in a Zero‐Sum Game scale across 36 countries | |
| Finch et al. | Comparison of NOHARM and DETECT in item cluster recovery: Counting dimensions and allocating items | |
| CN107845047B (en) | Dynamic scoring system, method and computer readable storage medium | |
| Mead | Assessment of fit of data to the Rasch model through analysis of residuals. | |
| Visser et al. | Construct equivalence of the OPQ32n for Black and White people in South Africa | |
| Hussain et al. | Assessment of bangla descriptive answer script digitally | |
| CN110096708B (en) | Calibration set determining method and device | |
| Chicca | Analyzing use of the graduate record examinations® general test in doctoral nursing education: Start, stop, continue, or modify? | |
| Krylovas et al. | Mathematical modelling of forecasting the results of knowledge testing | |
| Whitman et al. | Computer aversion and computer-use ethics in US and Asian cultures | |
| Khodabakhshzadehs et al. | The Relationships among Attitudes towards Cheating, Academic Self-Confidence, and General Language Ability among Iranian EFL Learners. | |
| KR20170004330A (en) | Applicant-customized evaluation and analysis system by grouping the test applicants and the method thereof | |
| Harder et al. | A pragmatic measure of immigrant integration | |
| Mazi | Examination of the Teachers Multicultural Perceptions: Sample of Hatay Province. | |
| JP2005331650A (en) | Learning system, information processor, information processing method and program | |
| Sezgin | Factors affecting mathematics literacy of students based on PISA 2012: A cross-cultural examination | |
| Ali et al. | The role of English language proficiency as determinant of PGDT candidate academic performance across selected public universities of Ethiopia | |
| Ryan | Issues, strategies, and procedures for applying standards when multiple measures are employed | |
| Bramley | Subject difficulty-the analogy with question difficulty |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |