CN109697274B - Examination paper judging method and examination paper judging system - Google Patents
Examination paper judging method and examination paper judging system Download PDFInfo
- Publication number
- CN109697274B CN109697274B CN201710981922.9A CN201710981922A CN109697274B CN 109697274 B CN109697274 B CN 109697274B CN 201710981922 A CN201710981922 A CN 201710981922A CN 109697274 B CN109697274 B CN 109697274B
- Authority
- CN
- China
- Prior art keywords
- word
- scoring
- cloud
- phrase
- answers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000014509 gene expression Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 11
- 238000012360 testing method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 238000012552 review Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000036651 mood Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013077 scoring method Methods 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/103—Formatting, i.e. changing of presentation of documents
- G06F40/106—Display of layout of documents; Previewing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/237—Lexical tools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
Abstract
The invention provides an appraising method and an appraising system, wherein in the appraising method, an electronic answer is compared with a standard answer prestored in a cloud scoring word bank, and if a certain word or phrase is prestored in the cloud scoring word bank, the word or phrase is highlighted. The examination paper system is provided with an examination paper cloud end and a plurality of examination paper terminals, the examination paper cloud end comprises a cloud end scoring word bank, examination paper readers use the examination paper terminals to conduct examination paper marking on subjective questions, and the examination paper terminals execute the examination paper method. The scoring keywords can be found in the electronic answers by the reader, and the appraising efficiency is greatly improved. If some words are correct and are not recorded in the standard answers or scores are difficult to give by taking the standard answers as the basis, the invention allows a reader to maintain the standard answers, and the reader can easily find that the word is not highlighted on the screen, and then the word can be selected and added into the cloud scoring word bank for other readers to refer to, so that the standard answers can be continuously updated.
Description
Technical Field
The invention relates to the technical field of multimedia teaching, in particular to a paper judging method and a paper judging system for multimedia teaching based on an internet teaching platform.
Background
In a training, examination, education system, etc., in order to assess the learning condition of a student, the student is usually examined and scored according to the examination result to know the mastering degree of the student on the learning knowledge. In general, in order to effectively evaluate a student, the test questions of a test examination may be classified into subjective questions and objective questions. The objective questions are mostly selected in a mode of selecting the questions, and the objective questions are conveniently scored by a computer because the answers of the objective questions are fixed, so that an old manual examination paper marking mode is avoided, examination paper marking time can be shortened, the labor cost of examination paper marking is saved, and examination paper marking efficiency is improved. For the subjective questions, the students usually use a discussion mode to solve the questions in the examination questions, and in the solution process of the subjective questions, the students usually need to exert themselves to solve the questions in the examination questions according to their own thinking mode. Therefore, the answer to the subjective question is only a reference, and is not an absolute standard, and therefore, in the prior art, the subjective question is generally manually checked. The manual marking has many defects and shortcomings, and mainly has two points, namely, the efficiency of the manual marking is low, and the marking standard of the manual marking is difficult to unify, because the number of people who mark the examination papers is large, and the examinees express a variety of questions in the answering process, and the answers and the appraising standard of the subjective question are relatively single, therefore, the appraising scale of the marking people of the subjective question is difficult to unify, and the free judgment right is too large.
As computer applications have developed, manual scoring also uses more computer technology. One is that the subjective questions are directly scanned and then recorded into the system in the form of pictures, and the examiner watches the pictures of the subjective questions in front of the computer, which is not different from the direct examination paper reading and is only convenient for the recording and statistics of the examination paper; another is that more technicians wish to use computer scoring instead of manual scoring.
An intelligent scoring method and apparatus, a computer device, and a computer readable medium are proposed in the prior art. Wherein the method comprises: obtaining user answers corresponding to the examination questions; generating a user word segmentation library corresponding to the user answer according to the user answer; and scoring the user answers according to the user word segmentation library, a standard word segmentation library corresponding to the pre-generated standard answers and the weight of each standard word segmentation in the standard word segmentation library. The scheme can intelligently grade the user answers of the examination questions with the types of the subjective questions, and can overcome the defects that the grading time is long and the labor cost is high due to the adoption of a manual paper marking mode in the prior art, so that the grading time of the examination questions can be effectively shortened, the labor cost can be effectively saved, and the grading efficiency of the user answers of the subjective questions with the types of the user answers can be greatly improved.
The prior art also provides a system and a method for automatically scoring the subjective questions based on the semantic similarity interval, wherein the method for automatically scoring the subjective questions based on the semantic similarity interval comprises the following steps: initializing a score of a topic to be scored, namely, a total; defining a block length L; dividing the reference answer into a plurality of character blocks with the length not greater than L to form a character block set R of the reference answer; dividing the answer to be evaluated into a plurality of character blocks with the length not more than L to form a character block set T of the answer to be evaluated; comparing the set R, T, and calculating the similarity SRT of the semantics of the two sets; mapping the SRT to a similarity interval, recording the score as Sfinal, and finishing the score; the automatic subjective question scoring system based on the semantic similarity interval comprises: the system comprises an answering terminal, a subjective question marking module and an examination score generating module. According to the scheme, the automatic marking and grading links of the subjective questions can realize grade division of the answers of the subjective questions according to the semantic similarity threshold, and the final grade is given through the value constraint and the grading of the similarity of each grade.
The two examination paper marking modes basically do not need people to participate, but the examination paper marking mode or the examination paper marking method is only suitable for common tests and exercises and cannot be suitable for large-scale important examinations. In addition, when the standard answer generated in advance is set, the set scale is difficult to grasp, and the standard answer cannot be adjusted in the later examination.
In addition, the prior art considers that for C program design courses set by almost all colleges and universities, only selection questions and judgment questions cannot be considered, and the program compiled by students cannot be directly judged to be zero score because the result is not completely consistent with the reference answer. In view of the fact that there may be multiple algorithms or writing modes for realizing the same problem by using programs, it is very difficult to completely realize automatic scoring by using a computer, so that the combination of automatic scoring and manual scoring is required, and the scoring efficiency is improved while the reasonability is considered. Therefore, when the subjective question is judged, if the input answer is the same as the reference answer, the score is returned; if the input answer is different from the reference answer, testing the input answer, wherein the test is that the reading program questions can be automatically reviewed after setting scores according to the answer types submitted by examinees; automatically evaluating questions with possibly non-unique answers such as program filling, error correction and the like in modes of sentence comparison, code substitution into program operation and the like, and automatically finishing the evaluation of other same answers by only once evaluating one answer; for programming questions, different scores are given according to whether the operation result is consistent with the expected answer or not (automatic review), whether the operation can be compiled or not (manual intervention review), and source code logic analysis (manual intervention review) for different inputs. The prior art introduces intelligent evaluation on subjective questions to a certain extent, but the prior art can only be used for subjective question judgment in the field of computers, subjective questions in other fields cannot be verified in a substitution calculation mode, and in addition, the content of a rating database maintained by a terminal user is not available.
In the prior art, an appraising method for subjective questions is also provided, which is used for carrying out image mode identification on images of non-drawing question type areas, namely images of the subjective question areas, extracting answer keywords, matching the keywords with keyword data, and carrying out non-drawing question type preliminary appraising according to matching results; teachers of different subjects access the corresponding folders of the database according to the course codes to review, and perform non-drawing type question supplementing and scoring; and combining the non-drawing question type primary scores and the non-drawing question type supplementary scores to obtain non-drawing question type scores. This prior art provides a final combination of machine scoring and manual scoring, where manual supplemental scoring is more objective than scoring of subjective questions. This document also does not disclose the content of the end-user maintained scoring database. When the teacher of each class is supplemented with time, the method is not different from the existing subjective question scoring mode, and only simple combination of machine scoring and manual scoring is adopted, and no reference is provided among scoring teachers.
There is also a prior art which uses an annotation method comprising: scanning a paper file to obtain a first image; acquiring text contents of a plurality of questions and a plurality of second images of a plurality of answers according to the first image; acquiring a plurality of search terms according to the plurality of second images; retrieving the annotation database according to the plurality of search terms; newly adding annotation content aiming at the searched search words, and storing the newly added annotation content into an annotation database; modifying the annotation content aiming at the searched search terms, and storing the modified annotation content into an annotation database; aiming at all the search terms, establishing an annotation list, wherein the annotation list comprises the search terms and corresponding annotation contents; and generating text content comprising a plurality of titles and corresponding annotation content according to the annotation list, and automatically sending the E-mail. According to the method, the annotation database is established by recording and summarizing the annotation information, and the annotation is retrieved through the search words, so that the annotation use efficiency is improved. The method effectively improves the working efficiency of teachers. But it is also a simple combination of machine scoring and manual scoring, and there is no reference between the scoring teachers. In particular, it is clearly understood in the art that the text of the annotation is not limited by space in the annotation process. The advantages are that: the annotation content text content has great flexibility. Aiming at the same problem, different teachers can independently give out own annotation content. The teacher can show the teaching work characteristics through the reading and the amending of homework, and transfer the knowledge and the experience of the teacher on the teaching work to students. This clearly gives the inspiration for independently made annotations between teachers, and personalized annotations are considered to be beneficial for teaching. This is equivalent to negating the necessity for reconciliation of the approval criteria, which would make it difficult for one skilled in the art to derive an on-line batch roll hint.
How to use a computer to efficiently finish manual marking and ensure marking quality in a standard and consistent manner is a technical problem which is expected to be solved in the field for a long time, and the invention is provided in view of the above.
Disclosure of Invention
The invention aims to provide a paper judging method and a paper judging system to solve the problems.
In order to realize the purpose, the invention adopts the following technical scheme:
a method of appraising a paper, comprising the steps of:
s1: starting;
s2: obtaining answers of the subjective questions and obtaining electronic answers;
s3: calling cloud scoring word bank information;
s4: highlighting the words or phrases appearing in the cloud scoring word stock in the electronic answers;
s5: judging whether to take words or phrases from the current electronic answer and add the words or phrases into a cloud scoring word stock, if so, executing S6; if not, go to S7;
s6: acquiring the selected word or phrase, adding the selected word or phrase into a cloud scoring word bank, and returning to S3;
s7: obtaining a score input by a reader;
s8: and (6) ending.
Preferably, S4 is specifically:
s401: highlighting words or phrases appearing in the cloud scoring word stock in the electronic answers according to different categories;
s402: judging whether the highlighted word or phrase is selected; if so, executing S403; if not, go to S5;
s403: and calling the related information of the word or the phrase from the cloud scoring word bank and displaying the related information on a screen.
Preferably, in step S6, the word or phrase in the answer is selected by double-clicking or more times on the position of a word or phrase on the display screen of the scoring terminal, automatically generating a front selector and a rear selector at the position, selecting a character or a character between the front and rear selectors, and selecting the specific content of the word or phrase by adjusting the positions of the front and rear selectors, and popping up a menu in a predetermined area of the screen or an area close to the selectors, where the menu at least includes command options uploaded to the cloud.
Preferably, if a touch-screen operation is used, a predefined gesture or a separate option is used to trigger the generation of the front and back selectors.
The application also provides a paper judging method, which comprises the following steps:
s1': starting;
s2': obtaining answers of the subjective questions and obtaining electronic answers;
s3': selecting a part of character areas in the electronic answers, and calling cloud scoring word bank information;
s4': highlighting the words or phrases appearing in the cloud scoring word stock in the electronic answers, taking the highlighted words or phrases in the partial character areas as entries, and displaying the related information of the entries;
s5': judging whether the entry is added into a cloud scoring word bank or not, and if so, executing S6'; if not, go to S7';
s6': acquiring the selected entries, adding the selected entries into a cloud scoring thesaurus, and returning to S3';
s7': obtaining a score input by a reader;
s8': and (6) ending.
Preferably, step S4' is specifically:
s4': highlighting words or phrases appearing in the cloud scoring word stock in the electronic answers according to different categories; taking a plurality of highlighted words or phrases in a designated answer character area as entries;
preferably, in step S6', the word or phrase in the answer is selected by double-clicking or more times on the position of a word or phrase on the display screen of the scoring terminal, and automatically generating a front selector and a rear selector at the position, where the character or character between the front and rear selectors is to be selected, and the selection of the specific content of the word or phrase may be determined by adjusting the positions of the front and rear selectors, and popping up a menu in a predetermined area of the screen or an area close to the selectors, where the menu at least includes command options uploaded to the cloud.
Preferably, in S4 ', the scoring terminal limits the number of keywords and the number of categories of the keywords contained in the entry, and when the number of categories or the number of occurrences of the keywords in the text region drawn by the reader is too large or exceeds a predetermined value, a part of the text region in S3' is considered as an invalid text region.
Preferably, in step S4', if the number and category of keywords included in the selected answer text region are not changed, but only the total number of words in the answer text region is changed, the appraisal terminal considers that the entry is not changed.
This application still provides an appraisal system, and it has an appraisal book high in the clouds and connects a plurality of appraisal book terminals in the high in the clouds, and the appraisal book high in the clouds includes high in the clouds word stock of grading, and the person of going up the paper utilizes appraisal book terminal to go up the examination of subjective question, its characterized in that: the appraising terminal uses the appraising method.
Has the advantages that:
1. the paper judging method is an paper judging method between manual paper marking and complete computer paper judging in the prior art, paper judging is more objective due to participation of paper marking persons, in the paper judging method, a computer is used for assisting in paper judging, specifically, electronic answers obtained by a paper judging terminal are compared with paper judging standards prestored in a cloud scoring word bank, and if a certain word or phrase is prestored in the cloud scoring word bank, the word or phrase is highlighted. The reader can easily find the scoring keyword in the electronic answer, and the work efficiency is greatly improved.
2. Keyword highlighting also has an important technical effect that it is basically correct if some words are encountered, but not recorded in the standard answer, or it is difficult to give a score using the standard answer as a basis. The invention provides a technical concept for a reader to maintain standard answers, the reader can easily find that the word is not highlighted on a screen, and the word can be selected and added into a cloud scoring word bank, so that the content of the standard answers is more comprehensive, and the technical scheme of the application is clearly distinguished from the prior art.
3. In the process of judging the subjective question, not only a single word or phrase is often required to be evaluated, but also the validity and correctness of the answer are considered by taking a plurality of keywords as a whole. Thus, the introduction of the entry solves this problem. The use of terms greatly improves the consistency of the scoring criteria. Firstly, the reader firstly defines an effective text area, namely, a plurality of keywords of different categories are concentrated to appear in the effective text area, so that the condition that the examinee wants to disperse listed keywords to expect scores is avoided; secondly, the disputed and problematic answers can be proposed in time and analyzed and solved quickly, and the disputed and problematic answers can not be predicted at the beginning of setting up the answers.
4. The invention further provides that the scoring terminal limits the number and types of the keywords contained in the vocabulary entry, when the character area drawn by the reader is too large, or the keywords are too many, the character area is considered to be an invalid character area, because if the character area is too large or the keywords are too many, the character area cannot be distinguished from the behaviors of the keywords listed by the examinee, and for a conventional expression form, a sentence with an independent meaning function cannot contain too many keywords. Therefore, the appraising terminal further judges whether the number of words contained in the selected answer character area is greater than a preset first threshold value or not, and whether the number of contained keywords is greater than a preset second threshold value or not, and if not, the creation or reading of the vocabulary entry can be carried out. When the reader moves the cursor, if the number and category of the keywords contained in the answer text area are not changed, only the total number of words in the answer text area is changed, and at this time, the appraising terminal considers that the entry is not changed. Therefore, the total amount of the entries can be greatly reduced, and the situation that the entries are excessively created and the judgment of a reader is influenced is prevented. When the cursor is moved by the reader, the number or the type of the keywords changes, and the entries recognized by the appraising terminal also change, so that the user can quickly see the explanations of different entries when moving the cursor, quickly know the scoring standards of different entries, and substantially quickly know the scoring standards after different keyword combinations.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the contents of the embodiments of the present invention and the drawings without creative efforts.
FIG. 1 is a flow chart of a first method of appraising of the invention;
FIG. 2 is a flow chart of a method for highlighting an electronic answer in accordance with the present invention; and
FIG. 3 is a flow chart of a second method of appraising of the invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the embodiments described herein are merely illustrative and are not intended to limit the present invention. Various changes and modifications may be effected by one of ordinary skill in the pertinent art without departing from the spirit of the present invention, which is within the scope of the appended claims and the appended claims.
According to the embodiment of the invention, the examination paper system is provided with an examination paper cloud and a plurality of terminals connected to the cloud, and the examiner can use the terminals to perform examination paper marking on the subjective questions.
Example 1:
as shown in fig. 1, a method of appraising a paper includes the steps of:
s1: starting;
s2: and acquiring answers of the subjective questions and obtaining electronic answers. When the user answers of a certain examination question are scored, the user answers corresponding to the examination question are obtained. For example, when the user answer to a question is in electronic form, the user answer to the question is read directly. When the user answer of the question is in a paper form, the characters in the user answer on the paper form can be identified by adopting a character identification technology to obtain the user answer in an electronic form. The method of obtaining the answer to the subjective question is not limited to the above two methods, and only the electronic answer needs to be obtained.
S3: and calling cloud scoring word bank information. The cloud scoring word bank is arranged at the cloud of the appraising system, the scoring standard of the subjective questions is preset in the cloud scoring word bank, the scoring standard of the subjective questions is the same as that of traditional manual scoring, the scoring standard of the subjective questions is mainly used for judging whether key words or phrases appear in answers or not, namely commonly called score points, and scores are given according to the covering condition of the score points by the subjective questions and other consideration factors. Keywords of answers of the subjective questions are also preset in the cloud scoring word bank, and together with the keywords, possible related expressions such as explanations of the keywords and synonyms of the keywords are included.
S4: highlighting the words or phrases in the electronic answers, wherein the words or phrases appear in the cloud scoring word stock. And comparing the obtained electronic answers with the score standards prestored in the cloud score word bank by the appraising terminal, and if a certain word or phrase belongs to the keywords or the related expressions prestored in the cloud score word bank, highlighting the word or phrase. The reader can find the scoring keywords in the electronic answers very easily, and the work efficiency is greatly improved.
S5: judging whether related words or phrases need to be acquired from the current electronic answers and added into the cloud scoring word bank, if so, executing S6; if not, S7 is executed. In the process of marking subjective questions, it is often encountered that some answers are also basically correct, but are not recorded in standard answers, or it is difficult to give scores using the standard answers as a basis. Generally, the reader directly gives a score according to own experience, so that the score has strong subjective judgment, and other readers may also suffer from the same problems, and the scale of everyone is difficult to unify. If the reader finds that a certain word can be used as a scoring item but is not recorded in the cloud scoring word bank in the electronic answer displayed in the step S4, the most intuitive expression is that the word is not highlighted on a screen, and the word or the word group can be selected and added into the cloud scoring word bank, so that the content of the standard answer is more comprehensive.
S6: and acquiring the selected word or phrase, adding the word or phrase into the cloud scoring word bank, and returning to the step S3. After a word or a phrase is added to the cloud-side scoring word bank, the terminal can compare the electronic answer with the cloud-side scoring word bank again, and the word can be highlighted after comparison. Importantly, if the word appears in other electronic answers, the word is highlighted on other terminals.
S7: and acquiring the score input by the reader. And the reader gives a final score according to the comprehensive presentation degree of the keywords appearing in the electronic answer and other scoring factors and inputs the final score into the terminal.
S8: and (6) ending.
Preferably, step S4 is specifically:
s401: highlighting words or phrases appearing in the cloud scoring word stock in the electronic answers according to different categories; this will be more beneficial for the reader to intuitively understand the scoring situation in the electronic answer. Such as distinguishing different categories of keywords by different colors.
S402: judging whether the highlighted word or phrase is selected; if so, executing S403; if not, go to S5;
s403: the relevant information of the word or phrase is called from the cloud-end scoring word stock and displayed on the screen, and then S5 is executed.
Steps S402 and S403 are mainly for facilitating the examiner to better understand the information related to the keyword or the related scoring criteria. In S402. In step S6 or S402, the answer may be selected in various ways, for example, a position where a word or a phrase is located is clicked on the display screen of the scoring terminal in two or more times, a front selector and a rear selector are automatically generated at the position, a character or a word between the front and rear selectors is selected, the selection of the specific content of the word or the phrase may be determined by adjusting the positions of the front and rear selectors, and a menu pops up in a predetermined area of the screen or an area close to the selectors, where the menu includes at least command options uploaded to the cloud. After the menu item is triggered, at least explanation content or labels are added to the words or the word groups, and the explanation content or the labels are used for explaining why the words are added to the cloud scoring word bank. The method of selecting a word or a phrase may be simply to stop the cursor at the position of the word or phrase on the screen.
If a touch screen type operation is used, a predefined gesture or a separate option may be used to trigger the generation of the front and back selectors. For example, clicking a certain area of the screen multiple times, pressing the screen for a long time, and these operation modes are conventional means in the art and are not listed.
Example 2:
in this example, the difference from example 1 is that:
a method of appraising a paper, comprising the steps of:
s1': starting;
s2': obtaining answers of the subjective questions and obtaining electronic answers;
s3': selecting a part of character areas in the electronic answers, and calling cloud scoring word bank information;
s4': highlighting the words or phrases appearing in the cloud scoring word stock in the electronic answers, taking the highlighted words or phrases in the partial character areas as entries, and displaying the related information of the entries;
s5': judging whether the entry is added into a cloud scoring word bank or not, and if so, executing S6'; if not, go to S7';
s6': acquiring the selected entries, adding the selected entries into a cloud scoring thesaurus, and returning to S3';
s7': obtaining a score input by a reader;
s8': and (6) ending.
The term concept is proposed here, and in the process of judging the subjective question, not only a single word or phrase is often required to be evaluated, but also the validity and correctness of the answer are considered by taking a plurality of keywords as a whole. Often multiple keywords are correct or related, but the final answer is inaccurate or irrelevant. It is often the case that the answerer presents answers at a space beyond the necessary limit in the hope that the reader will be able to find a point in the answers and give a higher score. On the one hand, it is inappropriate to give the reader this work of extracting the answer, and on the other hand, there may be many keywords scattered in the answer, which have no combined specific meaning, simply because the space is too long to be included in the answer. The concept of vocabulary entry is also because the standard answers cannot exhaust all the scoring criteria for different kinds of keywords combined together, some combinations are correct and listed in the standard answers, some are incorrect, but some are not accurate enough, and the scores should be discounted based on the standard answers. Thus, the presence of the entry solves this problem.
In step S4', the examination paper terminal first detects the answer character area selected by the reader, and determines how many different keywords appear in the area according to the highlighting condition of the keywords. The reader can then divide the selected region according to the actual condition of the answer, such as a complete expression paragraph, for example, a response to a specific question in the question, and these answer regions in a relatively complete expression form will be subjectively divided by the reader. Keyword information for the answer text region is then extracted, including, for example, a first keyword and a second keyword, which are identified as a first entry. Information corresponding to the entry is called from the cloud-based scoring thesaurus and displayed on the screen, and then S5' is performed. That is, if the first term is included in the cloud-based scoring thesaurus, then a specific evaluation about the first term is displayed. For example, the scorer will give a specific evaluation to this specific keyword combination, including but not limited to a series of information about the correctness, accuracy, relevance, suggested score, etc. of the entry, which forms a mapping relationship with the first entry.
In the conventional paper-marking mode, it is generally considered that the score is given only if the examinee writes the keyword in the answer, but as described in the foregoing analysis, the fact that the keyword has no logical listing in the answer does not mean that the score can be obtained, and the keyword has a logical arrangement to represent the correctness of the answer. Compared with the prior art, the use of the entries greatly improves the consistency of the scoring standards. Firstly, the reader firstly defines an effective text area, namely, a plurality of keywords of different categories are concentrated to appear in the effective text area, so that the condition that the examinee wants to disperse listed keywords to expect scores is avoided; secondly, the disputed and problematic answers can be proposed in time and analyzed and solved quickly, and the disputed and problematic answers can not be predicted at the beginning of setting up the answers.
However, if the cloud-based scoring thesaurus does not contain the first entry, the relevant information is not displayed on the screen. At this time, it is necessary to perform S5 ', S5': and judging whether to add the current entry into the cloud scoring word bank. At this time, the first entry is already indexed, but if the first entry is not sent to the cloud by the reader, the first entry is not recorded by the cloud scoring thesaurus, and the reader determines whether the first entry should be added into the standard answer or not according to the subjective judgment of the reader, and whether the first entry should be the standard answer universally suitable for the reading.
Preferably, the scoring terminal limits the number and types of keywords contained in the entry, and when the text area drawn by the reader is too large, or the keywords appear too much, the text area is considered to be an invalid text area, because if the text area is too large or the keywords are contained too much, the text area cannot be distinguished from the behaviors of the keywords listed by the examinee, and for a conventional expression form, a sentence with an independent meaning function does not contain too many keywords. Therefore, the appraising terminal further judges whether the number of words contained in the selected answer character area is greater than a preset first threshold value or not, and whether the number of contained keywords is greater than a preset second threshold value or not, and if not, the creation or reading of the vocabulary entry can be carried out.
In the process of moving the cursor by the reader, if the number and the category of the keywords contained in the answer text area are not changed, only the total number of the words in the answer text area is changed, and at this time, the appraising terminal considers that the entry is not changed. Therefore, the total amount of the entries can be greatly reduced, and the situation that the entries are excessively created and the judgment of a reader is influenced is prevented. In the cursor moving process of the reader, the number or the type of the keywords changes, and the vocabulary entry identified by the appraising terminal also changes, so that the user can quickly see the explanation of different vocabulary entries when moving the cursor, quickly know the scoring standards of different vocabulary entries, and actually quickly know the scoring standards of different keyword combinations.
In a preferred embodiment, the words or phrases or terms uploaded to the cloud-based scoring thesaurus by the reader are not recorded in the cloud-based scoring thesaurus until the advanced user confirms the words or phrases or terms.
The following provides a better understanding of the present invention by way of an example.
Example title: why does the little fram feel mr. korea "never so high"?
Standard answers: the little framber feels his love and miss his sorrow and distressed mood from mr. korea. Mr. Korea's love spirit makes the little Francisco feel that he is not only a French teacher, but also an love shiver, so he feels his image high.
Grading standard: the examinee can get full scores by only answering the love of mr korea to his country, the sorrow to lose his country and the little franchise to revery mr korea.
Based on the scoring standard, firstly presetting standard answers in a cloud scoring word bank: first-class keywords: making love; the second type of keywords: sadness and pain; keywords of the third type: the country and the country; fourth category keyword: tall, great, worship, admire; a fifth type of keyword: a little frauds; a sixth type of keyword: and (4) Korea Meyer.
And displaying the electronic answers of the examinees at the appraising terminal.
Answer one: the little francs experiences that mr. korea loses his sorrow and a difficult mood in our country. The little fraudster feels he very tall.
The first answer is greatly different from the standard answer, but the score points capture well, and a plurality of score point keywords are written. First, on the screen of the appraisal terminal, the words "home", "sad", "little boy", "hangeul" and "big high" will be highlighted, and the examiner can quickly find that the examinee does not write the score point of "love" of the first category keyword into the answer, and thus the score will be reduced as appropriate. Meanwhile, the reader can express the sadness and pain meanings of the second type of keywords when finding the word 'difficult to pass', and at the moment, the reader selects the word by a cursor and sends the word to a cloud scoring word bank. At this time, the second category of keywords includes sadness, suffering, and difficulty. When the second reader encounters the answer one again, the word "hard" will be highlighted on his terminal screen. Therefore, the answer content of the cloud scoring word bank is enriched, and the scoring standard is convenient to unify.
Answer two: the little fraudster feels that he is very sade from the tragic white complexion of mr. korea. This reflects the love of the little fraudster in our country, who thinks the image of mr. koraii is high.
In this answer, although most of the keywords appear, the information of "love of cerealose to the home country" cannot be directly obtained from the material given by the title, and the subject of love to the home country should be korean. It is not objective to use only the keyword for judgment, and the judgment of the entry is used in this case.
Firstly, only when a plurality of keywords simultaneously appear in a selected area of the answer, the keywords are used as an entry, the appraising terminal uses three keywords of 'Xiaoverlang', 'Guo nationality' and 'love' in the selected area as an entry, if the entry is recorded in the cloud scoring word bank at the same time, the appraising terminal can display auxiliary information of the entry, in the second answer, because the entry matched with the 'Xiaoverlang', 'Guo nationality' and 'love' is an incorrect answer and is not recorded in the cloud scoring word bank, the answer can not display any information related to the entry at the appraising terminal. Thus, the use of the entries facilitates objective judgment of the test paper. However, three keywords of "hangeul", "national country" and "love" are correct answers as a term, if the term is in the region of the characters selected by the reader, the appraisal terminal will call the related auxiliary information of the term from the cloud scoring thesaurus, and if the term is not created, the reader can actively create the term and send the term to the cloud scoring thesaurus, so as to add scoring suggestions or other scoring standards for the term.
When the answer two is scored, in the process that the cursor is moved by the reader, if the number and the category of the three keys of 'Xiaovershi', 'Guokou' and 'love' contained in the answer character area are not changed, and only the total number of words in the selected character area is changed, the scoring terminal considers that the entry is not changed.
For example, "this represents the love of the young francs to the home country, and he considers the shape of mr. koraii" and "this represents the love of the young francs to the home country, and he considers mr. koraii" and the keywords included in the two word-taking modes are not changed, so the entries of the answer character region are not changed, which is a significant improvement to the appraisal system. The judgment of the entries is designed to be irrelevant to the total amount of the characters in the selected area, so that the total amount of the entries can be greatly reduced, and the situation that the entries are excessively created and the judgment of a reader is influenced on the contrary is prevented. As described above, the scoring terminal should limit the number and types of keywords contained in the vocabulary entry, and when the keyword area extracted by the reader appears too much or the keyword types are too many, it is considered as an invalid character area, because if the character area contains too many keywords, it cannot be distinguished from the behavior of the examinee list keywords, for example, if the reader selects the four keywords "xiaofuranshi", "hangeul", "home", "love", in the answer two, such vocabulary entry would be unfavorable for scoring if it is created, because the probability of the combination of the four words is too many, and a large number of correct and wrong combinations are contained.
For the analysis of the cases, the examination paper judging method can firstly improve the working efficiency of the examiner through highlight display of the keywords, further, relevant information of corresponding terms can be provided according to the conditions of the terms formed by the keywords, and the examiner can modify and enrich the cloud data in the examination paper judging process no matter the keywords or the terms are used, so that the whole scoring process is more and more uniform and more scientific.
The hardware implementing the present invention is presented below as an alternative embodiment, as an appraisal terminal, which may be a computer device, such as in the form of a general purpose computing device. Components of a computer device may include, but are not limited to, one or more processors, a system memory, and a bus connecting various system components (including the system memory and the processors).
The above described buses are generally indicative in the art of one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. These architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
The computer device may also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), with one or more devices that enable a user to interact with the computer device, and/or with any devices (e.g., network card, modem, etc.) that enable the computer device to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface. Also, the computer device may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through a network adapter. The network adapter communicates with other modules of the computer device over the bus.
The computer device typically includes a variety of computer system readable media. Such media may be any available media that is accessible by a computing device and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) and/or cache memory. The computer device may further include other removable/non-removable, volatile/nonvolatile computer system storage media. The system memory may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the present invention described above for the paper. A program/utility that also has a set (at least one) of program modules may be stored, for example, in system memory, such program modules including an operating system, one or more application programs, other program modules, and program data, including an implementation of a network environment in each or some combination of these examples.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
For the appraising cloud, the main function is the data storage function, and many people in the prior art can use the appraising cloud, and the description is omitted here.
The foregoing describes preferred embodiments of the present invention, and is intended to provide a clear and concise description of the spirit and scope of the invention, and not to limit the same, but to include all changes, substitutions and alterations herein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A method of appraising a paper, comprising the steps of:
s1: starting;
s2: obtaining answers of the subjective questions and obtaining electronic answers;
s3: calling cloud scoring word bank information; the cloud scoring word bank is arranged at the cloud of the appraising system, and scoring standards of subjective questions are preset in the cloud scoring word bank;
s4: highlighting the words or phrases appearing in the cloud scoring word stock in the electronic answers; comparing the obtained electronic answers with scoring standards prestored in a cloud scoring word bank by the appraising terminal, and if a certain word or phrase belongs to keywords prestored in the cloud scoring word bank or related expressions thereof, highlighting the word or phrase;
s5: judging whether to take words or phrases from the current electronic answer and add the words or phrases into a cloud scoring word stock, if so, executing S6; if not, go to S7;
s6: acquiring the selected word or phrase, adding the selected word or phrase into a cloud scoring word bank, and returning to S3;
s7: obtaining a score input by a reader;
s8: finishing;
wherein, the step S4 further includes:
s401: highlighting words or phrases appearing in the cloud scoring word stock in the electronic answers according to different categories;
s402: judging whether the highlighted word or phrase is selected; if so, executing S403; if not, go to S5;
s403: calling the related information of the word or the phrase from the cloud scoring word bank and displaying the related information on a screen;
in step S6, the word or phrase in the answer is selected by double-clicking or more times on the position of a word or phrase on the display screen of the scoring terminal, automatically generating a front selector symbol and a rear selector symbol at the position, selecting a character or a word between the front and rear selector symbols, selecting the specific content of the word or phrase may be determined by adjusting the positions of the front and rear selector symbols, and popping up a menu in a predetermined area of the screen or an area close to the selector symbols, where the menu at least includes command options uploaded to the cloud.
2. The method of claim 1,
if a touch screen type operation is used, a predefined gesture or a separate option is used to trigger the generation of the front and back selectors.
3. A method of appraising a paper, comprising the steps of:
s1': starting;
s2': obtaining answers of the subjective questions and obtaining electronic answers;
s3': selecting a part of character areas in the electronic answers, and calling cloud scoring word bank information; the cloud scoring word bank is arranged at the cloud of the appraising system, and scoring standards of subjective questions are preset in the cloud scoring word bank;
s4': highlighting the words or phrases appearing in the cloud scoring word stock in the electronic answers, taking the highlighted words or phrases in the partial character areas as entries, and displaying the related information of the entries; comparing the obtained electronic answers with scoring standards prestored in a cloud scoring word bank by the appraising terminal, and if a certain word or phrase belongs to keywords prestored in the cloud scoring word bank or related expressions thereof, highlighting the word or phrase;
s5': judging whether the entry is added into a cloud scoring word bank or not, and if so, executing S6'; if not, go to S7';
s6': acquiring the selected entries, adding the selected entries into a cloud scoring thesaurus, and returning to S3';
s7': obtaining a score input by a reader;
s8': finishing;
wherein, step S4' specifically includes:
highlighting words or phrases appearing in the cloud scoring word stock in the electronic answers according to different categories;
taking a plurality of highlighted words or phrases existing in the defined answer character area as entries;
in step S6', the word or phrase in the answer is selected by double-clicking or more times on the position of a word or phrase on the display screen of the scoring terminal, automatically generating a front selector symbol and a rear selector symbol at the position, selecting a character or a character between the front and rear selector symbols, and selecting the specific content of the word or phrase, which can be determined by adjusting the positions of the front and rear selector symbols, and popping up a menu in a predetermined area of the screen or an area close to the selector symbols, where the menu at least includes command options uploaded to the cloud.
4. The method of claim 3,
in step S4', the scoring terminal limits the number of keywords and the number of categories of keywords contained in the entry,
when the number of keywords in the text area drawn by the reader exceeds a predetermined value or the number of categories exceeds a predetermined value, a part of the text area in step S3' is considered as an invalid text area.
5. The method of claim 3,
in step S4', if the number and category of the keywords included in the selected answer text region are not changed, but only the total number of words in the answer text region is changed, the appraisal terminal determines that the entry is not changed.
6. The utility model provides an appraisal paper system, its have an appraisal paper high in the clouds and connect a plurality of appraisal paper terminals in the high in the clouds, and the appraisal paper high in the clouds includes high in the clouds word stock of grading, and the person of marking utilizes appraisal paper terminal to carry out the marking to subjective question, its characterized in that:
the rating terminal performs the rating method according to any one of claims 1-5.
7. Computer arrangement comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the program, is adapted to carry out the method steps of any of claims 1-5.
8. A computer storage medium storing a program executable by a computer, the program when executed implementing the method steps of any of claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710981922.9A CN109697274B (en) | 2017-10-20 | 2017-10-20 | Examination paper judging method and examination paper judging system |
| PCT/CN2017/111809 WO2019075819A1 (en) | 2017-10-20 | 2017-11-20 | Paper grading method and paper grading system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710981922.9A CN109697274B (en) | 2017-10-20 | 2017-10-20 | Examination paper judging method and examination paper judging system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109697274A CN109697274A (en) | 2019-04-30 |
| CN109697274B true CN109697274B (en) | 2020-10-02 |
Family
ID=66172993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710981922.9A Active CN109697274B (en) | 2017-10-20 | 2017-10-20 | Examination paper judging method and examination paper judging system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109697274B (en) |
| WO (1) | WO2019075819A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110263651B (en) * | 2019-05-23 | 2023-06-16 | 五邑大学 | Method, device and storage medium for assisting in on-line reading test questions |
| CN110609953A (en) * | 2019-08-28 | 2019-12-24 | 苏州承儒信息科技有限公司 | Reading and amending method and system for internet education |
| CN110750694A (en) * | 2019-09-29 | 2020-02-04 | 支付宝(杭州)信息技术有限公司 | Data annotation implementation method and device, electronic equipment and storage medium |
| CN110705905B (en) * | 2019-10-15 | 2022-02-08 | 李晚华 | High-accuracy intelligent online paper marking method |
| CN114580375B (en) * | 2022-05-09 | 2022-08-12 | 南京赛宁信息技术有限公司 | Distributed online match subjective question marking and scoring method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593233A (en) * | 2008-05-26 | 2009-12-02 | 北京智慧东方信息技术有限公司 | A kind of appraisal system of Word operation questions |
| CN103942994A (en) * | 2014-04-22 | 2014-07-23 | 济南大学 | Computer assessment method for subjective questions |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150348433A1 (en) * | 2014-05-29 | 2015-12-03 | Carnegie Mellon University | Systems, Methods, and Software for Enabling Automated, Interactive Assessment |
| CN104463101B (en) * | 2014-11-06 | 2017-08-25 | 科大讯飞股份有限公司 | Answer recognition methods and system for character property examination question |
| CN104504023B (en) * | 2014-12-12 | 2017-08-04 | 广西师范大学 | A kind of high-accuracy subjective item computer automatic marking method based on domain body |
| CN106844296A (en) * | 2016-12-01 | 2017-06-13 | 网易（杭州）网络有限公司 | A kind of communication means and device |
-
2017
- 2017-10-20 CN CN201710981922.9A patent/CN109697274B/en active Active
- 2017-11-20 WO PCT/CN2017/111809 patent/WO2019075819A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593233A (en) * | 2008-05-26 | 2009-12-02 | 北京智慧东方信息技术有限公司 | A kind of appraisal system of Word operation questions |
| CN103942994A (en) * | 2014-04-22 | 2014-07-23 | 济南大学 | Computer assessment method for subjective questions |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019075819A1 (en) | 2019-04-25 |
| CN109697274A (en) | 2019-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109697274B (en) | Examination paper judging method and examination paper judging system | |
| Cobos et al. | A content analysis system that supports sentiment analysis for subjectivity and polarity detection in online courses | |
| Shima et al. | Overview of NTCIR-9 RITE: Recognizing Inference in TExt. | |
| CN112631997B (en) | Data processing method, device, terminal and storage medium | |
| Wyatt-Smith et al. | Multimodal reading and comprehension in online environments | |
| Gough | The patterns of interaction between professional translators and online resources | |
| Doolan | Generation 1.5 writing compared to L1 and L2 writing in first-year composition | |
| JP7147185B2 (en) | Information processing device, information processing method and information processing program | |
| Liu et al. | Effects of digital dictionary format on incidental acquisition of spelling knowledge and cognitive load during second language learning: Click-on vs. key-in dictionaries | |
| Khany et al. | A comparative study of introduction and discussion sections of sub-disciplines of applied linguistics research articles | |
| Yen et al. | WriteAhead: Mining grammar patterns in corpora for assisted writing | |
| Dastyar | Dictionary of education and assessment in translation and interpreting studies (TIS) | |
| Rothstein | Accessing relevant literature. | |
| Cheng et al. | Automated tracking of student revisions in response to teacher feedback in EFL writing: technological feasibility and teachers' perspectives | |
| Henkin et al. | Words of estimative correlation: Studying verbalizations of scatterplots | |
| Lee | An editable learner model for text recommendation for language learning | |
| JP2008090117A (en) | Learning system, similar question retrieval method, and program | |
| Goulart | Register variation in L1 and L2 student writing: A multidimensional analysis | |
| Shaw | Essay marking on-screen: Implications for assessment validity | |
| Heilmann | Profiling effects of syntactic complexity in translation: a multi-method approach | |
| KR101226057B1 (en) | Foreign language reading studying system using extracting keywords | |
| Kong et al. | Evaluating a bilingual text-mining system with a taxonomy of key words and hierarchical visualization for understanding learner-generated text | |
| Susilana et al. | Pedagogia Dictionary: Web Application Development. | |
| Liou et al. | A corpus-based study of phrasal verbs with key meanings in TED Talks | |
| Zhang | A study of the effects of user characteristics on mental models of information retrieval systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |