CN111767743A - Machine intelligent evaluation method and system for translation test questions - Google Patents
Machine intelligent evaluation method and system for translation test questions Download PDFInfo
- Publication number
- CN111767743A CN111767743A CN202010901018.4A CN202010901018A CN111767743A CN 111767743 A CN111767743 A CN 111767743A CN 202010901018 A CN202010901018 A CN 202010901018A CN 111767743 A CN111767743 A CN 111767743A
- Authority
- CN
- China
- Prior art keywords
- translation
- sentence
- answer
- word
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/51—Translation evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The application relates to examination paper evaluation and discloses a machine intelligent evaluation method and system for translation examination questions, which can greatly improve the evaluation efficiency of the translation examination questions and enable evaluation results to be more objective and accurate. The method comprises the following steps: acquiring a translation original text, an answering translation and a standard translation of a target test question; forming a first word set based on the translation words of each original text keyword in the translation original text; extracting key words in the standard translation to form a second word set; merging and de-duplicating the first word set and the second word set to obtain a keyword set; determining synonyms and near synonyms of each keyword in the keyword set to expand the keyword set; identifying keywords in the answer text according to the expanded keyword set; calculating the smoothness of each sentence in the answer translation; detecting spelling and grammar errors in the answer translations; and scoring the answer text according to the keywords in the answer text, the smoothness of each sentence in the answer text and the detection result of spelling and grammar errors.
Description
Technical Field
The application relates to examination paper evaluation, in particular to a machine intelligent evaluation technology for translation examination questions.
Background
With the development of computer technology, more and more examinations in teaching activities are reviewed in a computer-aided manner.
For language subject examinations, translation test questions are used as core examination question types, and currently, examination questions are evaluated mainly in a mode of manually comparing answer texts of examinees with standard answers, so that time and labor are consumed, and due to the fact that the evaluation criteria of different examinees are greatly different under the influence of subjective factors of the examinees, the evaluation results are often not objective and fair.
Disclosure of Invention
The application aims to provide a machine intelligent evaluation method and system for translation test questions, which can greatly improve the evaluation efficiency of the translation test questions and enable evaluation results to be more objective and accurate.
The application discloses a machine intelligent review method for translation test questions, which comprises the following steps:
acquiring a translation original text, an answering translation and a standard translation of a target test question;
extracting original text keywords in the translated original text, and determining translation words of each original text keyword based on a bilingual mutual translation dictionary library to form a first word set;
extracting key words in the standard translation to form a second word set;
merging and de-duplicating the first word set and the second word set to obtain a keyword set;
determining synonyms and synonyms of each keyword in the keyword set to expand the keyword set;
identifying the keywords in the answer translation according to the expanded keyword set;
calculating the smoothness of each sentence in the answer translation, wherein the smoothness refers to the reasonable degree of the logical sequence and the relation between words in the sentences;
detecting spelling and grammar errors in the answer translations;
and scoring the answer translation according to the keywords in the answer translation, the smoothness of each sentence in the answer translation and the detection result of the spelling and grammar errors.
In a preferred embodiment, the calculating the popularity of each sentence in the answer text, where the popularity refers to the degree of rationality of the logical order and relationship between words in the sentence, further includes:
respectively extracting a word sequence of each sentence in the answer translation and the standard translation;
calculating the probability value of the position of each word in each word sequence in the sentence by adopting a ternary language model according to the Markov assumption;
calculating a word currency probability value of each sentence according to the probability value of the position of each word in the sentence based on a Bayesian conditional probability model;
and calculating the compliance of each sentence in the answer text according to the word compliance probability value of each sentence in the answer text and the standard text.
In a preferred embodiment, the calculating the compliance degree of each sentence in the answer translation according to the word compliance probability value of each sentence in the answer translation and the standard translation further includes:
for each sentence in the answer translation, if the word currency probability value of the sentence is smaller than the word currency probability value of the corresponding sentence in the standard translation, the currency degree of the sentence is equal to the quotient of the word currency probability value of the sentence and the word currency probability value of the corresponding sentence in the standard translation;
and if the word currency probability value of the sentence is greater than or equal to the word currency probability value of the corresponding sentence in the standard translation, the currency degree of the sentence is 1.
In a preferred embodiment, before obtaining the translation original text, the answer text, and the standard text of the target test question, the method further includes:
constructing an original text keyword library, a translated text keyword library, a bilingual mutual translation dictionary library and a synonym thesaurus;
the extracting of the original text keywords in the translated original text and the determining of the translated words of each original text keyword based on the bilingual mutual translation dictionary library to form a first word set further comprises:
extracting original text keywords in the translation original text based on the original text keyword library;
determining translation words of each original text keyword based on a bilingual translation dictionary library to form a first word set;
the extracting the keywords in the standard translation to form a second word set further comprises:
performing sentence segmentation and word segmentation on the standard translation to obtain a word segmentation result;
matching each word in the word segmentation result with the keyword in the translation keyword library to obtain a matched second word set;
the determining synonyms and near synonyms of each keyword in the keyword set to expand the keyword set further includes:
and obtaining synonyms and synonyms of all keywords in the keyword set from the synonym near-sense word library so as to expand the keyword set.
In a preferred embodiment, the scoring the answer text according to the keyword in the answer text, the smoothness of each sentence in the answer text, and the detection result of the spelling and grammar error further includes:
determining the score of a corresponding sentence in the answer translation according to the score of each sentence in the translation original text;
calculating the number of keywords of each sentence in the answering translation and the number of keywords of the corresponding sentence in the standard translation;
calculating the number of spelling errors and the number of grammar errors in each sentence in the translation according to the detection results of the spelling errors and the grammar errors;
according to the formula
Calculating a score for said translation, whereina i Indicating the answeriThe score of the individual sentence is calculated,m i indicating the answeriThe number of keywords of an individual sentence,n i showing the standard translation and the answer translationiEach sentence corresponds to the number of keywords of the sentence,b i indicating the answeriThe sentence smoothness of each sentence is determined,c i indicating the answeriThe number of misspellings of an individual sentence,d i indicating the answeriThe number of grammatical errors of an individual sentence,krepresenting the total number of sentences of the translation,s 1、s 2、s 3、s 4and weight coefficients respectively representing the keyword accuracy, sentence smoothness, number of wrong words and number of wrong grammars of the translation.
In a preferred embodiment, after scoring the answer text according to the keyword in the answer text, the smoothness of each sentence in the answer text, and the detection result of the spelling and grammar error, the method further includes:
calculating the average value of the scores of the translation of all examinees in a preset examination group;
obtaining expected scores of the translation of all examinees in the preset examination group, and calculating an average value of the expected scores;
and adjusting the scores of the translations of the examinees according to the average value of the scores and the average value of the expected scores.
In a preferred embodiment, the adjusting the score of the translated sentence of each examinee according to the average value of the scores and the average value of the expected scores further comprises:
determining an expected average score range according to the average value of the expected scores, determining to adjust the scores upwards when the average value of the scores is smaller than the lower limit value of the expected average score range, and determining to adjust the scores downwards when the average value of the scores is larger than or equal to the upper limit value of the expected average score range;
after the determining to up-regulate the score or the determining to down-regulate the score, further comprising:
calculating a down-regulation or up-regulation base score as an absolute value of a difference between the average of the scores and the average of the expected scores;
according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result;
calculating an up-or down-regulation score for each test in the set of common test takers equal to the up-or down-regulation base score;
according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of excellent tests, whereinTRRepresents the up or down benchmark score,F 0represents the total score of the target test questions,F i represents the second in the set of excellent examineesiThe score of the individual test taker is determined,S i represents the second in the set of excellent examineesiThe up or down scores of individual examinees;
according to the formula
Calculating an upregulation of a score for each test in the set of bad testsOr down-regulating the value, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down adjustment values of each test in the set of bad tests,TRrepresents the up or down benchmark score,S i represents the second in the set of excellent examineesiThe up or down scores of individual examinees;
up-or down-regulating the score of the translation of each test taker according to the calculated up-or down-regulated value of the score of each test taker.
The application also discloses a machine intelligence system of reviewing of translation examination questions includes:
the acquisition module is used for acquiring translation original texts, answering translations and standard translations of the target test questions;
the keyword identification module is used for extracting original text keywords in the translated original text, determining translated words of each original text keyword based on a bilingual translation dictionary library to form a first word set, extracting keywords in the standard translated text to form a second word set, merging and de-duplicating the first word set and the second word set to obtain a keyword set, determining synonyms and synonyms of each keyword in the keyword set to expand the keyword set, and identifying the keywords in the answering translated text according to the expanded keyword set;
the smoothness calculation module is used for calculating the smoothness of each sentence in the answer translation text, and the smoothness refers to the reasonable degree of the logical sequence and the relation between words in the sentences;
the error detection module is used for detecting spelling and grammar errors in the answer translation;
and the scoring calculation module is used for scoring the answer translation according to the keywords in the answer translation, the sentence smoothness in the answer translation and the detection result of the spelling and grammar errors.
The application also discloses a machine intelligence system of reviewing of translation examination questions includes:
a memory for storing computer executable instructions; and the number of the first and second groups,
a processor for implementing the steps in the method as described hereinbefore when executing the computer-executable instructions.
The present application also discloses a computer-readable storage medium having stored therein computer-executable instructions which, when executed by a processor, implement the steps in the method as described above.
Compared with the prior art, the embodiment of the application at least comprises the following advantages and effects:
correct keyword information in answer translated texts is identified based on the translation original texts and the standard translated texts, the particularity of translation test questions, namely specific transliteration and transliteration, is fully considered, synonyms and synonyms of the keywords are expanded according to transliteration results obtained by the keywords of the translation original texts and the transliteration results obtained by the keywords of the standard translated texts, so that the translated matched keywords are more comprehensive and accurate, and the problem that correct keyword information extraction omission is caused by the fact that only the keywords of the standard translated texts and the limitations of the synonyms and the synonyms are considered, and translation scoring is inaccurate is avoided; and scoring the answer text according to the correct keyword information in the identified answer text, the computed compliance of each sentence in the answer text (the compliance refers to the reasonable degree of the logical sequence and the relation between the words in the sentence) and the detection result of spelling and grammar errors in the answer text, and automatically and intelligently scoring the answer text of the examinee by adopting a plurality of dimensions, so that the scoring principle and the scoring standard of the translation test questions are met, the scoring efficiency can be greatly improved compared with manual scoring, and the scoring result is more objective and more accurate.
Furthermore, word and language smoothness probability values in all sentences in the answer translation are calculated based on Markov hypothesis and a Bayesian conditional probability model, so that the smoothness calculation result conforms to the accuracy of natural language expression habits, the word and language smoothness probability values of all sentences in the standard translation are used as judgment standards, the smoothness of all sentences in the answer translation is calculated according to the word and language smoothness values of corresponding sentences in the answer translation, the judgment standards are unified, the grading difference caused by subjective factors of manual evaluation is avoided, and the grading objectivity and the accuracy are further improved.
In addition, the expected scores of the examinees are used as an adjustment standard, and the scores of the examinees are adjusted according to an up-regulation principle of 'less score for high-score examinees and more score for low-score examinees' and a down-regulation principle of 'less score for high-score examinees and more score for low-score examinees', so that the purpose of examination is achieved, and the psychological health problem caused by too large difference between the actual scores and the expected scores of the examinees is avoided.
The present specification describes a number of technical features distributed throughout the various technical aspects, and if all possible combinations of technical features (i.e. technical aspects) of the present specification are listed, the description is made excessively long. In order to avoid this problem, the respective technical features disclosed in the above summary of the invention of the present application, the respective technical features disclosed in the following embodiments and examples, and the respective technical features disclosed in the drawings may be freely combined with each other to constitute various new technical solutions (which are considered to have been described in the present specification) unless such a combination of the technical features is technically infeasible. For example, in one example, the feature a + B + C is disclosed, in another example, the feature a + B + D + E is disclosed, and the features C and D are equivalent technical means for the same purpose, and technically only one feature is used, but not simultaneously employed, and the feature E can be technically combined with the feature C, then the solution of a + B + C + D should not be considered as being described because the technology is not feasible, and the solution of a + B + C + E should be considered as being described.
Drawings
FIG. 1 is a flow chart of a machine intelligent review method of translated test questions according to a first embodiment of the present application.
Fig. 2 is a schematic structural diagram of a machine intelligent review system for translating test questions according to a second embodiment of the present application.
Detailed Description
In the following description, numerous technical details are set forth in order to provide a better understanding of the present application. However, it will be understood by those skilled in the art that the technical solutions claimed in the present application may be implemented without these technical details and with various changes and modifications based on the following embodiments.
Description of partial concepts:
and (3) translation answering: and translating the translated text of which the answer part is taken by the examinee in the test question.
Standard translation: and translating the translated text of the reference answer part in the test question.
Key words: words, phrases/fixed collocations, common expressions/idioms, etc.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The first embodiment of the present application relates to a machine intelligent review method for translated test questions, the flow of which is shown in fig. 1, and the method comprises the following steps:
in step 101, a translation original, a response translation and a standard translation of a target test question are obtained.
Alternatively, if the obtained translation original, answer translation and standard translation of the target test question are in a non-text form, the following steps may be performed after conversion into a text form.
Optionally, before the step 101, the following steps are further included:
and constructing an original text keyword library, a translated text keyword library and a bilingual mutual translation dictionary library.
In general, a corresponding subject can be constructed for different language subjects, namely a keyword library of original texts, a keyword library of translated texts and a bilingual mutual translation dictionary library. For example, in an examination for examining "translate to english" in the english subject, a chinese keyword library, an english keyword library, a bilingual translation dictionary library, and the like need to be constructed for the english subject.
Then, step 102 is entered, the original text keywords in the translated original text are extracted, and the translated words of each original text keyword are determined based on the bilingual translation dictionary base to form a first word set.
Optionally, this step 102 may further comprise the following sub-steps 102 a-102 b:
in step 102a, extracting original text keywords in the translation original text based on the original text keyword library; then, step 102b is executed to determine the translated words of each original text keyword based on the bilingual translation dictionary library to form a first word set. Then, step 103 is entered to extract the keywords in the standard translation to form a second word set. Optionally, the step 103 may further comprise the following sub-steps 103a to 103 b:
in step 103a, performing sentence segmentation and word segmentation on the standard translation to obtain a word segmentation result; and then, executing a step 103b, and matching each participle in the participle result with the keyword in the translation keyword library to obtain the matched second word set.
And then, entering step 104, and merging and de-duplicating the first word set and the second word set to obtain a keyword set.
Then, step 105 is entered to determine synonyms and synonyms of each keyword in the keyword set to expand the keyword set.
Optionally, this step 105 may be further implemented as: and obtaining synonyms and synonyms of all the keywords in the keyword set from the synonym near-sense word library so as to expand the keyword set.
For example, the original text keyword for obtaining the translation original text is K (there are a plurality of original text keywords in one sentence, and for simplicity of explanation, it is assumed that there is one keyword, and a plurality of keywords can be analogized), the translation term of K is queried according to the bilingual inter-translation dictionary library, and a plurality of translation terms are often found, and the translation terms corresponding to K are assumed to be K1, K2, and K3, so as to form a first word set. The keyword K4 is then identified from the standard translation to form a second set of words. And merging and de-duplicating the first word set { K1, K2, K3 } and the second word set { K4} (assuming that no coincidence exists and the situation of coincidence is similar), and obtaining a keyword set { K1, K2, K3, K4 }.
As a more specific example, (for simplicity, there is only one keyword, there may be multiple keywords in a sentence),
original text: his irritation core not with stand and the silence beacon of the night.
Translation: in the face of this quiet night beauty, he was not bothered to refrain from releasing a lot of ice.
Original text key words: beauty
Original text keyword translation word set: first set of words = { beauty, elegance }
The standard translation corresponds to the keywords: second set of words = { beauty view }
Combining the original text keyword translation word sets after duplication removal: keyword set = { beauty, elegance, beauty view }
Based on the expanded keyword set of synonyms and near synonyms: { beautiful, good, nice, elegant, graceful, beautiful, victory, landscape, scenery }
If only the keywords in the standard translation are considered (the keywords in the original text are not considered), the expanded keyword set only has { beauty, victory, landscape and scenery }. If the examinee translates beauty into "nice", although "peaceful and nice at night" is also an acceptable translation, it cannot be scored. This embodiment can accurately recognize such translation. Although this approach may not guarantee that the words in the expanded keyword set are all the most appropriate translations, other subsequent steps (e.g., ranking of the compliance) may be incorporated such that less appropriate translations are not ranked high.
Then, step 106 is entered to identify the keywords in the answer text according to the expanded keyword set.
Optionally, the step 106 may further comprise the following sub-steps 106 a-106 b:
in step 106a, the answering translation is divided into sentences and words to obtain word division results; then, step 106b is executed to match each participle in the participle result with the keywords in the expanded keyword set, so as to obtain the matched keywords (generally including the keywords with correct spelling) in the answer translation.
Then, step 107 is entered to calculate the popularity of each sentence in the answer text, where the popularity refers to the logical order and relationship between words in the sentence. Specifically, the compliance of the sentences in the present application is an evaluation of the sentence level except the keyword level in the translation test question (the "sentence similarity" is not a concept as in the comparison document), is a calculation of the degree of reasonableness of the word order (the logical order and relationship between words) of the sentences, and is calculated based on the language model of a large corpus training.
Optionally, the step 107 may further comprise the following sub-steps 107a to 107 d:
in step 107a, extracting a word sequence of each sentence in the answer translation and the standard translation respectively; then, step 107b is executed, and a probability value of the position of each word in each word sequence in the sentence is calculated by adopting an N-gram language model according to the Markov assumption; then, step 107c is executed, and word smoothness probability values of each sentence are calculated according to the probability values of the positions of the words in the sentence on the basis of the Bayesian conditional probability model; then, step 107d is executed to calculate the compliance of each sentence in the answer translation according to the word compliance probability value of each sentence in the answer translation and the standard translation.
Preferably, the N-gram language model is a trigram language model. In other embodiments, the N-gram language model may also be a binary, quaternary or quinary language model, or the like.
Optionally, this step 107d may be further implemented as: for each sentence in the answer translation, if the word currency probability value of the sentence is smaller than the word currency probability value of the corresponding sentence in the standard translation, the currency of the sentence is equal to the quotient of the word currency probability value of the sentence and the word currency probability value of the corresponding sentence in the standard translation, and if the word currency probability value of the sentence is larger than or equal to the word currency probability value of the corresponding sentence in the standard translation, the currency of the sentence is 1.
Then, step 108 is entered to detect spelling and grammar errors in the answer.
For example, but not limited to, a Language Tool, a Ginger Software, etc. are used to detect word errors and grammar errors in the answer and generate detection results.
It should be noted that the above-mentioned "steps 102 to 106", "step 107" and "step 108" are not limited to be executed sequentially, and may be executed in any other order or executed in parallel.
Then, step 109 is performed to score the answer text according to the keywords in the answer text, the smoothness of each sentence in the answer text, and the detection result of the spelling and grammar errors.
Optionally, the step 109 may further comprise the following sub-steps 109 a-109 d:
in step 109a, determining the score of the corresponding sentence in the reply text according to the score of each sentence in the translation original text; then, step 109b is executed to calculate the number of keywords of each sentence in the answer translation and the number of keywords of the corresponding sentence in the standard translation; then, step 109c is executed to calculate the number of spelling errors and the number of grammar errors in each sentence in the translation according to the detection result of the spelling and grammar errors; step 109d is then performed, according to the formula
Calculating a score for the translation, whereina i Indicating the answer translationiThe score of the individual sentence is calculated,m i indicating the answer translationiThe number of keywords of an individual sentence,n i indicating the standard translation and the answer translationiEach sentence corresponds to the number of keywords of the sentence,b i indicating the answer translationiThe sentence smoothness of each sentence is determined,c i indicating the answer translationiThe number of misspellings of an individual sentence,d i indicating the answer translationiThe number of grammatical errors of an individual sentence,krepresenting the total number of sentences of the translation,s 1、s 2、s 3、s 4and weight coefficients respectively representing the keyword accuracy, sentence smoothness, number of wrong words and number of wrong grammars of the translation.
Optionally, after the step 109, the following step (c) is further included:
in the first step, calculating the average value of the scores of the translation of all examinees in a preset examination group; secondly, executing a step II, acquiring expected scores of the translation of all examinees in the preset examination group, and calculating the average value of the expected scores; and thirdly, executing a step III of adjusting the score of the answer translation according to the average value of the score and the average value of the expected score.
Optionally, the step (c) may further include the following steps:
determining an expected average score range according to the average value of the expected scores;
when the average value of the score is smaller than the lower limit value of the expected average score range, the score is adjusted upwards according to the average value of the score and the average value of the expected score;
and when the average value of the score is larger than or equal to the upper limit value of the expected average score range, the score is downwards adjusted according to the average value of the score and the average value of the expected score.
Optionally, the "up-regulating the score according to the average value of the score and the average value of the expected score" described above is further implemented as the following steps a to f:
a. calculating an upper tuning base score as the absolute value of the difference between the average of the scores and the average of the expected scores; b. according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result; c. calculating the score of each examinee in the common examinee set to be equal to the reference score of the ascending tone; d. according to the formula
Calculating an upregulation value of the score of each test in the set of excellent tests, whereinTRIndicating the up or down benchmark score,F 0the total score of the target test questions is represented,F i indicates the best test groupiThe score of the individual test taker was determined,S i indicates the best test groupiThe up or down scores of individual examinees; e. according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of bad tests, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down score of each test in the bad test set,TRindicating the up or down benchmark score,S i indicates the best test groupiThe up or down scores of individual examinees; f. and the scores of the translation of each examinee are adjusted up according to the calculated up-regulation scores of the scores of each examinee.
Optionally, the "down-regulating the score according to the average value of the score and the average value of the expected score" described above is further realized as the following steps a to F:
A. calculating a down-regulation score as the absolute value of the difference between the average of the scores and the average of the expected scores; B. according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result; C. calculating the downward adjustment value of the score of each examinee in the common examinee set to be equal to the downward adjustment reference value; D. according to the formula
Calculating an up or down score for each test in the set of excellent tests, whereinTRIndicating the up or down benchmark score,F 0the total score of the target test questions is represented,F i indicates the excellenceFirst in the test taker's setiThe score of the individual test taker was determined,S i indicates the best test groupiThe up or down scores of individual examinees; E. according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of bad tests, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down score of each test in the bad test set,TRindicating the up or down benchmark score,S i indicates the best test groupiThe up or down scores of individual examinees; F. and the scores of the translation of each examinee are adjusted downwards according to the calculated downwards adjusted values of the scores of each examinee.
Alternatively, the above-mentioned "determining the expected average score range according to the average of the expected scores" may be set by manual input of the reviewer or by default by the system to the average of the expected scores ± a preset score, which may be, for example, but not limited to, 2, 3, 4, 5, etc.
The second embodiment of the application relates to a machine intelligent evaluation system of translation test questions, which is structurally shown in fig. 2 and comprises an acquisition module, a keyword recognition module, a smoothness calculation module, an error detection module and a grading calculation module.
Specifically, the obtaining module is used for obtaining the translation original text, the answering translation and the standard translation of the target test question.
Alternatively, if the obtained translation original, answer translation, and standard translation of the target test question are in a non-text form, they may be converted into a text form.
Optionally, the system may further include a storage module, which is configured to store the constructed original text keyword library, the translated text keyword library, the bilingual mutual translation dictionary library and the synonym thesaurus. In general, a corresponding subject can be constructed for different language subjects, namely a keyword library of original texts, a keyword library of translated texts and a bilingual mutual translation dictionary library. For example, in an examination for examining "translate to english" in the english subject, a chinese keyword library, an english keyword library, a bilingual translation dictionary library, and the like need to be constructed for the english subject.
The keyword recognition module is used for extracting original text keywords in the translated original text, determining translated words of each original text keyword based on a bilingual translation dictionary library to form a first word set, extracting keywords in the standard translated text to form a second word set, merging and de-duplicating the first word set and the second word set to obtain a keyword set, determining synonyms and synonyms of each keyword in the keyword set to expand the keyword set, and recognizing the keywords in the answering translated text according to the expanded keyword set.
Optionally, the keyword recognition module is further configured to extract original text keywords in the translated original text based on the original text keyword library, and determine a translated word of each original text keyword based on the bilingual translation dictionary library to form a first word set.
Optionally, the keyword recognition module is further configured to perform sentence segmentation and word segmentation on the standard translation to obtain a word segmentation result, and match each word in the word segmentation result with a keyword in the keyword library of the translation to obtain the matched second word set.
Optionally, the keyword recognition module is further configured to obtain synonyms and synonyms of the keywords in the keyword set from the synonym thesaurus to expand the keyword set.
Optionally, the keyword recognition module is further configured to perform sentence segmentation and word segmentation on the answer translation to obtain a word segmentation result, and match each word in the word segmentation result with a keyword in the expanded keyword set to obtain a matched keyword (generally including a keyword with a correct spelling) in the answer translation.
The smoothness calculation module is used for calculating the smoothness of each sentence in the answer translation text, and the smoothness refers to the reasonable degree of the logical sequence and the relation between words in the sentences.
In one embodiment, the compliance calculation module is further configured to extract a word sequence of each sentence in the translation and the standard translation, respectively, calculate a probability value of a position of each word in the word sequence in the sentence according to a markov hypothesis by using an N-gram language model, calculate a word compliance probability value of each sentence according to the probability value of the position of each word in the sentence based on a bayesian conditional probability model, and calculate a compliance of each sentence in the translation according to the word compliance probability value of each sentence in the translation and the standard translation.
Preferably, the N-gram language model is a trigram language model. In other embodiments, the N-gram language model may also be a binary, quaternary or quinary language model, or the like.
Optionally, for each sentence in the answer translation, if the word currency probability value of the sentence is less than the word currency probability value of the corresponding sentence in the standard translation, the currency of the sentence is equal to the quotient of the word currency probability value of the sentence and the word currency probability value of the corresponding sentence in the standard translation, and if the word currency probability value of the sentence is greater than or equal to the word currency probability value of the corresponding sentence in the standard translation, the currency of the sentence is 1.
The error detection module is used for detecting spelling and grammar errors in the answer. For example, but not limited to, a Language Tool, a Ginger Software, etc. are used to detect word errors and grammar errors in the answer and generate detection results.
The scoring calculation module is used for scoring the answer text according to the keywords in the answer text, the sentence smoothness in the answer text and the detection result of the spelling and grammar errors.
Optionally, the score calculating module is further configured to determine a score of a corresponding sentence in the reply translation according to the score of each sentence in the translation source text, calculate the number of keywords of each sentence in the reply translation and the number of keywords of the corresponding sentence in the standard translation, and calculate the number of misspellings and misgrammars in each sentence in the reply translation according to the detection result of the misspellings and the misgrammarsNumber of errors, and according to a formula
Calculating a score for the translation, whereina i Indicating the answer translationiThe score of the individual sentence is calculated,m i indicating the answer translationiThe number of keywords of an individual sentence,n i indicating the standard translation and the answer translationiEach sentence corresponds to the number of keywords of the sentence,b i indicating the answer translationiThe sentence smoothness of each sentence is determined,c i indicating the answer translationiThe number of misspellings of an individual sentence,d i indicating the answer translationiThe number of grammatical errors of an individual sentence,krepresenting the total number of sentences of the translation,s 1、s 2、s 3、s 4and weight coefficients respectively representing the keyword accuracy, sentence smoothness, number of wrong words and number of wrong grammars of the translation.
Optionally, the score calculating module is further configured to calculate an average value of the scores of the translations of all the examinees in the preset test population, obtain expected scores of the translations of all the examinees in the preset test population, calculate an average value of the expected scores, and adjust the scores of the translations according to the average value of the scores and the average value of the expected scores.
Optionally, the score calculating module is further configured to determine an expected average score range according to the average value of the expected scores, adjust the scores upward according to the average value of the scores and the average value of the expected scores when the average value of the scores is less than a lower limit value of the expected average score range, and adjust the scores downward according to the average value of the scores and the average value of the expected scores when the average value of the scores is greater than or equal to an upper limit value of the expected average score range.
Optionally, the score calculation module is further configured to calculate an upper adjusted base score as a difference between the average of the scores and the average of the expected scoresAbsolute value of (d); according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result; calculating the score of each examinee in the common examinee set to be equal to the reference score of the ascending tone; according to the formula
Calculating an up or down score for each test in the set of excellent tests, whereinTRIndicating the up or down benchmark score,F 0the total score of the target test questions is represented,F i indicates the best test groupiThe score of the individual test taker was determined,S i indicates the best test groupiThe up or down scores of individual examinees; according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of bad tests, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down score of each test in the bad test set,TRindicating the up or down benchmark score,S i indicates the best test groupiThe up or down scores of individual examinees; and the scores of the translation of each examinee are adjusted up according to the calculated up-regulation scores of the scores of each examinee.
Optionally, the score calculating module is further configured to calculate a down-regulation score as an absolute value of a difference between the average of the scores and the average of the expected scores; according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result; calculating the downward adjustment value of the score of each examinee in the common examinee set to be equal to the downward adjustment reference value; according to the formula
Calculating an up or down score for each test in the set of excellent tests, whereinTRIndicating the up or down benchmark score,F 0the total score of the target test questions is represented,F i indicates the best test groupiThe score of the individual test taker was determined,S i indicates the best test groupiThe up or down scores of individual examinees; according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of bad tests, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down score of each test in the bad test set,TRindicating the up or down benchmark score,S i indicates the best test groupiThe up or down scores of individual examinees; and the scores of the translation of each examinee are adjusted downwards according to the calculated downwards adjusted values of the scores of each examinee.
Alternatively, the expected average score range may be set, for example, by the reviewer operation or by default by the system, to the average of the expected scores ± a preset score, which may be, for example, but not limited to, 2, 3, 4, 5, etc.
The first embodiment is a method embodiment corresponding to the present embodiment, and the technical details in the first embodiment may be applied to the present embodiment, and the technical details in the present embodiment may also be applied to the first embodiment.
It should be noted that the machine intelligent review method and system for translation test questions related to the present application are mainly used for bilingual inter-translation question types in language subject examinations, such as but not limited to chinese-english inter-translation, chinese-japanese inter-translation, chinese-french inter-translation, english-korean inter-translation, and the like.
It should be noted that, as will be understood by those skilled in the art, the implementation functions of the modules shown in the embodiment of the machine intelligent review system for translating test questions can be understood by referring to the related description of the machine intelligent review method for translating test questions. The functions of the modules shown in the embodiment of the machine-intelligent review system for translating test questions can be realized by a program (executable instructions) running on a processor, and can also be realized by a specific logic circuit. The machine intelligent review system for translating test questions in the embodiment of the application can be stored in a computer readable storage medium if the system is realized in the form of a software functional module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the method of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
Accordingly, the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the computer-executable instructions implement the method embodiments of the present application. Computer-readable storage media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable storage medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
In addition, the embodiment of the application also provides a machine intelligent review system for translating the test questions, which comprises a memory for storing computer executable instructions and a processor; the processor is configured to implement the steps of the method embodiments described above when executing the computer-executable instructions in the memory. The Processor may be a Central Processing Unit (CPU), other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), or the like. The aforementioned memory may be a read-only memory (ROM), a Random Access Memory (RAM), a Flash memory (Flash), a hard disk, or a solid state disk. The steps of the method disclosed in the embodiments of the present invention may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
It is noted that, in the present patent application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the use of the verb "comprise a" to define an element does not exclude the presence of another, same element in a process, method, article, or apparatus that comprises the element. In the present patent application, if it is mentioned that a certain action is executed according to a certain element, it means that the action is executed according to at least the element, and two cases are included: performing the action based only on the element, and performing the action based on the element and other elements. The expression of a plurality of, a plurality of and the like includes 2, 2 and more than 2, more than 2 and more than 2.
All documents mentioned in this application are to be considered as being incorporated in their entirety into the disclosure of this application so as to be subject to modification as necessary. It should be understood that the above description is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present disclosure should be included in the scope of protection of one or more embodiments of the present disclosure.
Claims (10)
1. A machine intelligent review method for translating test questions is characterized by comprising the following steps:
acquiring a translation original text, an answering translation and a standard translation of a target test question;
extracting original text keywords in the translated original text, and determining translation words of each original text keyword based on a bilingual mutual translation dictionary library to form a first word set;
extracting key words in the standard translation to form a second word set;
merging and de-duplicating the first word set and the second word set to obtain a keyword set;
determining synonyms and synonyms of each keyword in the keyword set to expand the keyword set;
identifying the keywords in the answer translation according to the expanded keyword set;
calculating the smoothness of each sentence in the answer translation, wherein the smoothness refers to the reasonable degree of the logical sequence and the relation between words in the sentences;
detecting spelling and grammar errors in the answer translations;
and scoring the answer translation according to the keywords in the answer translation, the smoothness of each sentence in the answer translation and the detection result of the spelling and grammar errors.
2. The method of claim 1, wherein said calculating the popularity of each sentence in said answer, said popularity being a reasonable degree of logical order and relationship between words in the sentence, further comprises:
respectively extracting a word sequence of each sentence in the answer translation and the standard translation;
calculating the probability value of the position of each word in each word sequence in the sentence by adopting a ternary language model according to the Markov assumption;
calculating a word currency probability value of each sentence according to the probability value of the position of each word in the sentence based on a Bayesian conditional probability model;
and calculating the compliance of each sentence in the answer text according to the word compliance probability value of each sentence in the answer text and the standard text.
3. The method of machine-intelligent review of translation questions as set forth in claim 2, wherein said calculating the compliance of each sentence in said answer based on the word compliance probability value of each sentence in said answer and said standard translation further comprises:
for each sentence in the answer translation, if the word currency probability value of the sentence is smaller than the word currency probability value of the corresponding sentence in the standard translation, the currency degree of the sentence is equal to the quotient of the word currency probability value of the sentence and the word currency probability value of the corresponding sentence in the standard translation;
and if the word currency probability value of the sentence is greater than or equal to the word currency probability value of the corresponding sentence in the standard translation, the currency degree of the sentence is 1.
4. The method for machine-intelligent review of translation questions as set forth in claim 1, wherein before the obtaining of the translation original, the answer translation and the standard translation of the target question, further comprising:
constructing an original text keyword library, a translated text keyword library, a bilingual mutual translation dictionary library and a synonym thesaurus;
the extracting of the original text keywords in the translated original text and the determining of the translated words of each original text keyword based on the bilingual mutual translation dictionary library to form a first word set further comprises:
extracting original text keywords in the translation original text based on the original text keyword library;
determining translation words of each original text keyword based on the bilingual mutual translation dictionary library to form a first word set;
the extracting the keywords in the standard translation to form a second word set further comprises:
performing sentence segmentation and word segmentation on the standard translation to obtain a word segmentation result;
matching each word segmentation in the word segmentation result with the keywords in the keyword library to obtain a matched second word set;
the determining synonyms and near synonyms of each keyword in the keyword set to expand the keyword set further includes:
and obtaining synonyms and synonyms of all keywords in the keyword set from the synonym near-sense word library so as to expand the keyword set.
5. The method of machine-intelligent review of translation questions as set forth in claim 1, wherein said scoring said translations based on keywords in said translations, smoothness of sentences in said translations, and detection of said spelling and grammar errors further comprises:
determining the score of a corresponding sentence in the answer translation according to the score of each sentence in the translation original text;
calculating the number of keywords of each sentence in the answering translation and the number of keywords of the corresponding sentence in the standard translation;
calculating the number of spelling errors and the number of grammar errors in each sentence in the translation according to the detection results of the spelling errors and the grammar errors;
according to the formula
Calculating a score for said translation, whereina i Indicating the answeriThe score of the individual sentence is calculated,m i indicating the answeriThe number of keywords of an individual sentence,n i showing the standard translation and the answer translationiEach sentence corresponds to the number of keywords of the sentence,b i indicating the answeriThe sentence smoothness of each sentence is determined,c i indicating the answeriThe number of misspellings of an individual sentence,d i indicating the answeriThe number of grammatical errors of an individual sentence,krepresenting the total number of sentences of the translation,s 1、s 2、s 3、s 4and weight coefficients respectively representing the keyword accuracy, sentence smoothness, number of wrong words and number of wrong grammars of the translation.
6. The method for machine-intelligent review of translation questions according to any one of claims 1-5, wherein after scoring the answer based on the keywords in the answer, the smoothness of each sentence in the answer, and the detection result of the spelling and grammar errors, the method further comprises:
calculating the average value of the scores of the translation of all examinees in a preset examination group;
obtaining expected scores of the translation of all examinees in the preset examination group, and calculating an average value of the expected scores;
and adjusting the scores of the translations of the examinees according to the average value of the scores and the average value of the expected scores.
7. The method of machine-intelligent review of translation questions of claim 6, wherein said adjusting the score of said translation for each test taker based on said average value of scores and said average value of expected scores further comprises:
determining an expected average score range according to the average value of the expected scores, determining to adjust the scores upwards when the average value of the scores is smaller than the lower limit value of the expected average score range, and determining to adjust the scores downwards when the average value of the scores is larger than or equal to the upper limit value of the expected average score range;
after the determining to up-regulate the score or the determining to down-regulate the score, further comprising:
calculating a down-regulation or up-regulation base score as an absolute value of a difference between the average of the scores and the average of the expected scores;
according to the ranking of all the examinees in the preset examination group from high to low according to the scores of the translation, dividing each examinee and the scores thereof into an excellent examinee set, a common examinee set and a poor examinee set according to the ranking result;
calculating an up-or down-regulation score for each test in the set of common test takers equal to the up-or down-regulation base score;
according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of excellent tests, whereinTRRepresents the up or down benchmark score,F 0represents the total score of the target test questions,F i represents the second in the set of excellent examineesiThe score of the individual test taker is determined,S i represents the second in the set of excellent examineesiThe up or down scores of individual examinees;
according to the formula
Calculating an up-or down-regulation value of the score of each test in the set of bad tests, whereinfRepresenting the number of test takers in the set of excellent test takers,hrepresenting the number of test takers in the set of bad test takers,Srepresenting the up or down adjustment values of each test in the set of bad tests,TRrepresents the up or down benchmark score,S i represents the second in the set of excellent examineesiThe up or down scores of individual examinees;
up-or down-regulating the score of the translation of each test taker according to the calculated up-or down-regulated value of the score of each test taker.
8. A machine intelligence system of reviewing of translation examination questions, comprising:
the acquisition module is used for acquiring translation original texts, answering translations and standard translations of the target test questions;
the keyword identification module is used for extracting original text keywords in the translated original text, determining translated words of each original text keyword based on a bilingual translation dictionary library to form a first word set, extracting keywords in the standard translated text to form a second word set, merging and de-duplicating the first word set and the second word set to obtain a keyword set, determining synonyms and synonyms of each keyword in the keyword set to expand the keyword set, and identifying the keywords in the answering translated text according to the expanded keyword set;
the smoothness calculation module is used for calculating the smoothness of each sentence in the answer translation text, and the smoothness refers to the reasonable degree of the logical sequence and the relation between words in the sentences;
the error detection module is used for detecting spelling and grammar errors in the answer translation;
and the scoring calculation module is used for scoring the answer translation according to the keywords in the answer translation, the sentence smoothness in the answer translation and the detection result of the spelling and grammar errors.
9. A machine intelligence system of reviewing of translation examination questions, comprising:
a memory for storing computer executable instructions; and the number of the first and second groups,
a processor for implementing the steps in the method of any one of claims 1 to 7 when executing the computer-executable instructions.
10. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement the steps in the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010901018.4A CN111767743B (en) | 2020-09-01 | 2020-09-01 | Machine intelligent evaluation method and system for translation test questions |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010901018.4A CN111767743B (en) | 2020-09-01 | 2020-09-01 | Machine intelligent evaluation method and system for translation test questions |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111767743A true CN111767743A (en) | 2020-10-13 |
| CN111767743B CN111767743B (en) | 2020-11-27 |
Family
ID=72729148
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010901018.4A Active CN111767743B (en) | 2020-09-01 | 2020-09-01 | Machine intelligent evaluation method and system for translation test questions |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767743B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112836529A (en) * | 2021-02-19 | 2021-05-25 | 北京沃东天骏信息技术有限公司 | Method and device for generating target corpus sample |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101777044A (en) * | 2010-01-29 | 2010-07-14 | 中国科学院声学研究所 | System for automatically evaluating machine translation by using sentence structure information and implementing method |
| CN109460558A (en) * | 2018-12-06 | 2019-03-12 | 云知声（上海）智能科技有限公司 | A kind of effect evaluation method of speech translation system |
| CN109948166A (en) * | 2019-03-25 | 2019-06-28 | 腾讯科技（深圳）有限公司 | Text interpretation method, device, storage medium and computer equipment |
| US10649962B1 (en) * | 2017-06-06 | 2020-05-12 | Amazon Technologies, Inc. | Routing and translating a database command from a proxy server to a database server |
-
2020
- 2020-09-01 CN CN202010901018.4A patent/CN111767743B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101777044A (en) * | 2010-01-29 | 2010-07-14 | 中国科学院声学研究所 | System for automatically evaluating machine translation by using sentence structure information and implementing method |
| US10649962B1 (en) * | 2017-06-06 | 2020-05-12 | Amazon Technologies, Inc. | Routing and translating a database command from a proxy server to a database server |
| CN109460558A (en) * | 2018-12-06 | 2019-03-12 | 云知声（上海）智能科技有限公司 | A kind of effect evaluation method of speech translation system |
| CN109948166A (en) * | 2019-03-25 | 2019-06-28 | 腾讯科技（深圳）有限公司 | Text interpretation method, device, storage medium and computer equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112836529A (en) * | 2021-02-19 | 2021-05-25 | 北京沃东天骏信息技术有限公司 | Method and device for generating target corpus sample |
| CN112836529B (en) * | 2021-02-19 | 2024-04-12 | 北京沃东天骏信息技术有限公司 | Method and device for generating target corpus sample |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111767743B (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11449556B2 (en) | Responding to user queries by context-based intelligent agents | |
| US9959776B1 (en) | System and method for automated scoring of texual responses to picture-based items | |
| US9218339B2 (en) | Computer-implemented systems and methods for content scoring of spoken responses | |
| KR101680007B1 (en) | Method for scoring of supply type test papers, computer program and storage medium for thereof | |
| US20170177563A1 (en) | Methods and systems for automated text correction | |
| François et al. | FLELex: a graded lexical resource for French foreign learners | |
| CN110727796B (en) | Multi-scale difficulty vector classification method for graded reading materials | |
| US20120150529A1 (en) | Method and apparatus for generating translation knowledge server | |
| US20220318509A1 (en) | Entity recognition method and device, dictionary creating method, device and medium | |
| Toselli et al. | Making two vast historical manuscript collections searchable and extracting meaningful textual features through large-scale probabilistic indexing | |
| CN111914532A (en) | Chinese composition scoring method | |
| Pasini et al. | CluBERT: A cluster-based approach for learning sense distributions in multiple languages | |
| CN110659352A (en) | Test question and test point identification method and system | |
| Khader et al. | Textual entailment for Arabic language based on lexical and semantic matching | |
| CN115130465A (en) | Method and system for identifying knowledge graph entity annotation error on document data set | |
| CN107122582B (en) | diagnosis and treatment entity identification method and device facing multiple data sources | |
| Wadud et al. | Text coherence analysis based on misspelling oblivious word embeddings and deep neural network | |
| CN111767743B (en) | Machine intelligent evaluation method and system for translation test questions | |
| CN112214579B (en) | Machine intelligent review method and system for short answer questions | |
| Daðason | Post-correction of Icelandic OCR text | |
| Hao et al. | SCESS: a WFSA-based automated simplified chinese essay scoring system with incremental latent semantic analysis | |
| Stadsnes | Evaluating semantic vectors for norwegian | |
| Orasmaa et al. | Named entity recognition in Estonian 19th century parish court records | |
| Chiu et al. | Chinese spell checking based on noisy channel model | |
| Uban et al. | Automatically building a multilingual lexicon of false friends with no supervision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220909 Address after: Baiyun District of Guangzhou City, Guangdong province 510540 North Road No. 1633 is private science and Technology Park Branch Road, No. 1 Patentee after: Guangzhou Blue Pigeon Software Co.,Ltd. Address before: No. 1968, Nanxi East Road, Nanhu District, Jiaxing City, Zhejiang Province Patentee before: ZHEJIANG LANCOO TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |