WO2006134759A1

WO2006134759A1 - Sentence evaluating device and sentence evaluating program

Info

Publication number: WO2006134759A1
Application number: PCT/JP2006/310372
Authority: WO
Inventors: Laurence Anthony; Norio Hayashi; Kazuo Sone; Toshio Yamanashi; Kaori Maeda; Yasuko Yamagishi; Yayoi Tsubakimoto
Original assignee: Waseda University; The Japan Institute For Educational Measurement, Inc.
Priority date: 2005-06-15
Filing date: 2006-05-24
Publication date: 2006-12-21
Also published as: KR100932141B1; JPWO2006134759A1; KR20080021771A; JP4165898B2

Abstract

A sentence evaluating device (14) comprises a marking unit (21) for comparing the answer sentence made by the user and the right answer sentences stored in a right answer sentence database (18) and corresponding to the answer sentence, marking the answer sentence for each right answer sentence by a mark deduction method using a below-mentioned criterion, and adopting the height score as the score of the answer sentence. The marking a unit (21) has work group evaluating means (25) for marking the answer sentence in respect of the difference between the word group in the answer sentence and that in the correct answer sentence, a part-of-speech evaluating means (27) for making the answer sentence in respect of the parts of speech of the words in the answer sentence and the correct answer sentence, and final evaluating means (29) for finally marking the answer sentence after the markings by the evaluating means (25 to 28).

Description

Specification

Sentence evaluation apparatus and sentence evaluation program

Technical field

[0001] The present invention relates to a sentence evaluation apparatus and a sentence evaluation program, and more specifically, a sentence evaluation that can objectively and easily evaluate an answer sentence translated in a predetermined language with respect to a question sentence as an original sentence. The present invention relates to a device and a sentence evaluation program.

Background art

[0002] In English education sites such as junior high schools and high schools, scoring of English composition test questions given to students is often performed according to the teacher's own scoring standards. In other words, when the teacher grades the answer sentences created by the students, the answer sentence and the correct answer sentence are compared, and the judgment is made subjectively according to the degree of the difference. However, there are many English sentences that are the correct answers to the question sentences, so it is very time consuming to compare all the correct sentences with the student's answer sentences. Therefore, it is common for scoring teachers to score a single or a few correct sentences as a model answer, and to score a limited model answer against a student's answer sentence. The smaller the number of student answers that exactly match the model answer. The difference between the model answer and the answer is wide-ranging, such as word differences, word position differences, word spelling differences, word omissions, word utilization forms, and tense differences. It is difficult to pattern. From the above, when multiple teachers share the test questions, the evaluation of the difference between the model answer and the answer sentence is left to each teacher's subjectivity. Standards tend not to match perfectly. Even if one teacher scores all students, the scoring standards often change at the beginning and end of the scoring. For this reason, in general, it is difficult to ensure objectivity when scoring English compositions by humans.

[0003] By the way, there is known a translation ability evaluation system that evaluates a translated sentence created by a predetermined translator or automatic translation device (see Patent Document 1), and this system is applied to scoring English compositions. Can be applied. This translation ability evaluation system is a Japanese test A large number of English sentences (correct sentences) that are candidates for correct sentences are stored, and each correct sentence is compared with a translation sentence created corresponding to the Japanese test sentence, and for each correct sentence, An indicator called the translation accuracy rate is required, and the highest translation accuracy rate is adopted. The translation correct answer rate is calculated based on whether or not the same word as the word constituting the translated sentence exists in the same position as the correct sentence.

Patent Document 1: Japanese Patent Application Laid-Open No. 2002-140326

Disclosure of the invention

Problems to be solved by the invention

[0004] However, in the translation ability evaluation system, the translation sentence is evaluated only with the identity of the mutually opposed words between the correct sentence and the translation sentence. There is a risk that translations that do not exist will be evaluated. For example, if there is a group of words that are the same between the correct sentence and the translated sentence, but the positions of these words are different, the ability to derive the same group of words is completely different. There is an inconvenience that it is not considered and treated in the same way as when a completely different group of words is placed.

[0005] The present invention has been devised by paying attention to such inconveniences, and its purpose is to evaluate an answer sentence translated in a predetermined language with respect to a question sentence as an original sentence. The object is to provide a sentence evaluation apparatus and a sentence evaluation program that can be performed objectively.

Means for solving the problem

[0006] (1) In a sentence evaluation apparatus for evaluating an answer sentence translated in a predetermined language with respect to a question sentence as an original sentence in order to achieve the above-mentioned object,

A correct sentence database storing a correct sentence that is a translated sentence of the correct answer to the question sentence, and a scoring unit that performs a scoring process for the answer sentence by comparing the correct sentence and the answer sentence,

The scoring unit includes word group evaluation means for scoring by paying attention to a difference between word groups in the correct sentence and the answer sentence,

The word group evaluation means extracts the same or approximate word group between the correct sentence and the answer sentence, sets the same or approximate word group as an evaluation target word, and determines between the correct sentence and the answer sentence. Differences in the positions of the evaluation target words or the order of the evaluation target words Differentiating the points according to the difference, t.

[0007] (2) Further, the word group evaluation means adopts a configuration in which a difference is given to the score between the same word group and the approximate word group.

[0008] (3) Further, the word group evaluation means may change the word group between the correct answer sentence and the answer sentence into a difference in utilization form between words constituting the word group and Z or spelling error. Too much !, in some cases, it has a structure of approximating words.

[0009] (4) In addition, a dictionary database in which a predetermined word is stored together with its utilization form, the scoring unit based on the dictionary database, the utilization form of the word in the correct sentence and the answer sentence It is equipped with a utilization type evaluation means for scoring paying attention to the difference between

The utilization form evaluation means extracts a word whose utilization form is different between the correct answer sentence and the answer sentence as a new evaluation object word from words excluding the evaluation object word that has already been evaluated. For the target word, it is possible to adopt a configuration in which a score is differentiated according to the difference in position between the correct answer sentence and the answer sentence.

[0010] (5) The scoring unit further includes a spelling evaluation means for scoring by paying attention to a spelling error of a word in the answer sentence.

The spell evaluation unit extracts, as new evaluation target words, the correct sentence and a word whose spelling approximates between the answer sentences from the words excluding the evaluation target words that have already been evaluated. For the target word, it is also possible to adopt a configuration in which a difference is given in the score according to the difference in position between the correct answer sentence and the answer sentence.

[0011] (6) Also, a dictionary database is provided in which a predetermined word is stored together with the type of part of speech.

The scoring unit includes a part-of-speech evaluation unit that scores based on the identity of part-of-speech of words in the correct sentence and the answer sentence based on the dictionary database,

The part-of-speech evaluation means extracts a word whose part-of-speech type matches between the correct sentence and the answer sentence as a new evaluation target word from words excluding the evaluation target word that has already been evaluated. For the evaluation target word, it is possible to adopt a t ヽぅ composition that gives a difference in the score according to the position difference between the correct answer sentence and the answer sentence and the type of Z or part of speech. [0012] (7) Further, the scoring unit includes a final evaluation unit that makes a difference in score according to the number of remaining words excluding the evaluation target word in the correct answer sentence and the answer sentence. The composition can be taken together.

[0013] (8) Here, the final evaluation means, the difference is scored depending on whether or not the remaining word exists in the corresponding position between the correct sentence and the answer sentence It is preferable to adopt

(9) Further, the final evaluation means may adopt a configuration in which a difference is given to the score according to the type of part of speech of the remaining word.

[0015] (10) In the above, a plurality of the correct sentences are stored in the correct sentence database, and the scoring unit performs the scoring process of the answer sentences by the evaluation means for the correct sentences, respectively. It is advisable to adopt a configuration in which the best evaluation among the scoring results obtained for each correct sentence is the evaluation of the answer sentence.

[0016] (11) Further, the present invention compares the correct sentence that is a correct translation of the original problem sentence with the answer sentence that is translated from the above question sentence in a predetermined language. A sentence evaluation program for causing a computer to evaluate the answer sentence,

A word group that is the same or approximate between the correct answer sentence and the answer sentence is extracted as an evaluation object word, and a difference in position of each evaluation object word between the correct answer sentence and the answer sentence or an array of the evaluation object words A configuration is adopted in which the computer is caused to execute a process for differentiating the points according to the difference in order.

(12) Furthermore, the present invention compares the correct sentence that is a correct translation of the question sentence that is the original sentence with the answer sentence that is translated from the question sentence in a predetermined language. A sentence evaluation program that causes a computer to evaluate an answer sentence,

A word group that is the same or approximate between the correct answer sentence and the answer sentence is extracted as an evaluation object word, and a difference in position of each evaluation object word between the correct answer sentence and the answer sentence or an array of the evaluation object words A word group evaluation step that gives different points according to the difference in order;

From the words excluding the evaluation target word in the word group evaluation step, a word having the same part-of-speech type between the correct sentence and the answer sentence is extracted as a new evaluation target word. For the evaluation target word, the difference in position between the correct answer sentence and the answer sentence and a part of speech evaluation step that differentiates the score according to the type of z or part of speech;

A final evaluation step for differentiating the score according to the number of remaining words excluding the evaluation target word in each step,

The computer is executed in order.

[0018] (13) Here, each step is executed for each of a plurality of correct sentences, and the best evaluation is determined as the evaluation of the answer sentence from the scoring results obtained for each correct sentence. Thus, it is preferable to adopt a configuration in which the computer is executed.

[0019] Note that the "word group" is called a chunk, and is used as a general meaning of a single word or a set of a plurality of consecutive words in a predetermined sentence.

In addition, the “same word group” refers to a word group in which the comparison target sentences are different from each other in terms of a word before and after the word group adjacent to the comparison target sentence. For example, referring to Fig. 5, as the word groups that match each other between the answer sentence and the correct answer sentence, in addition to the three word groups in the frame in Fig. 5, "eating", "and", "more" , “Than”, “two”, “more than”, and “t han two”. Among them, the word group in the frame in FIG. 5 becomes the “same word group” in which the words before and after the word group are different between the comparison target sentences.

Further, the “approximate word group” refers to a word that is included in the same word group when a word before or after adjacent to the same word group is different under a predetermined condition.

The invention's effect

[0020] According to the inventions of (1) to (13) above, various evaluations including elements other than the determination of the identity of the word at the same position are made between the correct answer sentence and the answer sentence. Therefore, it is possible to objectively evaluate the answer sentence according to the actual situation.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

Example 1

FIG. 1 shows a schematic system configuration diagram of an English composition learning system according to the present embodiment. In this figure, an English composition learning system 10 is a system for evaluating an English composition created by a user for a predetermined question sentence (original sentence) in Japanese. The English composition learning system The system 10 sends and receives various information to the terminal 11 owned by the user who has been registered in advance and the computer 11 through the network 13 such as the Internet, and evaluates the English composition. And a server 14 as a sentence evaluation device.

[0023] In this English composition learning system 10, when a question sentence is delivered from the server 14 to each terminal 11 of the user and an English answer sentence created by the user is transmitted from the terminal 11 to the server 14 side, In the server 14, scoring, correction, and learning level determination of the answer sentence are automatically performed by a computer, and these results are transmitted to the user terminal 11.

The terminal 11 is not particularly limited, and various devices such as a personal computer, a portable information terminal, a mobile phone, etc. can be adopted as long as information necessary for the present invention can be exchanged. it can.

[0025] The server 14 includes software and Z or hardware, and includes a plurality of program modules such as a processor and Z or processing circuits. The server 14 includes a transmission / reception unit 16 that transmits / receives various information to / from the terminal 11, a problem sentence database 17 in which a large number of problem sentences are stored, and correct answers for each problem sentence in the problem sentence database 17. The correct sentence database 18 storing the correct sentences in English, the dictionary database 19 storing predetermined words together with the types of use and parts of speech, and any problem sentence transmitted from the transceiver 16 to the terminal 11 The question part 20 extracted from the question sentence database 17, the scoring part 21 for scoring the user's answer sentence received by the transmission / reception part 16, and the results scored by the scoring part 21 are stored and aggregated. And a scoring result processing unit 22 for transmitting related information such as a correct sentence together with the scoring result from the transmitting / receiving unit 16 to the terminal 11. It should be noted that the server 14 has other functions that provide various contents and the like, and is not the gist of the present invention, and therefore illustration and description thereof are omitted here.

The correct sentence database 18 stores all possible correct sentences for each question sentence stored in the question sentence database 17.

[0027] The scoring unit 21 compares the answer sentence created by the user with each correct sentence in the correct sentence database 18 corresponding to the answer sentence, and each correct answer sentence is based on the criteria described later. Scoring is performed using the deduction method, and the highest score is scored in the answer sentence. It works.

[0028] The scoring unit 21 includes a word group evaluation means 25 for scoring paying attention to a difference between word groups in the correct sentence and the answer sentence, and a utilization form of words in the correct sentence and the answer sentence. Utilization evaluation means 26 for scoring with a focus on the difference between them, part-of-speech evaluation means 27 for scoring with attention to the identity of the part of speech of the correct sentence and the answer sentence, the correct sentence and the answer sentence Spelling evaluation means 28 for scoring with a focus on misspellings in the words, and final evaluation means 29 for final scoring after scoring by each of these evaluation means 25-28.

[0029] Next, the scoring process of the English composition learning system 10 having the above configuration will be described with examples.

First, when the user requests the server 14 from the terminal 11, a predetermined question sentence is given from the questioning section 20, and the question sentence is transmitted from the transmission / reception section 16 to the terminal 11. The user then composes the question sentence in English and sends the answer sentence from the terminal 11 to the server 14. Here, for the explanation by actual example, it is assumed that the question sentence shown in Fig. 2 is given and the answer sentence shown in the figure is created by the user.

[0031] Then, the scoring unit 21 of the server 14 scores the answer sentences created by the user.

In this case, the user's answer sentence is compared with a number of correct answer sentences stored in the correct sentence database 18, and the score of the answer sentence is obtained as described later for each correct answer sentence. The highest score among the scores obtained for each correct sentence becomes the user's score, and is sent to the scoring result processing unit 22 together with the correct sentence that is the basis for the score, and the user's learning level is determined. Along with the level, various contents such as corrections and explanations of correct sentences are sent to the terminal 11.

Here, scoring by the scoring unit 21 is performed according to the procedure shown in the flowchart of FIG.

First, the user's answer sentence is taken into the scoring unit 21 (S101). Then, one correct answer sentence is extracted from the correct answer sentence database 18 from the correct answer sentences corresponding to the given question sentences. Here, the correct sentence database 18 stores a plurality of correct sentences as shown in FIG. 4, for example. In the following, explanation will be given by taking an example of scoring by comparing with the correct answer sentence “No. 1” in FIG. 4 among these correct answer sentences. First, the word group evaluation means 25 performs scoring focusing on differences between word groups that are one word or a group of a plurality of consecutive words (word group evaluation step: S 103). That is, here, by comparing the answer sentence and the correct sentence, the same word group is extracted between the answer sentence and the correct sentence, and the word group is set as an evaluation target word in the word group evaluation means 25. In the above example, as shown in Fig. 5, the word group enclosed in a frame in each sentence is the same word group between the answer sentence and the correct answer sentence, and these word groups become the evaluation target words. .

[0035] Next, for these evaluation target words, it is determined whether or not the position of the word from the beginning of the sentence matches between the answer sentence and the correct answer sentence. In the above example, the evaluation target word “Jack” has a matching position. The other evaluation target words “eating & 11 (1” and “1110 than two” do not have the same position.

[0036] Then, a deduction point of one point is given to each pair of identical word groups whose positions are different between the answer sentence and the correct answer sentence, and by adding them, the word group evaluation means 25 A deduction point at is required. In the above example, the two sets of words “eating & 11 (1” and “1110 than tw o” do not match each other between the answer sentence and the correct answer sentence, so one deduction point is given for each. Therefore, the total points deducted by the word group evaluation means 25 are 2 points.

[0037] Next, in the utilization form evaluation means 26, scoring is performed for each word except for the word group that is the evaluation target word in the word group evaluation means 25 (utilization form evaluation step: S 104). That is, here, the remaining words that were not evaluated by the word group evaluation means 25 are used between the answer sentence and the correct answer sentence based on the dictionary database 19 by comparing the answer sentence and the correct answer sentence. A word that is only a difference is extracted, and the word is used as an evaluation target word in the utilization type evaluation means 26. In the above example, as shown in Fig. 6, the word "have" and the word "has" surrounded by a frame are just the differences in usage between the answer sentence and the correct answer sentence. Is the evaluation target word here. In the following figures, words marked with “X” mean evaluation target words that have already been evaluated before that. The differences in the forms used here are the same as in the above-mentioned verbs, for example, in the case of nouns, the difference between the singular and plural, and in the case of adjectives and adverbs, the basic, comparative, and superlative classes. There are differences.

[0038] Furthermore, it is determined whether or not the position from the beginning of the sentence for these evaluation target words matches between the answer sentence and the correct answer sentence. In the example above, the word “have” in the answer sentence and the correct answer sentence The word “has” in the same position matches.

[0039] Here, if the position between the answer sentence and the correct sentence is not different as a deduction point for this utilization type evaluation, one point is given only for the difference in the utilization form, while the answer sentence and the correct sentence are given. If there is a difference in position between the two, two points are given for the difference in utilization and position. By summing up these deduction points, deduction points in the utilization evaluation means 26 are obtained. In the above example, the word “have” in the answer sentence and the word “has” in the correct sentence are in the same position, although the utilization form is different. There will be a total of 1 deduction point.

Next, the part-of-speech evaluation means 27 performs scoring focusing on the identity of the part-of-speech for each word excluding the evaluation target words in the word group evaluation means 25 and the utilization form evaluation means 26 (part-of-speech evaluation step). Top: S 105). That is, here, for the remaining words that were not evaluated by each evaluation means 25, 26, the part of speech was determined based on the data in the dictionary database 19, and these were compared by comparing the answer sentence and the correct sentence. The words with the same part of speech are extracted, and the words are evaluated by the part of speech evaluation means 27.

The same part of speech here is determined based on a preset part of speech system. This part-of-speech system is not particularly limited, but is based on major classifications such as “parts of speech” and “adjectives”, and those based on further classification of major classifications such as “common nouns” and “proprietary nouns”. Is done. In this embodiment, a part of speech system based on the latter classification is used.

In the above example, as shown in FIG. 7, the pair of the words “in” and “for” and the pair of the words “minutes” and “hours” surrounded by a frame are placed between the answer sentence and the correct sentence. The part of speech is a string of words that have the same part of speech, and each of these words is an evaluation target word.

Although not in the above-mentioned example, when there are multiple parts of speech that are identical to the part of speech of one sentence in the other sentence, the word close to the head of the other sentence is selected.

[0041] Further, it is determined whether or not the position of the sentence head force matches each other between the answer sentence and the correct sentence for each set of words that are the evaluation target words. In the example above, the pair of the word “in” in the answer sentence and the word “for” in the correct sentence do not match, and the word “minutes” in the answer sentence and the word “hours” in the correct sentence The positions of “” are also inconsistent. [0042] Here, as a deduction point for this part-of-speech evaluation, if there is no difference in position between the answer sentence and the correct answer sentence, one point is given for the difference in words, while between the answer sentence and the correct answer sentence. If the position is different in, two points are given for the difference in word and position. By adding the deduction points, deduction points in the part-of-speech evaluation means 27 are obtained. In the example above, the pair of the word “in” in the answer sentence and the word “for” in the correct sentence have the same part of speech, but the words are different and the positions do not match. Points are awarded. In addition, the pair of the word “minutes” in the answer sentence and the word “hours” in the correct sentence is also given 2 deduction points, and the deduction points in the part-of-speech evaluation means 27 are 4 points in total.

[0043] Next, the spelling evaluation means 28 focuses on the misspelling of the word in the answer sentence for each word excluding the evaluation target words in the word group evaluation means 25, the utilization form evaluation means 26 and the part-of-speech evaluation means 27. Scoring is performed (spell evaluation step: S106). That is, here, for the remaining words that were not evaluated in each evaluation means 25-27, the words whose spelling is approximated between them are extracted by comparing the answer sentence and the correct sentence. This is the target language for evaluation here.

Here, whether or not the spelling is approximated between each word of the answer sentence and the correct sentence is determined using a known Levenshtein distance. This Levenshtein distance depends on the number of steps (cost) that is the smallest of the steps that match the correct answer, taking into account missing characters, incorrect characters, and inserted characters for the words that are compared with each other. It's decided.

For example, if the correct word “puzzle” is spelled out and the word “pzzel” is used in the answer due to a spelling error,

To change the answer sentence “pzzel” to the correct answer sentence “puzzle”

(1) Insert the letter “u” between the letter “p” and the letter “

(2) Insert the character “1” between the character “and the character“ e ”,

(3) By deleting the last character `` 1 '',

These three steps are required, and “cost 3” here is the Levenshtein distance between “puzzle” and “pzzel”.

[0045] Then, when each word to be evaluated is compared between the answer sentence and the correct answer sentence, A pair of words whose one-way distance is equal to or less than a predetermined threshold is determined to be close to each other, and the word in the answer sentence is determined to be a misspelled word.

In the above example, as shown in FIG. 8, the word “dlinking” in the answer sentence enclosed in a frame and the word “drinking” in the correct answer sentence are spelled between the answer sentence and the correct answer sentence. Is a word that approximates, and these words are the words to be evaluated here, and the word “dlinking” in the answer is determined to be a spelling word.

[0047] Further, it is determined whether or not the position from the beginning of the sentence for these evaluation target words is the same between the answer sentence and the correct answer sentence. In the above example, the positions of the word “dlinking” in the answer sentence and the word “drinking” in the correct answer sentence do not match.

[0048] As a deduction point here, if the position between the answer sentence and the correct sentence is the same, one point is given for the spelling error, while the position between the answer sentence and the correct sentence is different. This gives two points for misspellings and position differences. By adding these deduction points, deduction points in the spell evaluation means 28 are obtained. In the above example, the word “dlinking” in the answer sentence and the word “drinking” in the correct sentence do not match, so two deduction points are given, and the deduction in the spell evaluation means 28 is given. A total of 2 points.

[0049] Next, in the final evaluation means 29, each of the evaluation means 25 to 28 described above scores the remaining words that are not regarded as evaluation target words (final evaluation step: S107). That is, here, deduction points are calculated for the remaining words as follows.

First, it is determined whether or not a word exists at the same position between the answer sentence and the correct answer sentence among the words not yet evaluated. In the above example, as shown in FIG. 9, the deduction point here is 0 points between words in the same position.

[0051] Then, among the remaining words, it is determined whether or not the ability to create a word-by-word pair between the answer sentence and the correct answer sentence. Two points are given for each position difference. In the above example, as shown in Fig. 10, the word “thity” in the answer sentence and the word “been” in the correct answer sentence are subject to evaluation, and because it is a set, two points are given as deduction points.

[0052] Finally, for words left in either the answer sentence or the correct answer sentence, one word One deduction point is awarded. In the above example, as shown in Fig. 11, the word “and”, the word “a”, and the word “half” in the correct sentence remain, and three points of three words are given as deduction points. .

[0053] Next, the deduction points obtained by each of the above means are added together, and the degree of approximation of the answer sentence with respect to the correct sentence is obtained by the following equation (1) (S108).

Here, if the degree of approximation is N, the total deduction points are P, the number of words in the answer sentence is W1, and the number of words in the correct sentence is W2,

N = 1-(P / (W1 + W2)) (1)

In the above example, as a result of FIGS. 5 to 11, the total deduction points are 14 points, the number of words in the answer sentence is 11 words, and the number of words in the correct answer sentence is 14 words.

1-(14 / (11 + 14)) = 0.44

[0055] The above scoring process is performed on each correct sentence (S109), and the highest degree of approximation is taken as the score of the answer sentence (S 110), and the correct sentence corresponding to the degree of approximation is the answer sentence. It is subject to correction.

[0056] In the above, the force with one or two points deducted by each means In accordance with each evaluation item, by changing the number of points to be deducted, weighting each item Monkey.

Therefore, according to such an embodiment, it is possible to score an answer sentence with various items, and it is possible to easily perform objective scoring according to the actual situation.

[0058] Next, another embodiment of the present invention will be described. In the following description, the same reference numerals are used for the same or equivalent components as in the first embodiment, and the description is omitted or simplified.

Example 2

The second embodiment has a feature in that the configuration of the scoring unit 21 is changed from that of the first embodiment, and the scoring method of the answer sentence, that is, the calculation method of the deduction points and the approximation degree N is changed.

That is, as shown in FIG. 12, the scoring unit 21 of the present embodiment has a word group evaluation means 25 for scoring paying attention to a difference between word groups in the correct answer sentence and the answer sentence, and A part-of-speech evaluation means 27 for scoring the correct answer sentence and the same part-of-speech word in the answer sentence. And a final evaluation means 29 for final scoring after scoring by each of these evaluation means 25, 27. Word group evaluation means 25, part-of-speech evaluation means 27, and final evaluation means 29 in this order. Point processing is performed.

[0061] The scoring method in the scoring unit 21 of the present embodiment will be described below with examples.

In this embodiment, the same question sentence, correct answer sentence, and answer sentence as in Example 1 are used, and the scoring process for the answer sentence will be described.

[0063] First, the word group evaluation means 25 performs scoring focusing on differences between word groups that are one word or a group of a plurality of consecutive words (word group evaluation step). That is, here, by comparing the answer sentence and the correct answer sentence, the same or approximate word group is extracted between the answer sentence and the correct sentence, and the word group is set as an evaluation target word in the word group evaluation means 25. . Specifically, the same word group between the correct answer sentence and the answer sentence is determined by mutual matching or the like. Then, for the previous or next word that is adjacent to the same word group, it is judged whether there is a difference in usage or spelling mistake between the correct answer and the answer, and a difference in usage or spelling is determined. In this case, the preceding or following word is included in the same word group to obtain an approximate word group. On the other hand, if the previous or next word adjacent to the same word group is not judged to be a difference in usage or misspelling, it will be regarded as the same word group including the word.

[0064] In the above example, the word group enclosed in a frame in each sentence in FIG. 13 is the same or approximated between the answer sentence and the correct answer sentence, and these word groups become the evaluation target words. .

In other words, the word group A “Jack have” in the answer sentence and the word group A “Jack has” in the correct answer sentence have the same “Jack” part, and the subsequent word “have” is the word “Jack has”. For the correct sentence “has”, it is judged that the usage is incorrect, and the word is approximated.

Also, the word group B “eating and dlinking” in the answer sentence and the word group B “eating and drinking J“ eating and ”part in the correct answer sentence are the same, and the“ dlinking ”of the answer sentence that is the subsequent word is the correct answer sentence. The word “drinking” is judged as a spelling error and is regarded as an approximate word group.

Furthermore, the word group C “more than two” in the answer sentence and the correct answer sentence are completely identical to each other, and are therefore regarded as the same word group. It should be noted that the determination of errors in the utilization form and spelling mistakes here is performed in the same manner as the utilization form evaluation means 26 and the spell evaluation means 28 described in the first embodiment.

[0067] Next, with respect to these evaluation target words that also have the same or similar word group power, one point deduction point is given to each correct word for each incorrect word and misspelled word. . In the above example, the word group A “Jack have” in the answer sentence contains one word “have” that is misused with respect to the word group A in the correct sentence, and the word group B “ `` eatin g and dlinking '' contains one word `` dlinking '' that is misspelled with respect to word group B of the correct sentence, so the deduction points for word group A and word group B are 1 point each. Become. In addition, the word group C “more than two” in the answer sentence has no difference in usage and spelling mistakes compared to the word group C in the correct answer sentence, so the deduction points are 0 points.

[0068] Next, paying attention to the difference in the arrangement order of the same or similar word groups, whether or not to give a deduction point is determined. In the above example, the answer sentences are arranged in the order of word group A, word group B, and word group C, and correct answer sentences are arranged in the order of word group A, word group B, and word group C. In such a case, the order is the same and there is no deduction, that is, 0 deduction points.

On the other hand, if the correct sentences are arranged in the order of word group A, word group B, and word group C, the answer sentences are arranged in the order of word group C, word group A, and word group B. To make it the same as the group of words, one step of moving word group C in the answer sentence after word group B is required. Multiply the required number of steps by 5 deduction points to calculate deduction points that focus on the order of word groups. Thus, when the number of required processes is one process, 5 points will be deducted.

Next, the part-of-speech evaluation unit 27 performs scoring focusing on the part-of-speech identity for each word excluding the evaluation target word in the word group evaluation unit 25 (part-of-speech evaluation step). That is, here, as in Example 1, for the remaining words that were not evaluated by the word group evaluation means 25, the part of speech was determined based on the data in the dictionary database 19, and the answer sentence and correct answer sentence were determined. As a result of the comparison, words having the same part of speech are extracted, and these words are evaluated by the part of speech evaluation means 27. In the above example, as shown in FIG. 14, the words “in” and “for” and the words “minutes” and “hours” surrounded by a frame have parts of speech between the answer sentence and the correct sentence, respectively. These are the same words, and these words are the evaluation target words here.

[0070] Here, the deduction points for this part-of-speech evaluation are the same between the answer sentence and the correct answer sentence. It depends on the type of part of speech. Specifically, if the part of speech that is the same is an article, 3 points will be deducted, 7 points will be deducted if it is a preposition, and 10 points will be deducted if it is any other part of speech. In the above example, the word “in” in the answer sentence and the word “for” in the correct sentence have 7 parts of deduction points because the part of speech is the preposition. On the other hand, the word “minutes” in the answer sentence and the word “hours” in the correct sentence are other parts of speech (nouns) other than the preposition, so the deduction point is 10 points and the deduction point in the part-of-speech evaluation means 27 is 17 points. It becomes.

[0071] As in Example 1, it is also determined whether or not the position from the beginning of the sentence is the same between the answer sentence and the correct sentence for the evaluation target word. You can change the number of points deducted.

[0072] Next, in the final evaluation means 29, each of the evaluation means 25 and 27 described above scores the remaining words that have not been evaluated (final evaluation step). Is required.

[0073] That is, for each word of the answer sentence and the correct answer sentence that have not yet been evaluated, deduction points are calculated for each part of speech, and the total deduction points are calculated for the final evaluation means 29. Desired. Specifically, in the same way as the part of speech evaluation means 27 of this example, if it is an article, it will be 3 deduction points, if it is a preposition, if it is another part of speech, it will be 7 points. Is done. Again, the dictionary database 19 described above is used to determine the part of speech type.

In the above example, as shown in FIG. 15, the word “thirty” in the answer sentence, the word “been”, the word “and”, the word “a”, the word “half” in the correct answer sentence, For each of the five words in total, points of deduction by part of speech are given. Here, the word “a” is an article, so there are 3 deduction points, and the other 4 words are not articles and prepositions, so the deduction points are 10 points each. A total of 43 points.

[0075] Note that the points deducted by each of the above means are not particularly limited, and various points can be arbitrarily adopted.

[0076] Next, the deduction points obtained by the above means are added together, and the degree of approximation of the answer sentence with respect to the correct sentence is obtained by the following equation (2). Here, N is the degree of approximation, P is the total deduction points, P is the number of words between the answer sentence and the correct sentence, and Wmax is the number of words. Pmax (10 points in this example)

N = 1-(P / (Wmax X Pmax)) (2)

In the above equation (2), when the approximation P is a negative value, the approximation P is set to 0.

In the above example, as a result of FIGS. 13 to 15, the total deduction points are 62 points, the number of words in the answer sentence is 11 words, and the number of words in the correct answer sentence is 14, so Wmax is 14.

The degree of approximation P is

1-(62 / (14 Χ 10)) = 0.56

[0079] Subsequently, as in Example 1, the above scoring process is performed for each correct sentence, and the highest degree of approximation P is regarded as the score of the answer sentence, and the correct sentence corresponding to the degree of approximation P is used. Will be subject to correction of the answer sentence.

Therefore, the same effects as in the first embodiment can be obtained also in the second embodiment.

Here, as a modification of the second embodiment, there is the following aspect. That is, the word of speech in the answer sentence is replaced by the correct word in the correct sentence after replacing the word that is determined to be misspelled with the correct word group in the word group that has been approximated by the spelling of the word in the answer sentence. It is also possible to perform steps. According to the above example shown in FIG. 13, for the word group B that is determined to be a gold approximate word group due to a spelling error, the word “dlinking” in the answer sentence is replaced with the word “drinking” in the corresponding correct sentence. Thus, for the remaining words that are not evaluated by the word group evaluation means 25, based on the data in the dictionary database 19, the part of speech is determined in consideration of the part of speech of the words before and after the word.

[0082] According to such a modification, it is possible to increase the accuracy in determining the part of speech. Specifically, when part-of-speech discrimination is performed for a word having a plurality of types of part-of-speech functions, the part-of-speech is specified in consideration of the part-of-speech of the word before and after the word. However, if you try to determine the part of speech of the word after the misspelled word (word rdlinkingj in the example in Figure 13) (the word “in” in the figure), the misspelled word may not exist. Because there are many, the part of speech of the misspelled word cannot be determined, and the part of speech of the word after considering the part of speech of the word cannot be specified. In this case, it is limited to one part of speech from a statistical point of view based on frequency of use. In this case, accurate part-of-speech discrimination according to the mode of use cannot be performed. In this regard, according to this modification, by replacing a misspelled word with a correct word, when specifying the part of speech of a word connected to the misspelled word, even if the word has multiple types of part of speech functions, The type of part-of-speech according to the intention of the creator (user) can be specified more accurately.

In each of the above embodiments, a distance learning type English composition learning system using a computer network is illustrated and described. However, the sentence evaluation apparatus of the present invention is not limited to this, and executes the processing of the present invention. Or as a computer with possible programs installed. In addition, a program that can execute the processing of the present invention is distributed to each terminal 11 via the Internet or the like, or a recording medium such as a CD-ROM storing the program is distributed to the user. It is also possible to execute all the above-described processes in the terminal 11 that performs the processing.

[0084] In each of the above embodiments, an answer sentence prepared in English is evaluated with respect to a Japanese question sentence. However, the present invention is not limited to this, and Japanese or other language sentences are evaluated. You may make it evaluate the answer sentence created in another language with respect to a question sentence.

[0085] Furthermore, the present invention can also be applied to the evaluation of translated sentences created by a translator or an automatic translator. For the evaluation of the translation capability of an automatic translator, a series of systems can be realized by linking a system to which the present invention is applied to an automatic translator.

In addition, the configuration of each part of the apparatus according to the present invention is not limited to the illustrated configuration example, and various modifications are possible as long as substantially the same operation is achieved.

Brief Description of Drawings

FIG. 1 is a schematic system configuration diagram of an English composition learning system according to Embodiment 1.

[Figure 2] A diagram showing the question and answer text used in the example

[Figure 3] Flow chart showing the scoring process.

[Fig. 4] An explanatory diagram of the contents stored in the correct sentence database.

[FIG. 5] A chart for explaining scoring in the word group evaluation step.

[Figure 6] A chart for explaining the scoring at the utilization type evaluation step.

[FIG. 7] A chart for explaining scoring in the part of speech evaluation step. [FIG. 8] A chart for explaining the scoring in the spell evaluation step.

圆 9] A chart to explain the scoring at the final evaluation step.

圆 10] A chart to explain the scoring at the final evaluation step.

圆 11] A chart to explain the scoring at the final evaluation step.

FIG. 12 is a schematic system configuration diagram of an English composition learning system according to Embodiment 2. [13] A chart for explaining the scoring at the word group evaluation step.

[14] A chart for explaining the scoring at the part-of-speech evaluation step.

圆 15] A chart to explain the scoring at the final evaluation step.

Claims

The scope of the claims

[1] Sentence evaluation device that evaluates the answer sentence translated in the specified language for the original question sentence ¾ [This; i l /,

The word group evaluation means extracts the same or approximate word group between the correct sentence and the answer sentence, sets the same or approximate word group as an evaluation target word, and determines between the correct sentence and the answer sentence. A sentence evaluation device characterized in that the score is differentiated according to a difference in position of each evaluation target word or a difference in the order of arrangement of the evaluation target words.

2. The sentence evaluation device according to claim 1, wherein the word group evaluation means gives a difference in score between the same word group and an approximate word group.

[3] In the case where the word group evaluation means is that the difference in the word group between the correct answer sentence and the answer sentence is only the difference in the utilization form between the words constituting the word group and the Z or spelling error The sentence evaluation device according to claim 1, wherein the word group is an approximate word group.

[4] It has a dictionary database in which predetermined words are stored along with their usage forms.

The scoring unit includes utilization form evaluation means for scoring based on the difference between utilization forms of words in the correct sentence and the answer sentence based on the dictionary database;

The utilization form evaluation means extracts a word whose utilization form is different between the correct answer sentence and the answer sentence as a new evaluation object word from words excluding the evaluation object word that has already been evaluated. 2. The sentence evaluation apparatus according to claim 1, wherein the target word is given a difference in score according to a difference in position between the correct answer sentence and the answer sentence.

[5] The scoring unit includes spelling evaluation means for scoring with a focus on misspellings of words in the answer sentence,

The spelling evaluation means extracts, as new evaluation target words, words whose spelling approximates between the correct sentence and the answer sentence from words excluding the evaluation target words that have already been evaluated. 5. The sentence evaluation device according to claim 1, wherein the new evaluation target word is given a difference in score according to a difference in position between the correct answer sentence and the answer sentence.

[6] A dictionary database is provided in which a predetermined word is stored together with the type of part of speech.

The part-of-speech evaluation means extracts a word whose part-of-speech type matches between the correct sentence and the answer sentence as a new evaluation target word from words excluding the evaluation target word that has already been evaluated. The evaluation target word is provided with a difference in points according to the difference in position between the correct answer sentence and the answer sentence and the type of Z or part of speech.

5. The sentence evaluation device according to any one of 5.

[7] The scoring unit includes a final evaluation unit that provides a difference in score according to the number of remaining words excluding the evaluation target word in the correct answer sentence and the answer sentence. Term

The sentence evaluation apparatus in any one of 1-6.

8. The final evaluation means, wherein the remaining evaluation means makes a difference in the score depending on whether or not the remaining word exists in a corresponding position between the correct sentence and the answer sentence. Sentence evaluation device.

9. The sentence evaluation apparatus according to claim 7, wherein the final evaluation means adds a difference in score according to the type of part of speech of the remaining word.

[10] The correct sentence database stores a plurality of the correct sentences,

In the scoring unit, the scoring process of the answer sentences by the respective evaluation means is performed for each correct sentence, and the best evaluation among the scoring results obtained for each correct sentence is evaluated. The sentence evaluation device according to any one of claims 1 to 9, wherein

[11] By comparing the correct sentence, which is the correct translation of the original problem sentence, with the answer sentence translated in the specified language for the question sentence, the evaluation of the answer sentence is made available to the computer. A sentence evaluation program to be executed,

The same or approximate word group between the correct answer sentence and the answer sentence is extracted as an evaluation object word, and the difference in position of each evaluation object word between the correct answer sentence and the answer sentence or the evaluation A sentence evaluation program that causes the computer to execute a process of making a difference in the score according to a difference in the order of arrangement of the target words.

[12] By comparing the correct sentence, which is the correct translation of the original problem sentence, with the answer sentence translated in the specified language for the question sentence, the evaluation of the answer sentence is made available to the computer. A sentence evaluation program to be executed,

From the words excluding the evaluation target word in the word group evaluation step, a word having the same part-of-speech type between the correct sentence and the answer sentence is extracted as a new evaluation target word. For the evaluation target word, the part-of-speech evaluation step for differentiating the points according to the position difference between the correct sentence and the answer sentence and the type of Z or part of speech;

A sentence evaluation program which causes the computer to execute in order.

[13] Each step is executed for each of the plurality of correct sentences, and the computer is configured to determine the best evaluation among the scoring results obtained for each of the correct sentences as the evaluation of the answer sentence. The sentence evaluation program according to claim 12, wherein the sentence evaluation program is executed.