WO2018116459A1

WO2018116459A1 - Sentence evaluation device, sentence evaluation method, and recording medium

Info

Publication number: WO2018116459A1
Application number: PCT/JP2016/088458
Authority: WO
Inventors: 英恵下村
Original assignee: 日本電気株式会社
Priority date: 2016-12-22
Filing date: 2016-12-22
Publication date: 2018-06-28
Also published as: JPWO2018116459A1; JP6717387B2

Abstract

Provided is a technology which makes it possible to perform evaluation for a logical relationship between elements included in a paper to be graded. A sentence evaluation device is provided with: an acquisition unit for acquiring a first logical relationship established between first events that constitute a first sentence; a logical relationship extraction unit for extracting, from a second sentence to be evaluated, a second logical relationship established between second events that constitute the second sentence; and an evaluation unit for evaluating the second sentence by comparing the first logical relationship and the second logical relationship, the first sentence being a sentence serving as a reference for the evaluation of the second sentence.

Description

Sentence evaluation apparatus, sentence evaluation method, and recording medium

This disclosure relates to a sentence evaluation device, a sentence evaluation method, and a recording medium.

A scoring support system has been developed to reduce the burden of manually scoring written answers. For example, in Patent Document 1, a predetermined feature group (Feature set) independent of a question (Test prompt) is defined, and a score is assigned according to a scoring equation weighted based on scored data or the like An example of a scoring support system is described.

Further, in Patent Document 2, a plurality of essays are evaluated to generate a feature group, and the weight of each feature is generated based on a score category and a feature group determined from each of the plurality of essays. Generating a model based on the weight for is described. The technique described in Patent Document 2 evaluates an essay according to a generated model.

US Patent Application Publication No. 2015/0199913 US Patent Application Publication No. 2011/0027769

Since the technique described in Patent Document 1 determines a score against general characteristics of a good essay such as grammar, length, and vocabulary, the essay is determined based on an evaluation standard specific to a certain question. Difficult to evaluate.

Since the technique described in Patent Document 2 can extract a unique feature for each question, an essay can be evaluated based on an evaluation standard specific to a certain question, and a score point associated with a feature included in the essay Since scoring is performed using, a score is given to the entire essay. Therefore, even if there are places in the essay that are intended to be evaluated by the examiner or evaluator of the problem, such as the logical relationship between the elements contained in the essay, the evaluation should be performed for each part. Is difficult.

Here, in order to grasp the contents of a sentence as a logical relationship, first, a technique for converting a sentence written in a natural language into a logical expression is required. An example of a technique for converting an English sentence into a logical expression is described in Non-Patent Document 1, and an example of a technique for converting a Japanese sentence into a logical expression is described in Non-Patent Document 2. According to these technologies, even if the sentences have the same content but different expressions, such as “Taro entered the room” and “Taro entered the room”, they are generalized and identical. Can be converted to a logical expression.

Furthermore, in the techniques described in Patent Document 1 and Patent Document 2, there is a high possibility that the evaluation is determined by the sufficiency of elements extracted as features. Therefore, in these techniques, if the elements are substantial, there is a possibility that a high evaluation will be obtained even if the logic flow is not correct. That is, with these techniques, there is a possibility that a logical relationship included in a sentence (for example, an answer to be evaluated) cannot be correctly evaluated.

The present disclosure has been made in view of the above problems, and an object thereof is to provide a technique capable of evaluating a logical relationship between elements included in a sentence to be evaluated.

A sentence evaluation device according to an aspect of the present disclosure includes: an acquisition unit that acquires a first logical relationship that is established between first events that constitute a first sentence; and a second sentence that is an evaluation target, the second sentence Evaluation of evaluating the second sentence by comparing the first logical relation with the second logical relation, and a logical relation extracting means for extracting the second logical relation that is established between the second events constituting the sentence And the first sentence is a sentence serving as a reference for evaluation of the second sentence.

The sentence evaluation method according to one aspect of the present disclosure acquires a first logical relationship that is established between first events constituting a first sentence that is a reference for evaluation, and the second sentence that is an evaluation target includes the first logical relationship. A second logical relationship established between second events constituting the second sentence is extracted, and the second sentence is evaluated by comparing the first logical relation with the second logical relation.

Note that a computer program that realizes the above-described apparatus or method by a computer and a computer-readable non-transitory recording medium that stores the computer program are also included in the scope of the present disclosure.

According to the present disclosure, it is possible to evaluate the logical relationship between elements included in the sentence to be evaluated.

It is a functional block diagram which shows an example of a function structure of the text evaluation apparatus which concerns on 1st Embodiment. It is a block diagram which shows an example of a structure of the text evaluation system containing the text evaluation apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of a process of the text evaluation apparatus which concerns on 1st Embodiment. It is a functional block diagram which shows an example of a function structure of the text evaluation apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the flow of a process which stores a scoring standard in the scoring standard memory | storage part in the text evaluation apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the scoring standard stored in the scoring standard memory | storage part. It is a flowchart which shows an example of the flow of the answer scoring process in the text evaluation apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the scoring process which a scoring part performs. It is a figure which shows an example of the scoring standard stored in the scoring standard memory | storage part. It is a flowchart which shows an example of the flow of the answer scoring process in the text evaluation apparatus which concerns on the modification 1 of 2nd Embodiment. It is a functional block diagram which shows an example of a function structure of the text evaluation apparatus which concerns on 3rd Embodiment. It is a block diagram which shows an example of a structure of the text evaluation system containing the text evaluation apparatus which concerns on 3rd Embodiment. It is a flowchart which shows an example of the flow of a process which stores a scoring standard in the scoring standard memory | storage part in the text evaluation apparatus which concerns on 3rd Embodiment. It is a figure which shows an example of the graph displayed on a display apparatus. It is a figure which shows an example of the screen when the weight input with respect to the graph shown in FIG. 14 is displayed on a display apparatus with the graph. It is a figure which shows an example of a function structure of the text evaluation apparatus which concerns on 4th Embodiment. It is a figure which shows an example of the answer information stored in an answer memory | storage part. It is a flowchart which shows an example of the flow of the answer scoring process in the text evaluation apparatus which concerns on 4th Embodiment. It is a figure which shows an example of the scoring result which the text evaluation apparatus which concerns on 4th Embodiment outputs. It is a figure which shows an example of a function structure of the text evaluation apparatus which concerns on 5th Embodiment. It is a figure which shows an example of the scoring standard which the scoring standard memory | storage part in 5th Embodiment stores. It is a figure which illustrates illustartively the hardware constitutions of the computer (information processing apparatus) which can implement | achieve each embodiment.

<First Embodiment>
A first embodiment of the present disclosure will be described with reference to the drawings. FIG. 1 is a functional block diagram illustrating an example of a functional configuration of the sentence evaluation apparatus 10 according to the present embodiment. FIG. 2 is a block diagram showing an example of the configuration of the text evaluation system 1 including the text evaluation device 10 shown in FIG.

2, the sentence evaluation system 1 includes a sentence evaluation device 10, an evaluation reference storage device 20, and an output device 30. The sentence evaluation apparatus 10 is communicably connected to the evaluation criterion storage apparatus 20 and the output apparatus 30 via a network or not.

The evaluation criterion storage device 20 stores information representing the first logical relationship that is established between the first events constituting the first sentence. The first sentence is a sentence serving as a reference for evaluation with respect to a second sentence described later. The first sentence is, for example, a model answer. In the following description, it is assumed that the first sentence is a model answer. The model answer indicates an example of a model answer for a certain problem associated with a second sentence (for example, an answer) to be evaluated by the evaluation unit 13 described later. The model answer is described in a natural sentence, for example. For example, the evaluation criterion storage device 20 may store information representing the first logical relationship obtained by converting a natural sentence as a model answer into a logical expression using the technique described in Non-Patent Document 2. Good. The evaluation criterion storage device 20 may be realized by a hard disk drive or the like, for example.

Here, the first event is, for example, an element included in the logical expression. The logical expression is expressed by the method described in Non-Patent Document 2, for example. The first event is expressed by, for example, a keyword or a key phrase. For example, the model answer is "War has occurred. Therefore, financial difficulties have occurred." Suppose there is. A schematic representation of this logical expression is called logical expression information. In this case, (war) and (financial difficulties) are the first events, and the arrow (→) between (war) and (financial difficulties) is the first logical relationship that holds between the first events. In the case of this example, the first logical relationship is a causal relationship in which the first event at the starting point of the arrow is the cause and the first event at the end point of the arrow is the result. Further, the first logical relationship is not limited to the causal relationship, and may be, for example, a hierarchical relationship representing a relationship of the lower concept to the higher concept, or a relationship (simultaneously indicating that the first event is an event that occurs simultaneously. It may be a relationship), or may be another relationship. As described above, the first logical relationship is a logical relationship established between the first events extracted from the logical expression, and includes information on the first events and information representing these relationships.

The output device 30 outputs a scoring result of the evaluation unit 13 described later. For example, the output device 30 may be a display device such as a liquid crystal display, or may be a printing device such as a printer.

Further, the evaluation criterion storage device 20 and the output device 30 may be incorporated in the sentence evaluation device 10.

As shown in FIG. 1, the sentence evaluation apparatus 10 according to the present embodiment includes an acquisition unit 11, a logical relationship extraction unit 12, and an evaluation unit 13.

The acquisition unit 11 acquires the first logical relationship that is established between the first events constituting the model answer. For example, the acquisition unit 11 acquires the first logical relationship from the evaluation criterion storage device 20. Note that the acquisition unit 11 may acquire the first logical relationship by converting the model answer input to the sentence evaluation device 10 into a logical expression instead of from the evaluation criterion storage device 20. The method by which the acquisition unit 11 acquires the first logical relationship is not particularly limited. The acquisition unit 11 supplies the acquired first logical relationship to the evaluation unit 13.

The logical relationship extraction unit 12 extracts a second logical relationship that is established between the second events that constitute the second sentence, from the second sentence that is the evaluation target. The second sentence is, for example, an answer. The answer is written in a natural sentence, for example. The logical relationship extraction unit 12 extracts the second logical relationship by converting the answer written in the natural sentence into a logical expression using the technique described in Non-Patent Document 2, for example. The logical relationship extraction unit 12 supplies the extracted second logical relationship to the evaluation unit 13. Hereinafter, the second sentence to be evaluated will be described as an answer to be evaluated.

Here, the second event is an element included in the logical expression like the first event, and is expressed by, for example, a keyword or a key phrase. The second logical relationship is a logical relationship established between such second events, for example, a causal relationship and a simultaneous relationship.

The evaluation unit 13 scores the answer to be evaluated by comparing the first logical relationship and the second logical relationship. The evaluation unit 13 receives the first logical relationship from the acquisition unit 11. The evaluation unit 13 receives the second logical relationship from the logical relationship extraction unit 12. Then, the evaluation unit 13 compares the first logical relationship and the second logical relationship, and determines whether or not they represent the same logical relationship. The evaluation unit 13 determines that the second logical relationship included in the answer is correct when the first logical relationship and the second logical relationship represent the same logical relationship. Then, the answer including the second logical relationship is scored. For example, when a predetermined score is associated with the first logical relationship, the evaluation unit 13 determines the predetermined score associated with the first logical relationship that represents the same logical relationship as the second logical relationship determined to be correct. Is used to evaluate the answer by scoring the answer.

FIG. 3 is a flowchart showing an example of the processing flow of the sentence evaluation apparatus 10 according to the present embodiment. As illustrated in FIG. 3, the acquisition unit 11 of the sentence evaluation device 10 according to the present exemplary embodiment acquires a first logical relationship that is established between the first events constituting the model answer (step S31). Further, the logical relationship extraction unit 12 extracts a second logical relationship that is established between the second events constituting the evaluation target answer from the evaluation target answer (step S32). Then, the evaluation unit 13 scores the answer to be evaluated by comparing the first logical relationship and the second logical relationship (step S33).

Thus, the sentence evaluation device 10 ends the process.

According to the sentence evaluation apparatus 10 according to the present embodiment, the first logical relationship that is established between the first events that constitute the model answer and the second logical relationship that is established between the second events that constitute the answer to be evaluated. The answer to be evaluated is evaluated by comparing. As described above, the first logical relationship and the second logical relationship are, for example, causal relationships. For example, the model answer is expressed by logical expression information “(A) → (B)”, the arrow (→) between (A) and (B) is caused by (A), and (B) is the result. The first event is (A) and (B), and the first logical relationship established between the first event (A) and the first event (B) is the first event (A ) And the causal relationship with the first event (B) as a result.

When the second logical relationship that is established between the second events constituting the answer is a causal relationship that is caused by the second event (A) and the second event (B), the first logical relationship, Since the two logical relationships are the same logical relationship, the evaluation unit 13 can determine that the second logical relationship included in the answer is correct. Therefore, the evaluation unit 13 can give a predetermined score, for example, to the second logical relationship included in this answer.

In addition, when the second logical relationship established between the second events constituting the answer is a causal relationship that causes the second event (B) and results in the second event (A), the second logical relationship is included. Logical expression information schematically expressing the logical expression is “(B) → (A)”. Since the logical expression information schematically expressing the logical expression including the first logical relationship is “(A) → (B)” as described above, the second event (A) and the second event ( B) is the same as the first event (A) and the first event (B) constituting the model answer, but is a logical relationship different from the first logical relationship and the second logical relationship. Therefore, the evaluation unit 13 can determine that the second logical relationship included in this answer is incorrect. Therefore, the evaluation unit 13 can give, for example, no score or a minus score to the second logical relationship included in the answer.

As described above, the sentence evaluation apparatus 10 according to the present embodiment has the second logical relationship that is established between the second events even if the second event that is an element included in the answer to be scored is the same as the first event. And the first logical relationship are compared to evaluate the answer. Therefore, the sentence evaluation apparatus 10 can correctly evaluate the correctness of the second logical relationship that is the logical relationship between the second events constituting the answer. That is, the sentence evaluation apparatus 10 can evaluate the logical relationship between elements included in the answer to be scored.

<Second Embodiment>
Next, a second embodiment of the present disclosure based on the above-described first embodiment will be described with reference to the drawings. FIG. 4 is a functional block diagram showing an example of a functional configuration of the sentence evaluation apparatus 100 according to the present embodiment. Note that the sentence evaluation apparatus 100 shown in FIG. 4 is described as an apparatus in which the sentence evaluation apparatus 10 and the evaluation reference storage apparatus 20 described above are integrated.

As shown in FIG. 4, the sentence evaluation apparatus 100 according to the present embodiment includes an exemplary answer receiving unit 110, a first extracting unit 120, a weight receiving unit 130, a scoring reference storage unit (evaluation reference storage device) 140, and an answer receiving unit. 150, a second extraction unit (logical relationship extraction unit) 160, an acquisition unit 170, and a scoring unit (evaluation unit) 180.

The model answer receiving unit 110 receives a model answer for the answer to be evaluated. The model answer is assumed to be a natural sentence in the present embodiment. The model answer accepting unit 110 accepts a model answer input by, for example, a question issuer, a model answer creator, or a problem evaluator using an input device such as a keyboard. In the following, questions that are asked by the question, creators of model answers, or evaluators of the problem are collectively referred to as simply evaluators. The model answer accepting unit 110 supplies the accepted model answer to the first extracting unit 120. In the present embodiment, as an answer evaluation method, an example is given in which the answer is scored. Therefore, in the following, the answer to be evaluated is also called the answer to be scored.

The first extraction unit 120 extracts a first logical relationship, which is a logical relationship established between events constituting the model answer, from the model answer. First, the first extraction unit 120 receives a model answer written in a natural sentence from the model answer receiving unit 110. The first extraction unit 120 converts the natural sentence that is the model answer into a logical expression. The first extraction unit 120 extracts the first logical relationship from the natural sentence by converting the natural sentence into a logical expression using the technique described in Non-Patent Document 2, for example. The first extraction unit 120 stores, in the scoring criterion storage unit 140, a plurality of first events included in the converted logical expression and a first logical relationship that holds between the first events.

Here, the first event indicates an element included in the logical expression. The first event is expressed by, for example, a keyword or a key phrase. An element may include the concept of a word. Specific examples of the first event and the first logical relationship will be described later. Hereinafter, the first event is also referred to as a first element.

The weight acceptance unit 130 accepts input of weights for each first element and each first logical relationship included in the model answer converted into a logical expression. The weight is, for example, a point assigned to each first element and each first logical relationship. The weight is the first logical relationship that matches the first event and the second logical relationship that match the second event when scoring an answer that includes the second event (described later) and the second logical relationship (described later). The number of points (partial points) given for. The weight may be a value (parameter) relative to a reference value (score), or may be a score itself. The weight can also be referred to as evaluation information (evaluation value) representing evaluation for the first event and the first logical relationship. In the present embodiment, the weight receiving unit 130 will be described as receiving points for each first element and each first logical relationship. The weight receiving unit 130 associates the received score with the related first element or first logical relationship stored in the scoring reference storage unit 140.

The scoring criterion storage unit 140 uses the scoring criterion (evaluation criterion) to associate the first element and the first logical relationship included in the logical expression converted by the first extracting unit 120 and the weight accepted by the weight accepting unit 130 with each other. ). The scoring unit 180 described later performs scoring according to the weight included in the scoring standard. An example of the scoring standard stored in the scoring standard storage unit 140 will be described later with different drawings.

The answer reception unit 150 receives an answer to be graded. In the present embodiment, it is assumed that the answer to be graded is a natural sentence. In the following, the answer to be scored is also simply referred to as an answer. The answer receiving unit 150 receives the answer by receiving the answer inputted by the creator of the answer via the network or the like, for example, via the network or the like. The answer reception unit 150 supplies the received answer to the second extraction unit 160.

The second extraction unit 160 is an example of the logical relationship extraction unit 12 in the first embodiment described above. The second extraction unit 160 extracts the second logical relationship that holds between the second events that constitute the scoring target answer from the scoring target answer. First, the second extraction unit 160 receives an answer written in a natural sentence. The second extraction unit 160 converts the natural sentence included in the received answer into a logical expression. The method in which the second extraction unit 160 converts the natural sentence included in the answer into a logical expression may be the same as or different from the method in which the first extraction unit 120 converts the natural sentence that is the model answer into the logical expression. May be. The second extraction unit 160 extracts a second logical relationship from the answer by converting a natural sentence included in the answer into a logical expression using an arbitrary method. The second extraction unit 160 supplies the second event included in the converted logical expression and the second logical relationship established between the second events to the scoring unit 180, respectively.

Here, the second event is an element included in the logical expression like the first event, and is expressed by, for example, a keyword or a key phrase. Specific examples of the second event and the second logical relationship will be described later. Hereinafter, the second event is also referred to as a second element.

The acquisition unit 170 is an example of the acquisition unit 11 in the first embodiment described above. The acquisition unit 170 acquires, from the scoring reference storage unit 140, a first logical relationship that is established between a plurality of first events that constitute an exemplary answer associated with an answer for which the scoring unit 180 scores. As described above, a score is associated with the first logical relationship. The acquisition unit 170 also acquires a first event associated with a score. The acquisition unit 170 supplies the acquired first event and first logical relationship to the scoring unit 180. At this time, it is preferable that the acquisition unit 170 acquires the first event and the first logical relationship in which the score is not 0. Thereby, the man-hour of the comparison process by the scoring part 180 mentioned later can be reduced.

The scoring unit 180 is an example of the evaluation unit 13 in the above-described first embodiment. The scoring unit 180 receives the second event and the second logical relationship from the second extraction unit 160. In addition, the scoring unit 180 receives the first event and the first logical relationship from the acquisition unit 170. Then, the scoring unit 180 compares the first logical relationship with the second logical relationship. The scoring unit 180 compares the first event with the second event. Here, the comparison method between the first logical relationship and the second logical relationship and the comparison method between the first event and the second event are not particularly limited, and any method may be adopted.

The scoring unit 180 gives a score associated with the first logical relationship as a score for the second logical relationship when the first logical relationship and the second logical relationship are the same as a result of the comparison. In addition, the scoring unit 180 assigns a score associated with the first event as a score for the second event when they are the same as a result of the comparison between the first event and the second event. Then, the scoring unit 180 takes the total of the points given to the second event and the second logical relationship as the score for the answer. In this way, the scoring unit 180 scores the answer.

FIG. 5 is a flowchart showing an example of a flow of processing for storing the scoring standard in the scoring standard storage unit 140 in the sentence evaluation apparatus 100 according to the present embodiment. As shown in FIG. 5, first, the model answer receiving unit 110 receives a model answer (step S51). Then, the first extraction unit 120 converts the model answer into a logical expression (step S52). For example, when the natural sentence that is the model answer is “war has occurred. Therefore, financial difficulties have occurred”, the first extraction unit 120 refers to the model answer as “war” and “financial difficulties”. The event is converted into a logical expression having the first element (first event). In this case, logical expression information schematically representing the converted logical expression is “(war) → (financial difficulties)”. This arrow (→) represents a causal relationship between events. The first logical relationship is a relationship that holds between such first elements (causal relationship between events in this example). When converting the model answer into a logical expression, the first extraction unit 120 determines, for example, a conjunction as a first logical relationship between elements before and after the conjunction, and generates a logical expression including the first logical relationship. The first logical relationship is not limited to the causal relationship between the first elements, and may be a simultaneous relationship between the first elements, for example. The simultaneous relationship represents that the first elements (events) occur simultaneously. For example, when the natural sentence that is the model answer is “Taxed for ordinary people, but nobles were not taxed”, the first extraction unit 120 calculates “Taxed for ordinary people” and “Taxed for nobles”. The first element is determined, and the first element is determined to be an event that has occurred simultaneously. Thereby, the first extraction unit 120 converts the natural sentence into a logical expression having the simultaneous relationship as the first logical relationship. In this case, the logical expression information of the converted logical expression is “(taxed for ordinary people) − (tax free for aristocrats)” using “-” representing the simultaneous relationship. The first extraction unit 120 stores the first element and the first logical relationship included in the converted logical expression in the scoring criterion storage unit 140.

Thereafter, the weight receiving unit 130 receives the first logical relationship included in the logical expression converted by the first extracting unit 120 and the input of the points for the first element (step S53). Thereby, by converting into a logical expression, a score can be given not only to each first element extracted from the model answer but also to the first logical relationship. That is, a score can be given not only to the first element intended by the evaluator but also to the connection between the first elements (first logical relationship). Then, the weight receiving unit 130 associates the score with the first element or the first logical relationship stored in the scoring criterion storage unit 140 related to the received score, and saves it as a scoring criterion (step S54). As a result, the scoring criterion storage unit 140 stores the first element and the first logical relationship associated with the scoring.

The scoring standard stored in the scoring standard storage unit 140 by this process will be described with reference to FIG. FIG. 6 is a diagram for explaining the scoring standard 60 stored in the scoring standard storage unit 140. 6A shows the scoring standard 60 using logical expression information, and FIG. 6B shows the scoring standard 60 expressed in a table format.

When the model answer received by the model answer receiving unit 110 is “War has occurred. Therefore, financial difficulties have occurred. Therefore, a tax increase has been performed.” It is converted into a logical expression expressed by logical expression information “(war) → (financial difficulties) → (tax increase)”. As described above, when the model answer includes a plurality of first logical relationships, the first extraction unit 120 extracts the plurality of first logic relationships from the model answer. Here, it is assumed that the evaluator has the following viewpoints (1) to (3) for evaluating the answer among the model answers.
(1) A causal relationship that causes the first element (war) and results from the first element (financial difficulties) (first logical relationship 64)
(2) Cause-and-effect relationship that causes the first factor (financial difficulties) and results from the first factor (tax increase) (first logical relationship 65)
(3) First element (tax increase) (first element 63)
The weight receiving unit 130 receives weights for the above (1) to (3). At this time, the evaluator may input the information indicating the above (1) to (3) and the scoring as text, or assign the scoring to the information (for example, text) expressing (1) to (3). You may enter. The weight receiving unit 130 only has to receive a score inputted by the evaluator by an arbitrary method. Assume that the points received by the weight receiving unit 130 are 10 points for (1) and 5 points for (2) and (3). As a result, as shown in FIGS. 6A and 6B, (1) the causal relationship that is caused by the first element (war) and the first element (financial difficulty) is “( 10 points in the arrow part (first logical relationship 64) including the information of the two first elements of “war” → (financial difficulties), (2) the first element (tax increase) ), 5 points in the part of the arrow (first logical relation 65) including the information of the two first elements “(financial difficulties) → (tax increase)”, (3) the first element Five points are associated with the (tax increase) portion (first element 63). Further, as shown in FIG. 6B, 0 points are associated with the first element 61 and the first element 62 that have not received the input of the scoring points. The 0 point score indicates that the first element or the first logical relationship with which the score is associated is not an object of evaluation.

Note that, as shown in FIG. 6A, the first logical relationship 64 includes an arrow including information on the starting point (war) and the ending point (financial difficulty) of the arrow in “(war) → (financial difficulty)”. Information expressed in parts. Similarly, the first logical relationship 65 is information represented by an arrow portion including information on the starting point (financial difficulty) and the ending point (tax increase) of the arrow. In FIG. 6B, a logical relationship (in this case, a causal relationship) including information on the starting point and the ending point of the arrow is expressed using an arrow (⇒) in order to distinguish it from the logical expression information. In other words, “(war) → (financial difficulties)” is logical expression information, and “(war) → (financial difficulties)” is caused by the first element (war) and the first element (financial trouble) is the result. The first logical relationship 64 is a causal relationship.

Note that the scoring standard 60 does not have to be in the form shown in FIG. 6 and may be any one in which the first element or the first logical relationship is associated with the scoring. Further, the scoring standard 60 may include information (question number or the like) indicating a problem associated with the answer in addition to the information shown in FIG.

Note that the model answer may be a logical expression. In this case, the first extraction unit 120 may store the model answer received from the model answer receiving unit 110 in the scoring criterion storage unit 140.

FIG. 7 is a flowchart showing an example of the answer scoring process in the sentence evaluation apparatus 100 according to the present embodiment. As shown in FIG. 7, the answer receiving unit 150 receives an answer to be graded (step S71). Then, the second extraction unit 160 converts the answer into a logical expression (step S72). Thereby, the second extraction unit 160 can extract a plurality of second elements and one or more second logical relationships from the answer.

Also, the acquisition unit 170 acquires the first element and the first logical relationship associated with the scoring included in the scoring criteria associated with the answer to be scored from the scoring criterion storage unit 140 (step S73). When it is known in advance which scoring standard is the scoring standard associated with the answer received by the answer receiving unit 150, step S73 may be performed simultaneously with step S71 or step S72, or step S71 or step It may be performed before S72. When the answer is, for example, an essay, it is common that a plurality of second logical relationships are included in the answer. Therefore, in the present embodiment, description will be made assuming that acquisition unit 170 acquires a plurality of second elements and a plurality of second logical relationships associated with points other than 0 points.

Next, the scoring unit 180 scores the answer to be scored by comparing the first logical relationship with the second logical relationship and comparing the first element with the second element (step S74). Specifically, the scoring unit 180 determines whether one or more first logical relationships received from the acquisition unit 170 are included in the second logical relationship included in the answer. In addition, the scoring unit 180 determines whether or not one or more first elements received from the acquisition unit 170 are included in the second element included in the answer.

Then, when the second logical relationship that matches the first logical relationship is included in the answer, the scoring unit 180 gives a score associated with the first logical relationship to the second logical relationship. In addition, when the second element that matches the first element is included in the answer, the scoring unit 180 assigns a score associated with the first element to the second element. The scoring unit 180 takes the sum of the points given to the second element and the second logical relationship as the score for the answer. In this way, the scoring unit 180 scores the answer.

The scoring process of the scoring unit 180 will be further described with a specific example. Assume that the answer described below is an answer to the problem associated with the model answer shown in FIG. FIG. 8 is a diagram for explaining the scoring process performed by the scoring unit 180. As shown in FIG. 8, the answer 80 is “A tax increase caused a war. This war caused financial difficulties.” The logical expression information of the logical expression converted by the second extraction unit 160 is “(tax increase) → (war) → (financial difficulties)” as shown in FIG.

Therefore, the second extraction unit 160 can extract the following (A) to (E) from the answer.
(A) Second element (tax increase) (second element 81)
(B) Second element (war) (second element 82)
(C) Second element (financial difficulties) (second element 83)
(D) Causal relationship that causes the second factor (tax increase) and results from the second factor (war) (second logical relationship 84)
(E) Cause-and-effect relationship that causes the second element (war) and results from the second element (financial difficulties) (second logical relationship 85)
The acquisition unit 170 acquires a first element and a first logical relationship with which a score other than 0 is associated. As shown in FIG. 6, the first element associated with a score other than 0 is the first element 63, and the first logical relationship associated with the score other than 0 is the first logical relationship 64 and A first logical relationship 65. Therefore, the acquisition unit 170 acquires the following (1) to (3) from the scoring reference storage unit 140.
(1) First element (tax increase) (first element 63) and scoring (5 points)
(2) Causal relationship (first logical relationship 64) resulting from the first element (war) and 10 points (3) cause of the first element (financial difficulty) Causal relationship (first logical relationship 65) resulting in 1 element (tax increase) and 5 points of scoring And the scoring unit 180 has a second element that matches the first element 63 “(tax increase)” as the second element It is determined whether or not the plurality of second elements extracted by the extraction unit 160 are included. As described above, since “(tax increase)” is the second element 81, the scoring unit 180 determines that the second element that matches the first element 63 is included in the answer. Then, the scoring unit 180 gives five points (partial points 86) associated with the first element 63 to the second element 81 that coincides with the first element 63.

Similarly, the scoring unit 180 determines whether or not the “(war) → (financial difficulty)” that is the first logical relationship 64 is included in the second logical relationship extracted by the second extraction unit 160. Then, the scoring unit 180 assigns 10 points (partial points 87) associated with the first logical relationship 64 to the second logical relationship 85 that matches the first logical relationship 64.

In addition, the scoring unit 180 determines whether or not “(financial difficulty) → (tax increase)”, which is the first logical relationship 65, is included in the second logical relationship extracted by the second extraction unit 160. As shown in FIG. 8, since the answer does not include the second logical relation that matches the first logical relation 65, the score associated with the first logical relation 65 is not given to the answer.

In this way, the scoring unit 180 performs the scoring process for the answer. Note that the scoring unit 180 may output a scoring result. The scoring result output by the scoring unit 180 may be a total score of the partial points, or information that associates the partial points with the second logical relationship and / or the second element to which the partial points are assigned. Also good. The scoring result output method may be any method. For example, the scoring result may be output to the output device 30 described in the first embodiment, or may be output to another device via a network. Good.

Further, the result obtained by scoring by the scoring unit 180 may not be a score. For example, the scoring unit 180 may use information indicating the rank of the answer (for example, excellent, good, acceptable) as the scoring result. In this case, the score (weight) may be information (for example, the number of points) used to represent the rank. In addition, the scoring unit 180 may use pass / fail as a scoring result. In this case, the scoring unit 180 may determine pass / fail by comparing the obtained score with a predetermined threshold value.

In the above description, the scoring unit 180 determines that the first logical relationship and the second logical relationship match. However, the “match” in this determination may not be a complete match. The degree of “match” may be changed by a method of comparing the first logical relationship and the second logical relationship. The same applies to the comparison between the first element and the second element. In the comparison between the first element and the second element, for example, if the concept represented by the first element and the concept represented by the second element are the same, it may be determined that they match.

Also, for example, in the case where the logical relationship is a simultaneous relationship indicating that an event representing an element has occurred at the same time, the scoring unit 180 can determine that there is a match even if the front and back of the element are different. For example, in the case of the model answer “Taxed for ordinary people, but nobles were not taxed”, the logic of “(Taxed for ordinary people)-(Taxable for nobles)” using “-” representing the simultaneous relationship. A logical expression expressed by expression information is obtained. In this case, even if the logical expression information of the logical expression obtained from the answer is not “(taxable to ordinary people) – (tax free to nobility)” but also “(tax free to nobility) – (taxable to ordinary people)” The unit 180 can determine that the second logical relationship and the first logical relationship match.

In the present embodiment, an example has been described in which the processing for storing scoring criteria and the processing for receiving and scoring answers are performed by the same device, but may be performed by different devices. Good. That is, the model answer receiving unit 110, the first extracting unit 120, the weight receiving unit 130, and the scoring reference storage unit 140 are devices different from the sentence evaluation device 100 (for example, the evaluation reference storage device 20 shown in FIG. 2). It may be realized. In this case, the acquisition unit 170 may acquire the first logical relationship from this other device. The model answer receiving unit 110, the first extracting unit 120, and the weight receiving unit 130 may be realized by an apparatus different from the sentence evaluation apparatus 100.

In addition, the first extraction unit 120 and the second extraction unit 160 may be realized by one block. Even in this case, the method used when converting the model answer and the answer into a logical expression may be the same or different.

As described above, the sentence evaluation apparatus 100 according to the present embodiment is similar to the sentence evaluation apparatus 10 according to the first embodiment described above, and the second logical relationship and the first logic established between the second elements. Grade answers by comparing relationships. When the second logical relationship established between the second elements is different from the first logical relationship even if the second element that is the element included in the answer to be scored is the same as the first element, It can be determined that the second logical relationship is different from the first logical relationship. Therefore, the sentence evaluation apparatus 100 can correctly evaluate the correctness of the second logical relationship that is the logical relationship between the second elements constituting the answer. That is, the sentence evaluation apparatus 100 can evaluate the logical relationship between elements included in the answer to be scored.

In addition, according to the sentence evaluation apparatus 100 according to the present embodiment, the scoring unit 180 scores the answer to be scored according to the first logical relationship and / or the weight associated with the first element. In other words, the scoring unit 180 scores according to the weight associated with the first logical relationship that matches the second logical relationship and the weight associated with the first element that matches the second element, included in the answer. Mark the target answer. As a result, the weight associated with the first logical relationship is assigned to the second logical relationship that matches the first logical relationship, and the weight associated with the first element is assigned to the second element that matches the first element. can do.

For example, when the evaluator intends all the first logical relations included in the model answer as the evaluation target, a score may be associated with all the first logical relations. Further, when the evaluator intends to add only a specific first logical relationship among the first logical relationships included in the model answer, the score may be associated with only the specific first logical relationship. Thereby, the text evaluation apparatus 100 can further perform evaluation for each location that the evaluator intends to evaluate.

In addition, when the scoring unit 180 evaluates not only the second logical relationship included in the answer but also the second element, the sentence evaluation apparatus 100 can also evaluate the sufficiency of the second element included in the answer. it can.

Also, the acquisition unit 170 may acquire a plurality of first logical relationships, and the second extraction unit 160 may acquire a plurality of second logical relationships. Then, the scoring unit 180 may compare each of the plurality of second logical relationships with each of the plurality of first logical relationships. As a result, the scoring unit 180 can assign a partial point for each second logical relationship based on the comparison result with the first logical relationship in each of the plurality of second logical relationships. In addition, the scoring unit 180 can give partial points for each second element by comparing each of the plurality of second elements with each of the plurality of first elements. Therefore, even if the answer includes a plurality of second logical relationships, the scoring unit 180 uses the second logical relationship and the partial points assigned to the second element to accurately score the answer to be scored. can do. Furthermore, by assigning partial points to the second logical relationship and the second element included in the answer, it is possible to utilize the scoring result as more detailed feedback for utilizing the ability of the answer creator.

(Modification 1)
In the second embodiment described above, the method of adding partial points has been described as the method of scoring by the scoring unit 180. However, the scoring method reduces points when the second logical relationship included in the answer is incorrect. It may be a method. The configuration of the sentence evaluation apparatus 100 in this modification is the same as that of the sentence evaluation apparatus 100 shown in FIG.

In the case of a method of deducting when the scoring method is wrong, the weight received by the weight receiving unit 130 is information that can be deducted. For example, the weight received by the weight receiving unit 130 is a negative score. An example of the scoring standard 90 stored in the scoring standard storage unit 140 at this time will be described with reference to FIG. The number of the first logical relationship and the number of the first element included in the scoring standard 90 correspond to the numbers shown in FIG.

9 The scoring standard 90 shown in FIG. 9 and the scoring standard 60 shown in FIG. For example, as shown in FIG. 9, -5 points are associated with the first element 63 as a score. This negative score indicates that if the first element 63 is not included in the answer, points are deducted.

FIG. 10 is a flowchart showing an example of the answer scoring process in the sentence evaluation apparatus 100 according to this modification. As shown in FIG. 10, the answer receiving unit 150 receives an answer to be graded (step S101). Then, the second extraction unit 160 converts the answer into a logical expression (step S102). Thereby, the second extraction unit 160 can extract a plurality of second elements and one or more second logical relationships from the answer.

Also, the acquisition unit 170 acquires the first element and the first logical relationship associated with the scoring included in the scoring criteria associated with the answer to be scored from the scoring criterion storage unit 140 (step S103). Further, the acquisition unit 170 acquires an initial value from the scoring reference storage unit 140 (step S104). The initial value may be stored in the scoring criterion storage unit 140 when the model answer is input. The initial value may be a predetermined value. When the initial value is a predetermined value, the scoring unit 180 may acquire the initial value instead of the acquiring unit 170.

Then, the scoring unit 180 scores the answer to be scored by comparing the first logical relationship with the second logical relationship and comparing the first element with the second element (step S105). Specifically, the scoring unit 180 determines whether one or more first logical relationships received from the acquisition unit 170 are included in the second logical relationship included in the answer. In addition, the scoring unit 180 determines whether or not one or more first elements received from the acquisition unit 170 are included in the second element included in the answer. As a result of the determination, when the first logical relationship is not included in the second logical relationship included in the answer, the scoring unit 180 holds the score associated with the first logical relationship as a score to be deducted. Further, as a result of the determination, when the first element is not included in the second element included in the answer, the scoring unit 180 holds the score associated with the first element as a score to be deducted. As shown in FIG. 9, since the points to be deducted are negative points, the total number of deducted points held is also a negative point. Therefore, the scoring unit 180 adds the negative score to the initial value, so that the scoring unit 180 can score the answer by a method of deducting from the initial value.

Note that the score in FIG. 9 is a positive value as in FIG. 6, and the scoring unit 180 may be configured to score the answer by a deduction method by setting an initial value. In addition, for each question included in the scoring criteria, information on the scoring method and the subtracting method is associated. In the case of an answer to a question associated with information on the deduction method, the deduction method may be used for scoring.

Also, the symbols of the stipulations other than the 0 point associated with the first logical relationship and the first element may not be the same. For example, +10 points may be associated with the first logical relationship 64 as a scoring point, and −5 points may be associated with the first logical relationship 65 as a scoring point. At this time, for example, the scoring unit 180 refers to the sign of the stipulation associated with the first logical relation, and the second logical relation that matches the first logical relation 64 associated with the constellation with a positive sign is the answer. If included, add points, otherwise do nothing. In addition, the scoring unit 180 deducts if the answer does not include the second logical relationship that matches the first logical relationship 65 associated with the negative sign, and does nothing if it is included. As described above, the scoring unit 180 may change the scoring method and the deduction method according to the sign of the scoring.

Also, both positive and negative points may be associated with the first logical relationship and the first element. For example, +10 points and −5 points may be associated with the first logical relationship 64. In this case, the scoring unit 180 may add +10 points when the second logical relationship that matches the first logical relationship 64 is included in the answer, and may deduct 5 points when it is not included.

Thus, the scoring method of the scoring unit 180 may be a scoring method or a deduction method. Even with such a configuration, the sentence evaluation apparatus 100 according to the present modification is a case where a plurality of second logical relationships are included in the answer, similarly to the sentence evaluation apparatus 100 in the second embodiment described above. Even if it exists, it is possible to accurately score the answer to be scored. Moreover, the sentence evaluation apparatus 100 can perform more detailed evaluation for each part that the evaluator intends to evaluate by scoring using the scoring method and the deduction method.

Also, the scoring unit 180 may output the first logical relationship (the first logical relationship that is the target of deduction) that is not included in the answer together with the deduction number, for example, in a list format. Thereby, the creator of an answer can grasp the grounds for deduction.

(Modification 2)
In the second embodiment and the first modification described above, it is determined whether or not the first logical relationship extracted from the model answer is included in the answer. However, in the case of a problem in the form of describing a natural sentence, the answer may include sentences that are actually correct but are not assumed by the evaluator. Below, the structure which outputs such a sentence as a sentence by which the right / wrong judgment was not performed is demonstrated.

The configuration of the sentence evaluation apparatus 100 in this modification is the same as that of the sentence evaluation apparatus 100 shown in FIG. The acquisition unit 170 acquires all the first logical relationships and the first elements extracted from the model answer to the problem associated with the answer to be scored, together with the associated score. The first logical relationship and the first element acquired at this time include those associated with 0 points.

The scoring unit 180 compares the first logical relationship with the second logical relationship. Specifically, the scoring unit 180 determines whether or not the scoring criterion includes a first logical relationship that matches the second logical relationship. Then, when the first logical relationship that matches the second logical relationship is included in the scoring criteria, the scoring unit 180 assigns a score associated with the first logical relationship to the second logical relationship. In addition, the scoring unit 180 determines whether or not the first element that matches the second element is included in the scoring standard. Then, the scoring unit 180 assigns a score associated with the first element that matches the second element to the second element.

Also, the scoring unit 180 does not assign a score to the second logical relationship that does not match the first logical relationship and the second element that does not match the first element. Here, the first logical relationship and the first element include those associated with 0 points. The second logical relationship and the second element to which a score is assigned indicate not only a positive score but also a 0-point score. That is, the second logical relationship and the second element to which the scoring unit 180 does not assign a score are the second logical relationship or the second element for which the score processing has not been performed. The scoring unit 180 uses the second logical relationship and the second element that do not give such a score as the second logical relationship and the second element that are not included in the scoring criteria, for example, in a manner that the evaluator can grasp. May be output together. The scoring unit 180 may output the second logical relationship and the second element to which no scoring is assigned by storing them in the scoring criterion storage unit 140 or other storage device as another solution candidate group.

Thereby, even when the second logical relationship and the second element not included in the scoring criteria are not included in the model answer, the evaluator determines whether the second logical relationship and the second element are correct. can do.

In addition, when the evaluator determines that the second logical relationship and the second element can constitute a correct answer (another answer) different from the model answer, the sentence evaluation apparatus 100 reads the sentence described by the answer creator. May be newly registered in the scoring criterion storage unit 140 as another model answer, or the registered model may be determined as another model solution. The first logical relationship and the first element included in the answer may be modified. Thereby, the evaluator can grasp the possibility that the second logical relationship and the second element included in the answer are different correct answers.

<Third Embodiment>
Next, a third embodiment of the present disclosure will be described with reference to the drawings. FIG. 11 is a diagram illustrating an example of a functional configuration of the sentence evaluation apparatus 200 according to the present embodiment. Moreover, FIG. 12 is a figure which shows an example of a structure of the text evaluation system 2 containing the text evaluation apparatus 200 which concerns on this Embodiment. In addition, the same code | symbol is attached | subjected to the block which has the same function as the block contained in the text evaluation apparatus 100 which concerns on 2nd Embodiment mentioned above, and the detailed description is abbreviate | omitted.

As shown in FIG. 12, the sentence evaluation system 2 includes a sentence evaluation apparatus 200 and a display apparatus 40. The display device 40 and the sentence evaluation device 200 are communicably connected via a network or not. The display device 40 may be the output device 30 shown in FIG. The display device 40 displays a screen based on the control signal transmitted from the text evaluation device 200. The display device 40 is realized by, for example, a liquid crystal display. The display device 40 may be configured to be incorporated in the text evaluation device 200.

The sentence evaluation apparatus 200 includes a display control unit 210 in addition to the function of the sentence evaluation apparatus 100. As shown in FIG. 11, the sentence evaluation apparatus 200 includes an exemplary answer receiving unit 110, a first extracting unit 120, a weight receiving unit 130, a scoring criterion storage unit 140, an answer receiving unit 150, and a second extracting unit 160. And an acquisition unit 170, a scoring unit 180, and a display control unit 210.

The display control unit 210 creates a graph using the first element and the first logical relationship. Specifically, the first element and the first logical relationship included in the logical expression converted by the first extraction unit 120 are received from the first extraction unit 120. The display control unit 210 generates a graph in which each of the received plurality of first elements is a node, and the first logical relationship established between a certain first element and another first element is an edge. Then, the display control unit 210 causes the display device 40 to display the generated graph. For example, the display control unit 210 generates a control signal for displaying a graph and transmits the control signal to the display device 40 to display the graph on the display device 40.

The evaluator inputs weights for nodes and edges to the graph displayed on the display device 40 via an input device such as a mouse or a keyboard. The weight receiving unit 130 receives the weight input in this way. The weight receiving unit 130 supplies the received weight to the display control unit 210. The display control unit 210 associates the weight for the node with the first element, associates the weight for the edge with the first logical relationship, and stores them in the scoring criterion storage unit 140. The display control unit 210 may display the graph displayed on the display device 40 on the display device 40 in association with the weight input by the evaluator.

FIG. 13 is a flowchart illustrating an example of a flow of processing for storing a scoring criterion in the scoring criterion storage unit 140 in the sentence evaluation device 200 according to the present embodiment. As shown in FIG. 13, first, the model answer receiving unit 110 receives a model answer (step S131). Then, the first extraction unit 120 converts the model answer into a logical expression (step S132). For example, when the natural sentence that is the model answer is “war has occurred. Therefore, financial difficulties have occurred”, the first extraction unit 120 refers to the model answer as “war” and “financial difficulties”. The event is converted into a logical expression having the first element (first event). The logical expression information of this logical expression is “(war) → (financial difficulties)”. The first extraction unit 120 supplies the first element and the first logical relationship included in the converted logical expression to the display control unit 210.

The display control unit 210 creates a graph using the first element and the first logical relationship received from the first extraction unit 120, and displays the created graph on the display device 40 (step S133). Thereby, the display apparatus 40 displays a graph.

FIG. 14 shows an example of the graph 141 displayed on the display device 40. Here, the natural sentence that is the model answer is “War has occurred. Therefore, financial difficulties have occurred.” The first extraction unit 120 extracts the first element (61 to 63) and the first logical relationship (64, 65) shown in FIG. 6 from the model answer. The display control unit 210 sets the first elements (61 to 63) as nodes (142 to 144) and sets the first logical relationships (64 and 65) as edges (145 and 146). A graph 141 as shown is generated and displayed on the display device 40. As shown in FIG. 14, the display control unit 210 displays logical relationship information (147, 148) indicating the logical relationship of the first logical relationship in the vicinity of the edge (145, 146). May be. As described above, the edge 145 represents the first logical relationship 64, and the first logical relationship 64 is a causal relationship in which the first element (war) is the cause and the first element (financial difficulty) is the result. As illustrated in FIG. 14, the display control unit 210 may display “causal relationship” in the vicinity of the edge 145 as the logical relationship information 147 of the edge 145. At this time, the display control unit 210 may hold the logical relationship information representing the first logical relationship in itself and cause the display device 40 to display the held logical relationship information. Further, as shown in FIG. 14, the display control unit 210 may display “causal relationship” in the vicinity of the edge 146 as the logical relationship information 148 of the edge 146.

Referring back to FIG. 13, when the evaluator inputs the weights for the nodes and edges to the graph 141 displayed on the display device 40, the weight receiving unit 130 receives the input weights (step S134).

FIG. 15 shows an example of a screen when the display control unit 210 displays the weight input to the graph 141 shown in FIG. 14 on the display device 40 together with the graph 141. Assume that the evaluator inputs 10 points for the

edge

145, 5 points for the

edge

146, and 5 points for the node 144 as weights. As shown in FIG. 15, the display device 40 displays “10 points” that are points 151 to the edge 145 in association with the edge 145. Similarly, on the display device 40, “5 points” which are the scoring points 152 for the edge 146 are displayed in association with the edge 146. In addition, the display device 40 displays “5 points”, which are the allocation points 153 for the node 144, in association with the node 144.

Thereafter, the display control unit 210 associates the weights for the nodes (142 to 144) with the first elements (61 to 63) and associates the weights for the edges (145 and 156) with the first logical relationship (64, 65). The display control unit 210 includes first elements (61 to 63) associated with weights for the nodes (142 to 144) and first logical relationships (64 and 65) associated with weights for the edges (145 and 156). Is stored in the scoring reference storage unit 140. Thereby, the scoring standard storage unit 140 stores the scoring standard 60 as shown in FIG. 6B (step S135).

The scoring unit 180 scores the answer using the first element and the first logical relationship associated with the weight stored in the scoring criterion storage unit 140 in this way.

As described above, in the present embodiment, the sentence evaluation device 200 generates a graph from the model answer and causes the display device 40 to display the graph. Thus, the evaluator can easily grasp which part of the model answer is the logical relationship intended to be evaluated. Therefore, the evaluator can easily input the weight for the first logical relationship and the first element intended to be evaluated.

Note that the graph generated by the display control unit 210 may be a directed graph having an orientation as shown in FIG. 14 or an undirected graph. In addition, the graph generated by the display control unit 210 may be a mixed graph having characteristics of both a directed graph and an undirected graph. The display control unit 210 may change the generated graph according to the first logical relationship. For example, the display control unit 210 may generate a directed graph indicating that a symmetry rule is established between the first elements.

<Fourth embodiment>
Next, a fourth embodiment of the present disclosure will be described with reference to the drawings. FIG. 16 is a diagram illustrating an example of a functional configuration of the sentence evaluation device 300 according to the present embodiment. In addition, the same code | symbol is attached | subjected to the block which has the same function as the block contained in the text evaluation apparatus 100 which concerns on 2nd Embodiment mentioned above, and the detailed description is abbreviate | omitted.

The text evaluation device 300 includes a second extraction unit 360 instead of the second extraction unit 160 of the text evaluation device 100, a scoring unit 380 instead of the scoring unit 180, and an answer storage unit (second text storage). Part) 310. As shown in FIG. 16, the sentence evaluation apparatus 300 includes an exemplary answer receiving unit 110, a first extracting unit 120, a weight receiving unit 130, a scoring criterion storage unit 140, an answer receiving unit 150, and a second extracting unit 360. And an acquisition unit 170, a scoring unit 380, and an answer storage unit 310. Note that the sentence evaluation device 300 includes a second extraction unit 360 instead of the second extraction unit 160 of the sentence evaluation device 200 described above, a scoring unit 380 instead of the scoring unit 180, and further includes an answer storage unit 310. It may be a configuration.

In this embodiment, it is assumed that the answer accepted by the answer accepting unit 150 is the answer 80 shown in FIG. As shown in FIG. 8, the answer 80 is “The tax increase caused the war. This war caused financial difficulties.” The second extraction unit 360 converts the answer into a logical expression including the second element 81, the second element 82, the second element 83, the second logical relationship 84, and the second logical relationship 85, as shown in FIG. At this time, the second extraction unit 360 extracts each second element together with information indicating a range in an extraction source (answer) in which the second element is included. The information indicating this range may be information that indicates from which position in the answer guidance to which position the second element is included, such as the start address of the second element and its size. For example, in the case of the second element 81 (tax increase), the information indicating the range in the answer including the second element 81 may be position information indicating that there are two characters from the first letter of the answer.

Also, the second extraction unit 360 extracts each second logical relationship together with information indicating the range at the extraction source in which the second logical relationship is included. The information indicating this range may be a range including two second elements related to each second logical relationship. For example, this range may be from the start position of one of the two second elements to the end position of the other. For example, in the case of the second logical relationship 84, this range is a sentence including the second element 81 (tax increase) and the second element 82 (war) included in the second logical relationship 84 “tax increase”. May have caused the war. "

Then, the second extraction unit 360 associates the answer, each second logical relationship or each second element, and information indicating the range in the answer of each second logical relationship or each second element with each other, and the answer storage unit 310 is stored as answer information. FIG. 17 is a diagram illustrating an example of answer information 171 stored in the answer storage unit 310. As shown in FIG. 17, the answer information 171 includes an answer, a second logical relationship or second element, and a second logical relationship or range of the second element in the answer. Note that the answer information 171 shown in FIG. 17 also includes the second logical relationship or the number of the second element shown in FIG. 8 for convenience of explanation. The second logical relationship or the range of the second element in the answer will be described as the start position and its size as shown in FIG. The expression method of the start position and its size is not particularly limited, and may be the number of bits or the number of characters. The information included in the answer information 171 is not limited to this.

When a plurality of independent logical expressions are generated from the answer, the answer information 171 includes, for each sentence related to the independent logical expression, among the answers, the second logical relationship or the second element, and the second in the answer. Two logical relationships or a range of the second element may be associated with each other. For example, a logical expression A, a logical expression B, and a logical expression C that are independent from each other are generated, and natural sentence portions (sentences) used in the generation are a sentence a, a sentence b, and a sentence c, respectively. And In this case, the second extraction unit 360 may store the second logical relationship and the second element included in the logical expression A in the answer storage unit 310 in association with the sentence a. Similarly, the second extraction unit 360 stores the second logical relationship included in the logical expression B and the second element in the answer storage unit 310 in association with the sentence b, and the second logical relationship included in the logical expression C. The second element may be stored in the answer storage unit 310 in association with the sentence c.

Note that the answer storage unit 310 may be realized by the same storage device as the scoring reference storage unit 140. The answer storage unit 310 may be realized by a storage device different from the sentence evaluation device 300.

The scoring unit 380 compares the first logical relationship and the second logical relationship in the same manner as the scoring unit 180. The scoring unit 380 compares the first element with the second element. If the result of comparison between the first logical relationship and the second logical relationship is that they are the same logical relationship, the scoring unit 380 assigns a score associated with the first logical relationship as a score for the second logical relationship. In addition, as a result of the comparison between the first element and the second element, the scoring unit 380 assigns a score associated with the first element as a score for the second element when they are the same element.

In addition, the scoring unit 380 refers to the answer storage unit 310 and associates the second logical relation or the second element related to the answer to be scored with the second logical relation and the second element to which a scoring is given. The obtained second logical relationship or the range of the second element in the answer is acquired together with the answer. Then, the scoring unit 380 sets a result obtained by associating the acquired answer, the range, and the second logical relationship and the second element to which the scoring is given, as a scoring result. Note that the scoring unit 380 may output the scoring result to the output device 30 described above. When the output device 30 is a display device, the scoring unit 380 may generate and output a screen to be displayed on the output device 30. An example of the scoring result output by the scoring unit 380 will be described later.

FIG. 18 is a flowchart illustrating an example of the flow of answer scoring processing in the sentence evaluation apparatus 300 according to the present embodiment. In the present embodiment, description will be made assuming that the scoring unit 380 includes processing for generating a screen to be displayed on the output device 30.

As shown in FIG. 18, the answer receiving unit 150 receives an answer to be graded (step S181). The second extraction unit 360 converts the answer into a logical expression (step S182). In addition, the second extraction unit 360 stores the answer information 171 described above in the answer storage unit 310 (step S183).

Also, the acquisition unit 170 acquires from the scoring criterion storage unit 140 the first element and the first logical relationship associated with the scoring included in the scoring criterion associated with the answer to be scored (step S184).

Next, the scoring unit 380 scores the answer to be scored by comparing the first logical relationship with the second logical relationship and comparing the first element with the second element (step S185). Then, the scoring unit 380 outputs the range representing the second logical relationship corresponding to the first logical relationship and the range representing the second element corresponding to the first element among the answers as a scoring result together with the answer (step S186). . Specifically, the scoring unit 380 obtains the range of the second logical relationship to which the score in the answer is given together with the related answer as the range of the second logical relationship corresponding to the first logical relationship. Further, the scoring unit 380 acquires the range of the second element to which the score in the answer is assigned as the range of the second element corresponding to the first element. Then, the scoring unit 380 encloses a portion of the acquired range in the answer, for example, and generates and outputs a screen on which the second logical relationship included in the range and the score assigned to the second element are displayed. An example of the scoring result 191 output by the scoring unit 380 is shown in FIG. As with the scoring results shown in FIG. 8, the second element (tax increase) is 5 points, the second element (war) is the cause, and the second element (financial difficulty) is the result, and the second logical relationship is 10 points. Is given.

Here, the scope of the answer representing the second logical relationship that is caused by the second element (war) and the second element (financial difficulty) is the part of “It became a financial difficulty in this war.” To do. As shown in FIG. 19, the scoring section 380 is surrounded by a frame of “Tax increased” with 10 points and “Tax increase” with 5 points. Note that the scoring unit 380 may represent this part in a manner different from the others, and is not limited to the frame enclosure. Further, in the vicinity of the portion surrounded by the frame, a score assigned to the corresponding second element or second logical relationship is displayed. As shown in FIG. 19, the scoring unit 380 may include the total score in the scoring result 191. Thereby, the creator of an answer can grasp | ascertain what part of the answer got what evaluation.

As described above, the answer storage unit 310 in the present embodiment stores the answer, the extracted second logical relationship, and the range representing the second logical relationship in the answer in association with each other. The scoring unit 380 outputs a range representing the second logical relationship corresponding to the first logical relationship among the answers together with the answer. Thereby, the creator of the answer can know which part of the answer written by himself / herself has been evaluated. As a result, the scoring result can be used as more detailed feedback for utilizing the ability of the answer creator to improve the ability. Furthermore, since the creator of the answer can check the evaluation of the answer written by himself / herself, the creator of the answer can use the scoring result for an objection or the like.

<Fifth embodiment>
In the present embodiment, a case will be described in which the evaluator further gives a condition to the first logical relationship and / or the first element included in the model answer. Depending on the content of the question that is presented, the evaluator may want to include a certain first element or a certain first logical relationship in the answer. In such a case, the evaluator may increase the evaluation for the answer including the first logical relationship or the first element that is necessarily included, and may evaluate the evaluation for the answer that is not so lower.

In the present embodiment, a sentence evaluation apparatus 400 that can appropriately evaluate an answer even in such a case will be described. FIG. 20 is a functional block diagram illustrating an example of a functional configuration of the sentence evaluation device 400 according to the present embodiment. In addition, the same code | symbol is attached | subjected to the block which has the same function as the block contained in the text evaluation apparatus 100 which concerns on 2nd Embodiment mentioned above, and the detailed description is abbreviate | omitted. As shown in FIG. 20, the sentence evaluation apparatus 400 includes an exemplary answer receiving unit 110, a first extracting unit 120, a weight receiving unit 430, a scoring criterion storage unit 440, an answer receiving unit 150, and a second extracting unit 160. And an acquisition unit 470 and a scoring unit 480. The sentence evaluation device 400 includes a weight reception unit 430 instead of the weight reception unit 130 of the sentence evaluation device 100, includes a scoring reference storage unit 440 instead of the scoring reference storage unit 140, and acquires an acquisition unit 470 instead of the acquisition unit 170. The scoring unit 480 is provided instead of the scoring unit 180. The sentence evaluation device 400 includes a weight reception unit 430 instead of the weight reception unit 130 of the sentence evaluation device 200, includes a scoring reference storage unit 440 instead of the scoring reference storage unit 140, and an acquisition unit instead of the acquisition unit 170. 470 may be provided, and the scoring unit 480 may be provided instead of the scoring unit 180. The text evaluation device 400 includes a weight reception unit 430 instead of the weight reception unit 130 of the text evaluation device 300, includes a scoring reference storage unit 440 instead of the scoring reference storage unit 140, and an acquisition unit instead of the acquisition unit 170. 470 may be provided, and the scoring unit 480 may be provided instead of the scoring unit 180.

The weight receiving unit 430 receives information representing a predetermined condition for the first logical relationship and / or the first element in addition to the function of the weight receiving unit 130 described above. For example, the weight receiving unit 430 receives information indicating an essential condition. Then, the weight receiving unit 430 stores information indicating the received essential condition in the scoring criterion storage unit 440 in association with the related first logical relationship and / or the first element.

The scoring standard storage unit 440 stores scoring standards. FIG. 21 is a diagram showing an example of the scoring standard 66 stored in the scoring standard storage unit 440 in the present embodiment. In FIG. 21, the number of the first logical relationship and the number of the first element are the same as the numbers shown in FIG. The difference between the scoring standard 60 and the scoring standard 66 is that a string of flags representing essential conditions is added. This flag is expressed by “1” indicating that it is an essential condition and “0” indicating that it is not an essential condition, but the flag is not limited to these expressions. Note that the method for expressing the essential conditions is not limited to this. As shown in FIG. 21, a flag having a value “1” is associated with the first logical relationship 64. The weight receiving unit 430 associates a flag having a value of “0” in FIG. 21 with respect to the first logical relationship and the first element that are not accepted as an essential condition.

The acquisition unit 470 acquires the first element and the first logical relationship in which the score is not 0 or the flag is “1”. At this time, the acquiring unit 470 acquires the score and the flag value in association with each of the first element and the first logical relationship. The acquisition unit 470 supplies the acquired first element and first logical relationship to the scoring unit 480.

The scoring unit 480 scores the answer by comparing the first logical relationship and the second logical relationship in the same manner as the scoring unit 180. At this time, the scoring unit 480 compares the first logical relationship with the flag “1” with the second logical relationship. The scoring unit 480 compares the first element with the flag “1” and the second element. As shown in FIG. 21, the first logical relationship whose flag is “1” is only the first logical relationship 64. Therefore, the scoring unit 480 compares the first logical relationship 64 and the second logical relationship, and if these represent the same logical relationship, assigns “10 points” that are points associated with the first logical relationship 64. .

When the answer accepted by the answer accepting unit 150 is the answer shown in FIG. 8, the first logical relation 64 and the second logical relation 85 indicate the same logical relation. 2. “10 points” is assigned to the logical relationship 85.

If the answer received by the answer accepting unit 150 does not include the second logical relationship having the same logical relationship as the first logical relationship 64, the scoring unit 480 indicates that the answer does not include an indispensable condition. It may be determined and information indicating that may be output as a scoring result. For example, if the scoring unit 480 determines that the essential condition is not included in the answer, the scoring unit 480 compares the other first logical relationship with the second logical relationship and compares the first element with the second element. A lower evaluation score (for example, 0 points) may be output as a scoring result without performing it. In addition, the scoring unit 480 may output a value obtained by subtracting a predetermined score so as to obtain a lower evaluation score from the total score of the scoring, or a predetermined ratio so as to obtain a lower evaluation score. A value obtained by multiplying may be output as a scoring result.

Further, the weight receiving unit 430 may receive, for example, a priority order as the predetermined condition. Then, the scoring unit 480 compares the first logical relationship or the first element associated with the priority order with the second logical relationship or the second element in descending order of priority. As a result of the comparison, if the answer does not include the second logical relationship having the same logical relationship as the first logical relationship or the second element that is the same element as the first element, the scoring unit 480 May output a lower rating as the scoring result.

Thus, the first logical relationship and the first element may be further associated with a predetermined condition. The scoring unit 480 compares the first logical relationship and the first element associated with the predetermined condition with the second logical relationship and the second element, respectively, and determines the answer to be scored according to the result of the comparison. to grade. Thereby, the text evaluation apparatus 400 can perform scoring that reflects the evaluation intended by the evaluator. For example, when the essential condition is not included and the scoring unit 480 is configured not to perform processing for the other first logical relationship and the other first element, the sentence evaluation apparatus 400 takes time for evaluation. Cost can be reduced.

Note that the scoring unit 480 may output the scoring result as information indicating pass / fail instead of the score, similarly to the scoring unit 180 described above. For example, when the essential condition is not included in the answer, the scoring unit 480 may output a scoring result “fail”. As a result, the sentence evaluation apparatus 400 does not indicate how much the answer is satisfied with the first logical relationship and the first element intended by the evaluator, but the contents that the evaluator considers to be basic or important in the answer. Whether it is included or not can be determined.

(About hardware configuration)
In each embodiment of the present invention, each component of each device represents a functional unit block. Part or all of each component of each device is realized by an arbitrary combination of an information processing device 900 and a program as shown in FIG. 22, for example. FIG. 22 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 900 that realizes each component of each apparatus. The information processing apparatus 900 includes the following configuration as an example.

CPU (Central Processing Unit) 901
ROM (Read Only Memory) 902
-RAM (Random Access Memory) 903
A program 904 loaded into the RAM 903
A storage device 905 that stores the program 904
A drive device 907 that reads / writes data from / to the recording medium 906
A communication interface 908 connected to the communication network 909
An input / output interface 910 for inputting / outputting data
-Bus 911 connecting each component
Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. The program 904 that realizes the function of each component of each device is stored in advance in the storage device 905 or the RAM 903, for example, and is read by the CPU 901 as necessary. The program 904 may be supplied to the CPU 901 via the communication network 909, or may be stored in the recording medium 906 in advance, and the drive device 907 may read the program and supply it to the CPU 901.

There are various modifications to the method of realizing each device. For example, each device may be realized by an arbitrary combination of an information processing device 900 and a program that are different for each component. A plurality of constituent elements included in each device may be realized by an arbitrary combination of one information processing device 900 and a program.

Also, some or all of the constituent elements of each device are realized by other general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus.

Some or all of the components of each device may be realized by a combination of the above-described circuit and the like and a program.

When some or all of the constituent elements of each device are realized by a plurality of information processing devices and circuits, the plurality of information processing devices and circuits may be centrally arranged or distributedly arranged. Also good. For example, the information processing apparatus, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system and a cloud computing system.

Note that each of the above-described embodiments is a preferred embodiment of the present disclosure, and the scope of the present disclosure is not limited only to the above-described embodiments, and those skilled in the art do not depart from the gist of the present disclosure. However, it is possible to construct a form in which various modifications are made by correcting or substituting the above-described embodiments.

Some or all of the above embodiments can be described as in the following supplementary notes, but are not limited thereto.

(Appendix 1)
An acquisition means for acquiring a first logical relationship established between first events constituting the first sentence;
Logical relationship extraction means for extracting a second logical relationship that is established between the second events constituting the second sentence from the second sentence that is the object of evaluation;
An evaluation means for evaluating the second sentence by comparing the first logical relationship with the second logical relationship;
With
The sentence evaluation apparatus according to claim 1, wherein the first sentence is a sentence serving as a reference for evaluation of the second sentence.

(Appendix 2)
A weight is associated with the first logical relationship,
The evaluation means evaluates the second sentence according to the weight;
The sentence evaluation apparatus according to Supplementary Note 1, wherein

(Appendix 3)
The first sentence is a model answer;
The second sentence is a graded answer;
The evaluation means scores the answer according to the weight by comparing the first event and the second event.
The sentence evaluation apparatus according to supplementary note 2, characterized by:

(Appendix 4)
The first event is associated with the weight;
The evaluation means evaluates the second sentence according to the weight by comparing the first event and the second event;
The sentence evaluation apparatus according to Supplementary Note 2 or 3, characterized in that:

(Appendix 5)
The acquisition means acquires a plurality of the first logical relationships,
The logical relationship extraction means extracts a plurality of the second logical relationships from the second sentence,
The evaluation means evaluates the second sentence by assigning a partial point for each of the second logical relationship and for each of the second events based on the weight.
The sentence evaluation apparatus according to supplementary note 4, characterized by:

(Appendix 6)
Display control means for causing a display device to display a graph having the first event as a node and the first logical relationship established between a certain first event and another first event as an edge;
Weight accepting means for accepting input of the weight for the node and the weight for the edge;
Storage means for storing the first logical relationship associated with a weight for the edge and the first event associated with a weight for the node;
The sentence evaluation apparatus according to appendix 4 or 5, further comprising:

(Appendix 7)
Information representing a predetermined condition is further associated with at least one of the first logical relationship and the first event,
The evaluation means compares the first logical relationship and the first event associated with the predetermined condition with the second logical relationship and the second event, respectively, and according to a result of the comparison, Evaluate 2 sentences,
The sentence evaluation apparatus according to any one of supplementary notes 4 to 6, characterized in that:

(Appendix 8)
The first event and the second event are expressed by keywords or key phrases,
The sentence evaluation apparatus according to any one of supplementary notes 1 to 7, characterized in that:

(Appendix 9)
A second sentence storage means for associating and storing the second sentence, the extracted second logical relation, and a range representing the second logical relation in the second sentence;
The evaluation means outputs a range representing the second logical relationship corresponding to the first logical relationship in the second sentence together with the second sentence.
The sentence evaluation apparatus according to any one of supplementary notes 1 to 8, characterized in that:

(Appendix 10)
Obtaining a first logical relationship that is established between first events constituting a first sentence that is a sentence serving as a reference for evaluation;
Extracting a second logical relationship that holds between second events constituting the second sentence from the second sentence to be evaluated;
Evaluating the second sentence by comparing the first logical relationship with the second logical relationship;
A sentence evaluation method characterized by that.

(Appendix 11)
A weight is associated with the first logical relationship,
The second sentence is evaluated according to the weight;
The sentence evaluation method according to supplementary note 10, characterized by:

(Appendix 12)
A process of acquiring a first logical relationship established between first events constituting a first sentence that is a sentence serving as a reference for evaluation;
A process of extracting a second logical relationship established between second events constituting the second sentence from the second sentence to be evaluated;
A process of evaluating the second sentence by comparing the first logical relationship and the second logical relationship;
A computer-readable non-transitory recording medium that records a program that causes a computer to execute the program.

(Appendix 13)
A weight is associated with the first logical relationship,
The process of evaluating the second sentence evaluates the second sentence according to the weight.
The recording medium according to appendix 12, characterized in that:

DESCRIPTION OF SYMBOLS 1 Text evaluation system 2 Text evaluation system 10 Text evaluation apparatus 11 Acquisition part 12 Logical relationship extraction part 13 Evaluation part 20 Evaluation criteria memory | storage device 30 Output device 40 Display apparatus 100 Text evaluation apparatus 110 Model answer reception part 120 1st extraction part 130 Weight Reception unit 140 Grading standard storage unit 150 Answer receiving unit 160 Second extraction unit 170 Acquisition unit 180 Grading unit 200 Sentence evaluation device 210 Display control unit 300 Sentence evaluation device 310 Answer storage unit 360 Second extraction unit 380 Scoring unit 400 Sentence evaluation device 430 Weight receiving unit 440 Scoring reference storage unit 470 Acquisition unit 480 Scoring unit

Claims

An acquisition means for acquiring a first logical relationship established between first events constituting the first sentence;
Logical relationship extraction means for extracting a second logical relationship that is established between the second events constituting the second sentence from the second sentence that is the object of evaluation;
An evaluation means for evaluating the second sentence by comparing the first logical relationship with the second logical relationship;
With
The sentence evaluation apparatus according to claim 1, wherein the first sentence is a sentence serving as a reference for evaluation of the second sentence.
A weight is associated with the first logical relationship,
The evaluation means evaluates the second sentence according to the weight;
The sentence evaluation apparatus according to claim 1.
The first sentence is a model answer;
The second sentence is a graded answer;
The evaluation means scores the answer according to the weight by comparing the first event and the second event.
The sentence evaluation apparatus according to claim 2, wherein:
The first event is associated with the weight;
The evaluation means evaluates the second sentence according to the weight by comparing the first event and the second event;
The sentence evaluation apparatus according to claim 2 or 3, wherein
The acquisition means acquires a plurality of the first logical relationships,
The logical relationship extraction means extracts a plurality of the second logical relationships from the second sentence,
The evaluation means evaluates the second sentence by assigning a partial point for each of the second logical relationship and for each of the second events based on the weight.
The sentence evaluation apparatus according to claim 4.
Display control means for causing a display device to display a graph having the first event as a node and the first logical relationship established between a certain first event and another first event as an edge;
Weight accepting means for accepting input of the weight for the node and the weight for the edge;
Storage means for storing the first logical relationship associated with a weight for the edge and the first event associated with a weight for the node;
The sentence evaluation apparatus according to claim 4, further comprising:
Information representing a predetermined condition is further associated with at least one of the first logical relationship and the first event,
The evaluation means compares the first logical relationship and the first event associated with the predetermined condition with the second logical relationship and the second event, respectively, and according to a result of the comparison, Evaluate 2 sentences,
The sentence evaluation apparatus according to any one of claims 4 to 6, wherein
The first event and the second event are expressed by keywords or key phrases,
The sentence evaluation apparatus according to any one of claims 1 to 7, wherein
A second sentence storage means for associating and storing the second sentence, the extracted second logical relation, and a range representing the second logical relation in the second sentence;
The evaluation means outputs a range representing the second logical relationship corresponding to the first logical relationship in the second sentence together with the second sentence.
The sentence evaluation apparatus according to any one of claims 1 to 8, wherein
Obtaining a first logical relationship that is established between first events constituting a first sentence serving as a reference for evaluation;
Extracting a second logical relationship that holds between second events constituting the second sentence from the second sentence to be evaluated;
Evaluating the second sentence by comparing the first logical relationship with the second logical relationship;
A sentence evaluation method characterized by that.
A weight is associated with the first logical relationship,
The second sentence is evaluated according to the weight;
The sentence evaluation method according to claim 10.
A process of acquiring a first logical relationship established between first events constituting a first sentence serving as a reference for evaluation;
A process of extracting a second logical relationship established between second events constituting the second sentence from the second sentence to be evaluated;
A process of evaluating the second sentence by comparing the first logical relationship and the second logical relationship;
A computer-readable non-transitory recording medium that records a program that causes a computer to execute the program.
A weight is associated with the first logical relationship,
The process of evaluating the second sentence evaluates the second sentence according to the weight.
The recording medium according to claim 12.