JPH07104865B2 - Sentence cutting device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sentence cutting device for cutting sentences written in a natural language, and more particularly to a sentence cutting device for cutting sentences written in a table or a graph.

[0002]

2. Description of the Related Art In the case of machine translation of a document described in natural language, it is necessary to divide the document into sentences, that is, to cut the sentence.

Conventionally, in such a case, in Japanese sentences, sentence cutting is performed based on the punctuation mark ".", And in English sentences, sentence cutting is performed based on the period "." (For example, JP-A-64-61863).

If there are punctuation marks or periods in the document, sentence cutting can be easily performed based on the above-mentioned conventional technique.

However, since a document in a table or a document in a graph generally does not use a punctuation mark or a period, when a sentence of a plurality of lines exists in a table or a graph, in the above-mentioned conventional technique, it is consecutive 1 It is not possible to determine whether the sentence is folded or a plurality of sentences are written in parallel.

Therefore, a technique of determining whether or not adjacent morphemes (words) are connectable based on the part of speech of the adjacent morphemes (for example, Japanese Patent Laid-Open No. 61-16367) is used. It can be considered to cut a document or a document in a graph. That is, based on the part of speech of the last morpheme on the i-th line and the part of speech of the first morpheme on the (i + 1) th line, it is determined whether or not they are connectable. It is determined that the line and the (i + 1) th line are continuous sentences, and if the connection is impossible, the i-th line and the (i + 1) th line are different sentences.

[0007]

However, since the above-mentioned technique only determines whether or not the sentence is a continuous sentence based on the parts of speech of the adjacent morphemes, there is a problem that a determination error is likely to occur. For example, it is generally difficult to connect a sequence of nouns and verbs, but there are cases where they are connected like "I go," so it is only necessary to determine whether or not they are continuous sentences based on the parts of speech of adjacent morphemes. Judgment error is likely to occur.

An object of the present invention is to provide a sentence cutting device which is less likely to cause a judgment error when cutting a document in a table or a graph.

[0009]

In order to achieve the above object, the present invention resides in a layout analysis means for extracting a table portion and a graph portion in an original text, and a table analysis portion and a graph portion extracted by the layout analysis means. Text
Text extraction means for extracting each unit that can be regarded as one sentence,
Morphological analysis and syntactic analysis are performed for each unit that can be regarded as one sentence extracted by the text extraction unit, and an analysis unit that determines the connection cost between each line of the unit that can be regarded as one sentence and the presence or absence of modification, and the analysis unit A determination unit that determines whether or not each line of the unit that can be regarded as one sentence is continuous based on the connection cost between lines and the presence or absence of modification, and a unit that can be regarded as one sentence based on the determination result of the determination unit A sentence dividing / combining means for cutting is provided.

[0010]

The layout analysis means extracts the table portion and the graph portion in the original text, and the text existing in the table portion and the graph portion is extracted by the text extracting means for each unit that can be regarded as one sentence.

The analyzing means is 1 extracted by the text extracting means.
Morphological analysis and syntactic analysis are performed for each unit that can be regarded as a sentence, and the connection cost between each line of the unit that can be regarded as one sentence and the presence or absence of modification are obtained.

The determination means determines whether or not each line of the unit extracted by the text extraction means is continuous, based on the connection cost between each line and the presence or absence of modification obtained by the analysis means.

The sentence dividing / combining unit cuts a unit which can be regarded as one sentence extracted by the extracting unit according to the determination result of the determining unit.

[0014]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 1 is a block diagram of an embodiment of the present invention. Input means 10, storage means 20, layout analysis means 30, text extraction means 40, and analysis means 50.
And a judgment unit 60, a sentence division / coupling unit 70, and an output unit 80.

The input means 10 is for inputting an original text described in natural language, and comprises a floppy disk device, a magnetic tape device, an optical reading device (OCR), a keyboard and the like.

The storage means 20 stores the original text input by the input means 10, and is constituted by a storage device in a computer such as a memory device or a magnetic disk device.

The layout analysis means 30 converts the original text stored in the storage means 20 into a table portion, a graph portion,
It is divided into a text portion (character string portion) and the table portion and the graph portion are passed to the text extracting means 40. Storage means 20
The textual text stored in the various text interchange standard formats (eg,
If it is described in SGML, Postscript, etc.), an analysis program for these document exchange standard formats can be used as the layout analysis means 30.

The text extracting means 40 extracts the text existing in the table portion and the graph portion passed from the layout analyzing means 30 for each unit that can be regarded as one sentence, and the analyzing means 50
And a function of passing the sentence to the sentence dividing / combining means 70.

The analysis means 50 performs a morpheme analysis and a syntax analysis on the text passed from the text extraction means 40, a function of obtaining a connection cost between lines based on the analysis result, and a second function based on the analysis result. A function for creating a modification presence / absence information indicating presence / absence of existence of a morpheme that modifies the leading morpheme on and after the line, and a connection cost and modification presence / absence information to the determination unit 60. It has a function of outputting. The connection cost is information indicating the ease of connection, and in the present embodiment, "low", "medium",
Ease of connection shall be represented by three levels of "high". Also, the lower the connection cost, the easier the connection.

The judging means 60 has a function of judging whether or not a certain row and the next row are continuous based on the connection cost and the modification presence / absence information from the analyzing means 50.

The sentence dividing / combining means 70 has a function of outputting the text delivered from the text extracting means 40 in a divided form or a combined form according to the judgment result of the judging means 60.

The output means 80 is means for achieving input / output with the outside, and is constituted by a floppy disk device, a magnetic tape device, a printer, a display device and the like.

FIG. 2 is a flow chart showing a processing example of the text extracting means 40, and FIG. 3 is a flow chart showing a processing example of the analyzing means 50.
FIG. 4 is a diagram showing an example of the judgment criteria of the judging means 60, and the operation of this embodiment will be described below with reference to the drawings.

The input means 10 inputs the original text described in natural language and stores it in the storage means 20.

When the original text is stored in the storage means 20, the layout analysis means 30 divides the original text into a table portion, a graph portion and a text portion, and the table portion and the graph portion are sent to the text extraction means 40. hand over. Now, for example, if the original text stored in the storage means 20 is as shown in FIG.
1, the graph portion 52, and the table portion 53 are divided, and the graph portion 52 and the table portion 53 are passed to the text extracting means 40.

When the table portion or the graph portion is transferred from the layout analyzing means 30 to the text extracting means 40, the text extracting means 40 shown in FIG.
The process shown in the flowchart of FIG.

When the front part is passed (step S1 is Y
ES), the text extraction means 40 first treats the top column of the table as a processing target (step S2), regards the text in the top column as one continuous sentence, and passes it to the analysis means 50 (step S3).

After that, the text extracting means 40 judges whether or not there is an unprocessed column in the table portion (step S
4) If it is determined that there is an unprocessed column, the process target is set to the next column (step S5), and then step S3.
Return to processing. If it is determined in step S4 that there is no unprocessed column, the text extracting means 40 ends the process.

When the graph portion is passed from the layout analysis means 30 (NO in step S1), the text extracting means 40 searches the passed graph portion from top to bottom (step S6).

When a character is detected (step S7)
Is YES), the text extracting means 40 determines whether the detected character is an unprocessed character (step S8).

When it is determined that the character is an unprocessed character (YES in step S8), the text extracting means 40 reads the character in the row direction starting from the character detected in step S7 until a blank column appears (step S9). ).

Thereafter, the text extracting means 40 goes to the next line and judges whether or not a character exists in the same column as the column from which the character string was read in step S9 (step S1).
0).

When it is determined that the character exists, the process returns to step S9 and the characters are read in the line direction until a blank column appears in the next line.

The processes of steps S9 and S10 are repeated until the result of the determination in step S10 is NO,
If the decision result in the step S10 is YES, the text extracting means 40 regards the unoutputted text among the texts read in the step S9 as one continuous sentence and passes it to the analyzing means 50 and the sentence dividing / combining means 70. (Step S11), the process returns to step S6. That is, the text extraction means 40 regards the text surrounded by the blanks as the text of the graph portion and outputs it to the analysis means 50 and the sentence division / connection means 70 as one continuous sentence.

When the text extracting means 40 has performed a search up to the end of the graph portion (step S12 is Y
ES) ends the processing.

When the text in the table or the graph is delivered from the text extracting means 40, the analyzing means 50 starts the processing shown in the flow chart of FIG.

First, the analysis means 50 is the text extraction means 4
It is judged whether or not the text passed from 0 has a plurality of lines (step S21).

When it is determined that the text has a plurality of lines, the analysis unit 50 performs a morpheme analysis on the text and obtains a connection cost between each morpheme based on the part of speech of the morpheme (step S22). Further, the analysis means 50 performs a syntax analysis to check the modification relation of each morpheme (step S2).
3).

After that, the analyzing means 50 obtains the connection cost between the respective rows based on the connection cost between the respective morphemes obtained at the step S22 (step S24). That is, the connection cost between the last morpheme of the i-th row and the head morpheme of the (i + 1) th row is the connection cost between the i-th row and the (i + 1) th row. However, one morpheme is 2 in the i-th row and the (i + 1) -th row.
If it spans rows, the connection cost between the i-th row and the (i + 1) -th row is set to the value that indicates the easiest connection, that is, "low".

When the connection cost between each line is obtained, the analysis means 50 makes a second determination in step S25 based on the result of the syntax analysis.
For each morpheme existing at the beginning of each line after the line, modification presence / absence information indicating whether or not a morpheme that modifies it is generated.

That is, when the morpheme that modifies the leading morpheme of the i-th row is before and including the (i-1) -th row, it is between the (i-1) -th row and the i-th row. The modification presence / absence information is set to “present” indicating that there is modification, and if not, the modification presence / absence information between the (i−1) th row and the i-th row is set to “absent” indicating no modification. However, one morpheme is 2 in the (i-1) th row and the i-th row.
When it extends over a line, the modification presence / absence information between the (i−1) th line and the i-th line is set to “present” indicating that there is modification regardless of the presence or absence of modification.

When the connection cost between each row and the presence / absence of modification are obtained in steps S24 and S25, the analysis means 50 passes the connection cost between each row and the presence / absence of modification to the determination means 60.
(Step S26).

When the connection cost and modification presence / absence information between each line are sent from the analysis unit 50, the judging means 60 judges whether or not each line is continuous according to the judgment criteria shown in FIG. It is passed to the dividing / combining means 70.

That is, the connection cost between the i-th row and the (i + 1) th row passed from the analyzing means 50 is "low", and the connection cost is connected between the i-th row and the (i + 1) th row. If it is easy, the determination unit 60 determines that the i-th row and the (i + 1) -th row are continuous regardless of the presence / absence of the modification presence / absence information. When the connection cost between the i-th row and the (i + 1) -th row passed from the analysis means 50 is “medium” or “high”, the determination means 60 determines that the modification presence / absence information is “none”. Is determined not to be continuous, and if “present” is determined to be continuous.

When the judgment result is passed from the judgment means 60, the sentence dividing / combining means 70 divides or combines the texts passed from the text extracting means 40 according to the judgment result, and outputs it to the output means 80. Further, the text divided or combined by the sentence dividing / combining means 70 can be input to another document processing program.

[0047]

As described above, according to the present invention, the text segmentation of the table portion and the graph segment is performed based on the connection cost between each line and the presence / absence of modification. Therefore, the sentence segmentation with high reliability is performed. There is an effect that can be.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing example of a text extracting means 40.

FIG. 3 is a flow chart showing a processing example of the analyzing means 50.

FIG. 4 is a diagram showing an example of a determination criterion of a determination means 60.

FIG. 5 is a diagram showing an example of original text.

[Explanation of symbols]

 10 ... Input means 20 ... Storage means 30 ... Layout analysis means 40 ... Text extraction means 50 ... Analysis means 60 ... Judgment means 70 ... Sentence division / joining means 80 ... Output means

Claims (2)

[Claims] 1. A layout analysis means for extracting a table portion and a graph portion in the original text, and a text for extracting the text existing in the table portion and the graph portion extracted by the layout analysis means for each unit that can be regarded as one sentence. An extraction unit, an analysis unit that performs morphological analysis and syntactic analysis for each unit that can be regarded as one sentence extracted by the text extraction unit, and obtains the connection cost between each line of the unit that can be regarded as one sentence and the presence or absence of modification; Determination means for determining whether or not each line of the unit that can be regarded as one sentence is continuous based on the connection cost between each line and the presence or absence of modification obtained by the means, and the one sentence based on the determination result of the determination means. A sentence segmentation device comprising sentence segmentation / combination means for segmenting sentences that can be regarded. 2. The text extracting unit regards the text of the table portion as a unit in which the text existing in each column can be regarded as one sentence, and the text of the graph portion is one sentence each. The sentence cutting device according to claim 1, which is a unit that can be regarded as.