JP7480560B2 - Text extraction device and program - Google Patents

Description

The present invention relates to a text extraction device and a program.

Conventionally, in order to edit text contained in PDF (Portable Document Format) data of books, magazines, etc., a task has been performed in which a part of the displayed PDF data is designated to be used for editing the text, and then the text contained in the designated area is extracted.
PDF data is data in a file format that stores the state of an image when it is actually printed on paper, and includes various meta-information.

FIG. 1 is a diagram for explaining a part of meta information of PDF data.
The PDF data 91 shown in FIG. 1 is made up of text 92, text lines 93, and text boxes 94 (text objects), which have a hierarchical structure.
The text 92 is a one-character region. The text 92 includes character data (the character itself) such as "o" as well as information such as the font, font size, position information of the one-character region, and size of the region as meta information.
A text line 93 is a one-line area made up of a plurality of texts 92. The text line 93 has meta-information such as the position information of the one-line area and the size of the area.
A text box 94 is a rectangular area that groups together a plurality of text lines 93. The text box 94 has meta information such as the position information of the rectangular area and the size of the area.

PDF data has the feature of being able to embed information. For example, a designer creating PDF data may embed information in advance as text, such as which document is to be placed at which position in the page data, and how. Information embedding devices that embed text into document files have been disclosed (for example, Patent Document 1).

JP 2015-12444 A

As described above, characters embedded for editing purposes are invisible characters that are not visible when displayed as PDF data. When such invisible characters are embedded, the PDF data will have a text box that contains the embedded characters and a text box for the displayed text.
Therefore, a case where characters are extracted from PDF data and output using an existing method will be described with reference to FIG. 2 as an example.
For example, in the case of PDF data 71 shown in Fig. 2A, a designated area 72 indicating the range that the user wishes to output as editable characters includes text boxes 71a and 71b. Sentence 75 is data resulting from outputting the characters included in the designated area 72. Sentence 75 is data resulting from outputting the character data included in text boxes 71a and 71b included in the designated area 72 based on position information of each piece of character data. When the character data of the text boxes does not overlap, as in this case, the output sentence 75 can be understood as sentence.
On the other hand, in the case of PDF data 81 shown in Fig. 2(B), the designated area 82 includes a text box 81a and a part of a text box 81b. Here, the text box 81b includes embedded invisible characters. Sentence 85 is data resulting from outputting the characters included in the designated area 82. Sentence 85 is the output of character data included in the text box 81a and part of the text box 81b included in the designated area 82 based on position information of each piece of character data. When the character data of the text boxes overlap, as in this case, the output sentence 85 becomes incomprehensible as sentence.

The present invention aims to provide a text extraction device and program that are designed to output text that can be understood as text.

The present invention solves the above problems by the following solving means.
A first invention is a text extraction device that extracts character strings according to positions of embedded characters from page data having a plurality of text objects, and includes: a character extraction means that extracts characters in a specified processing target area from the displayed page data; an object identification means that identifies the text object including the characters extracted by the character extraction means; a determination means that determines whether or not there is overlap between characters in the plurality of text objects identified from the processing target area by the object identification means; and an output object determination means that determines output content according to a determination result by the determination means.
A second invention is a sentence extraction device according to the first invention, wherein when the determination means determines that there is an overlap as a result of the determination, the output object determination means calculates an area ratio of each of the identified text objects to the processing target area, and determines the output content according to the calculated area ratio.
A third invention is a sentence extraction device according to the first invention, further comprising a recognized character acquisition means for acquiring the characters included in the processing target area by optical character recognition, and when the determination means determines that there is an overlap as a result of the determination made by the determination means, the output object determination means calculates a degree of match between the characters acquired by the recognized character acquisition means and sentences included in each of the text objects identified for the processing target area, and determines the output content in accordance with the calculated degree of match.
A fourth invention is a sentence extraction device according to the first invention, wherein the output object determination means determines, when the determination means determines that there is no overlap as a result of the determination, that a sentence contained in each of the plurality of text objects is to be the output content.
A fifth invention is a sentence extraction device, which is any of the first to fourth inventions, and which includes a sentence output means for arranging the output content determined by the output object determination means based on meta information corresponding to the text object and outputting the output content to an editing screen.
A sixth invention is a sentence extraction device which, in any of the first to fifth inventions, is provided with an area receiving means for receiving designation of a designated area including sentences in the page data, and the character extraction means extracts the characters in the processing target area, with the designated area being the processing target area.
A seventh aspect of the present invention is the text extraction device according to any one of the first to sixth aspects of the present invention, wherein the page data is data in a PDF format.
An eighth aspect of the present invention is a program for causing a computer to function as any one of the sentence extraction devices according to the first to seventh aspects of the present invention.

The present invention provides a text extraction device and program that are designed to output text that can be understood as text.

FIG. 11 is a diagram for explaining a part of meta information of PDF data. FIG. 1 is a diagram showing an example of outputting characters from PDF data according to an existing method. 1 is a functional block diagram of a text output device according to an embodiment of the present invention; 5 is a flowchart showing a sentence output process of the sentence output device according to the present embodiment. 10 is a flowchart showing an overlap process of the text output device according to the embodiment. 11A and 11B are diagrams for explaining overlap processing of the text output device according to the embodiment; 11A and 11B are diagrams for explaining overlap processing of the text output device according to the embodiment; 5A to 5C are diagrams illustrating an example of processing performed by the text output device according to the embodiment; 5A to 5C are diagrams illustrating an example of processing performed by the text output device according to the embodiment; 5A to 5C are diagrams illustrating an example of processing performed by the text output device according to the embodiment;

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited to this example.
(Embodiment)
FIG. 3 is a functional block diagram of the text output device 1 according to the present embodiment.

<Text output device 1>
The text output device 1 (text extraction device) is a device for outputting text (character strings) of PDF data (page data) in an editable manner. The text output device 1 is used, for example, by a user who wants to edit text included in PDF data. The text output device 1 is, for example, a personal computer (PC). The text output device 1 may also be a tablet terminal or a portable terminal having computer functions such as a smartphone. The text output device 1 may also be a server or the like.

As shown in FIG. 3, the text output device 1 includes a control unit 10 , a storage unit 30 , an input unit 36 , a display unit 37 , and a communication interface unit 39 .
The control unit 10 is a central processing unit (CPU) that controls the entire text output device 1. The control unit 10 appropriately reads and executes an operating system (OS) and application programs stored in the storage unit 30, thereby cooperating with the above-mentioned hardware to execute various functions.
The control unit 10 includes a page data output unit 11, an area acceptance unit 12 (area acceptance means), a character extraction unit 13 (character extraction means), an object identification unit 14 (object identification means), an overlap determination unit 15 (determination means), an output object determination unit 16 (embedded character object identification means), and a text output unit 17 (text output means).

The page data output unit 11 outputs the PDF data to be edited to the display unit 37. For example, when the user selects a desired document file in PDF format, the control unit 10 accepts the selected document file and displays the PDF data on the display unit 37. The PDF data is data of a print image, for example, data for one page. The PDF data may be, for example, layout data. The PDF data also includes text. The PDF data may be data of only text, or may include illustrations for explanation.

The area receiving unit 12 receives the designation of a specified area (area to be processed) that designates a partial area of the PDF data. The area receiving unit 12 may receive a rectangular designated area by the user using the input unit 36, such as a mouse, to drag from the upper left to the lower right of the PDF data displayed on the display unit 37 so as to include the text that the user wishes to edit.

The character extraction unit 13 extracts character data included in the designated area accepted by the area acceptance unit 12, based on meta-information obtained by analyzing the PDF data.
The object identification unit 14 identifies a text box (text object) including the characters extracted by the character extraction unit 13. The object identification unit 14 can identify a text box based on meta information obtained by analyzing the PDF data.
The overlap determination unit 15 determines whether or not there is overlap between characters in the multiple text boxes identified by the object identification unit 14. The overlap determination unit 15 grasps the position information and size of one character region from meta information of the characters that make up the extracted character data. Then, the overlap determination unit 15 can determine whether or not there is overlap between characters using the grasped position information and size of one character region.

The output object determination unit 16 determines the output contents according to the determination result by the overlap determination unit 15 .
More specifically, when the overlap determination unit 15 determines that there is an overlap, the output object determination unit 16 calculates the area ratio of each of the text boxes specified by the object specification unit 14 to the designated area, and determines the output contents according to the calculated area ratio. Here, the output object determination unit 16 may determine the text included in the text box with the calculated area ratio being high as the output contents. Then, the control unit 10 may determine the text included in the text box with the calculated area ratio being low as embedded characters. Here, embedded characters are invisible characters that are not usually included in displayed PDF data or printed PDF data. Also, the output object determination unit 16 may determine both the text included in the text box with the calculated area ratio being high and the text included in the text box with the calculated area ratio being low as the output contents.

Furthermore, when the judgment result by the overlap judgment unit 15 indicates that there is no overlap, the output object determination unit 16 determines the sentences contained in each of the text boxes identified by the object identification unit 14 as the output content.
Here, the output content determined by the output object determination unit 16 may include not only a sentence (character string) but also a display mode of the sentence. As a display mode of the sentence, the output object determination unit 16 may, for example, make the character color, character size, and character font of the embedded character different from the character color, character size, and character font of characters other than the embedded character. Furthermore, as a display mode of the sentence, the output object determination unit 16 may, for example, add the embedded character by underlining or parentheses to the end of the characters other than the embedded character.

The text output unit 17 outputs an editing screen in which the output contents determined by the output object determination unit 16 are arranged based on the meta information of each text and text box. In this case, the text output unit 17 may arrange embedded characters on the editing screen based on the display mode of the text and output the text. The text output unit 17 may also store the output text in the text storage unit 33.

The storage unit 30 is a storage area such as a hard disk, a semiconductor memory device, etc. for storing programs, data, etc. required for the control unit 10 to execute various processes.
The storage unit 30 includes a program storage unit 31 , a document file storage unit 32 , and a text storage unit 33 .
The program storage unit 31 is a storage area for storing various programs. The program storage unit 31 stores a program 31a.
The program 31 a is a program for carrying out various functions executed by the control unit 10 of the text output device 1 .

The document file storage unit 32 is a storage area for storing various document files. Here, the document files stored in the document file storage unit 32 may be files containing the contents of magazines, books, etc.
The document file storage unit 32 may be included in, for example, a document server (not shown) communicatively connected to the text output device 1 .
The text storage unit 33 is a storage area for storing the text output by the text output unit 17. When the stored text is subsequently edited, the text storage unit 33 may further store the edited text.

The input unit 36 is, for example, an input device such as a keyboard or a mouse.
The display unit 37 is, for example, a display device such as an LCD (Liquid Crystal Display).
The communication interface unit 39 is an interface for communicating with other devices via a communication network.

<Processing of Text Output Device 1>
Next, the process of the text output device 1 will be described.
FIG. 4 is a flowchart showing a text output process of the text output device 1 according to this embodiment.
FIG. 5 is a flowchart showing the overlap process of the text output device 1 according to the present embodiment.
6 and 7 are diagrams for explaining the overlap process of the text output device 1 according to the present embodiment.

4, in step S (hereinafter referred to as "S") 11, the control unit 10 (page data output unit 11) outputs the PDF data to the display unit 37. For example, the user may select one of the document files stored in the document file storage unit 32, and the control unit 10 may output the PDF data of the selected document file.
In S12, the control unit 10 (area receiving unit 12) receives, through a user operation on the PDF data displayed on the display unit 37, a designation of a designated area including a text that the user wishes to edit.
In S13, the control unit 10 (character extraction unit 13) extracts character data included in the specified area based on meta information of the PDF data.
In S14, the control unit 10 (object identification unit 14) identifies text boxes included in the specified area based on meta information of the PDF data.

In S15, the control unit 10 (overlap determination unit 15) determines whether or not there is overlap between characters in the identified multiple text boxes.
In S16, the control unit 10 (overlap determination unit 15) determines whether or not there is overlap between characters. If there is overlap (S16: YES), the control unit 10 proceeds to S17. On the other hand, if there is no overlap (S16: NO), the control unit 10 proceeds to S18.
In S17, the control unit 10 performs overlap processing.

Here, the overlap process will be described with reference to FIG.
In S21 of FIG. 5, the control unit 10 (output object determination unit 16) calculates the degree of overlap as an area ratio with respect to the designated region for each text box including overlapping characters.
An example of a method for calculating the degree of overlap is shown in FIG.
FIG. 6 shows a method for calculating the degree of overlap between the designated area 41 and the text box 42. In FIG.

6A is shown as a rectangular area having a length _w1 in the X-axis direction and a length _h1 in the Y-axis direction. Also, the text box 42 is shown as a rectangular area having a length _w2 in the X-axis direction and a length _h2 in the Y-axis direction. In this case, the overlapping portion can be represented as a rectangular area having a length _w3 in the X-axis direction and a length _h3 in the Y-axis direction.
As one method of calculating the area ratio, if the degree of overlap of the text box 42 with respect to the designated area 41 is IoU (Intersection over Union), the degree of overlap IoU can be expressed by the formula shown in Fig. 6B. In other words, the larger the rectangular area having a length _w3 in the X-axis direction and a length _h3 in the Y-axis direction, which is the overlapping portion between the designated area 41 and the text box 42, the larger the degree of overlap IoU becomes.

FIG. 7A shows a specific example of calculating each degree of overlap IoU.
7A shows a designated area 51 and text boxes 51a and 51b. In this example, text boxes 51a and 51b overlap within the designated area 51. Therefore, the control unit 10 calculates the degree of overlap IoU for each text box.
First, as shown in Fig. 7B, the control unit 10 calculates the degree of overlap IoU between the designated area 51 and the text box 51a. The control unit 10 can calculate the degree of overlap IoU in this case to be 0.9 by applying the formula in Fig. 6B.
Next, as shown in Fig. 7C, the control unit 10 calculates the degree of overlap IoU between the designated area 51 and the text box 51b. The control unit 10 can calculate the degree of overlap IoU in this case to be 0.1 by applying the formula in Fig. 6B.

In S22 of Fig. 5, the control unit 10 (output object determination unit 16) identifies the character data in the text box with the calculated low area ratio as an embedded character. In the example shown in Fig. 7, the control unit 10 identifies the character data included in the text box 51b as an embedded character.
In S23, the control unit 10 (output object determination unit 16) determines the text in the text box with the calculated high area ratio as the output content. After that, the control unit 10 moves the process to S19 in FIG.

On the other hand, in S18 of FIG. 4, the control unit 10 (output object determination unit 16) determines the sentences included in each of the text boxes as the output contents.
In S19, the control unit 10 (text output unit 17) arranges and outputs the determined output contents on the editing screen based on the meta information of each text and text box. The control unit 10 (text output unit 17) also stores the characters output to the editing screen in the text storage unit 33. Thereafter, the control unit 10 ends this process.

Next, an example of a text output by using the text output device 1 will be described.
8 to 10 are diagrams showing a processing example of the text output device 1 according to the present embodiment.
The PDF data 61 shown in Fig. 8 includes a text box 61a and a text box 61b. The text box 61b is a text box for invisible embedded characters, and actually includes characters provided at a position deep inside the surface. Therefore, in Fig. 8, the data is illustrated in a hierarchical structure to clearly show that the characters are embedded. The same applies to Figs. 9 and 10.

When the user designates a designated area 62 in the PDF data 61, the control unit 10 extracts the character data contained in the designated area 62 and identifies the text box (S13 and S14 in FIG. 4). Here, since text box 61a and text box 61b do not overlap, there is no overlap between the characters in the two text boxes. Therefore, the control unit 10 determines that there is no overlap (NO in S16 in FIG. 4), determines the character data of both text boxes 61a and 61b contained in the designated area 62 as the output content, and arranges and outputs the content on the editing screen 63 based on the meta information (S18 and S19 in FIG. 4).

In this way, in the example shown in FIG. 8, there is no overlap between text boxes, so the control unit 10 determines that there is no overlap between characters in the text boxes, and outputs the output characters, including the embedded characters, to the editing screen without processing whether the character data contained in the text boxes is an embedded character (overlap processing). This is because the sentences are not combined with other sentences to make them incomprehensible.

9 includes a text box 64a and a text box 64b. The text box 64b is a text box containing invisible embedded characters.
When the user designates a designated area 65 in the PDF data 64, the control unit 10 extracts character data included in the designated area 65 and identifies a text box (S13 and S14 in FIG. 4). Since the text box 64a and the text box 64b overlap, the control unit 10 next determines whether or not there is an overlap between the characters based on the meta information of each character in the text box 64a and the text box 64b (S15 in FIG. 4). Then, the control unit 10 determines that there is no overlap between the characters (NO in S16 in FIG. 5), and determines the character data of both the text box 64a and the text box 64b included in the designated area 65 as the output content, and arranges and outputs the data on the editing screen 66 based on the meta information (S18 and S19 in FIG. 4).

Thus, in the example shown in FIG. 9, in addition to all of the characters in text box 64a, some of the characters in text box 64b that are included in specified area 65 are displayed on editing screen 66. In other words, all of the characters included in specified area 65 are output on editing screen 66. In this example, embedded characters are also output, but this is because the characters do not overlap, and so even if they are output, they will not be combined with other sentences and become incomprehensible.

10 includes a text box 67a and a text box 67b. The text box 67b is a text box for invisible embedded characters.
When the user designates the designated area 68 in the PDF data 67, the control unit 10 extracts character data included in the designated area 68 and identifies a text box (S13 and S14 in FIG. 4). Here, since the text box 67a and the text box 67b overlap, the control unit 10 next determines whether or not there is an overlap between the characters based on the meta information of each character in the text box 67a and the text box 67b (S15 in FIG. 4). The control unit 10 determines that there is an overlap between the characters (YES in S16 in FIG. 5), and performs overlap processing (FIG. 5). The control unit 10 identifies the character in the text box 67b with the calculated low area ratio as an embedded character (S22 in FIG. 5), and determines the text in the text box 67a with the calculated high area ratio as the output content (S23 in FIG. 5). Then, the control unit 10 arranges the determined output content on the editing screen 69 based on the meta information and outputs it (S19 in FIG. 4).

10, only all the characters in the text box 64a included in the specified area 65 are displayed on the editing screen 69. The characters in the text box 64b are not output. Thus, the text output device 1 can prevent the characters in the text box 64b from being combined and output, and can output a text that is understandable as a text.
10, in which the characters in the text box 64b are not output, is merely an example, and the characters in the text box 64b may also be output. However, it is necessary to make the display mode different from that of the characters in the text box 64a so that it can be recognized that they are different sentences.

As described above, the text output device 1 of the present embodiment has the following advantages.
(1) Extract character data contained in a specified area of PDF data, identify text boxes, determine whether or not there is overlap between characters in multiple text boxes, and determine output contents according to the determination result.
Therefore, the sentences output on the editing screen will depend on the results of determining whether or not there is overlap between characters. For example, if there is overlap between characters, the overlapping sentences will be output separately, making it possible to make the sentences understandable.

(2) When it is determined that there is overlap between characters in a plurality of text boxes, the area ratio of each of the text boxes to the designated region is calculated, and the output contents are determined according to the calculated area ratio.
Therefore, the output contents can be determined by utilizing the degree of overlap based on the position information of the specified area and the text box. Therefore, if the user specifies an area so that the specified area includes the text to be output, the area ratio will be high, and the output contents will be as intended by the user.

(3) If it is determined that there is no overlap between characters in a plurality of text boxes, the sentences contained in each of the plurality of text boxes included in the specified area are determined as the output content.
Therefore, the text contained in the specified area can be used as the output content, and the output content can be as intended by the user.

(4) An editing screen is output in which the output contents are arranged based on the meta information corresponding to the text boxes.
Therefore, the text can be output on the editing screen in the same way as the PDF data, including the position and size of each character.
(5) A designation of a designated area including text in the PDF data is accepted, and character data included in the accepted designated area is extracted based on meta information.
Therefore, the text in the area specified by the user can be output.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments. Furthermore, the effects described in the embodiments are merely a list of the most favorable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. Note that the above-mentioned embodiments and the modified forms described below can be used in appropriate combinations, but detailed explanations will be omitted.

(Modifications)
(1) In the present embodiment, the text included in the designated area designated by the user is output, but the present invention is not limited to this. For example, the page data, which is all of the PDF data displayed on the display unit as PDF data, may be used as the processing target area, and the text may be output.

(2) In this embodiment, the output content is determined using the degree of overlap of text boxes, but this is not limiting. Characters contained in a specified area of PDF data may be obtained using OCR (optical character recognition), and the output content may be determined based on the degree of match between the obtained characters and the text contained in each text box. With this method, the output content can be determined by taking advantage of the fact that characters obtained using OCR do not include invisible characters such as embedded characters.

(3) In the present embodiment, when there is overlap between characters in multiple text boxes, the text contained in the text box with the highest calculated area ratio among the multiple text boxes is determined as the output content. However, this is not limited to this. For example, the display mode of the text contained in the text box with the lowest calculated area ratio may be set to a different mode from the display mode of the text contained in the text box with the highest calculated area ratio, and both texts may be output.

(4) In this embodiment, when there is no overlap between characters in multiple text boxes, the text contained in the text boxes is determined as the output content. In this case, the text in each text box may be displayed in a different manner. This makes it possible to grasp the unity of the text at a glance.

(5) In this embodiment, PDF data has been described as an example, but this is not limiting. Other types of page data may be used in the same manner as long as they have multiple text boxes and meta information for each sentence or character that makes up the sentence.

(6) In this embodiment, the text output device has been described as a device including an input unit and a display unit, but is not limited to this. The input unit and display unit may be included in, for example, a user terminal, and the text output device may not include an input unit and a display unit. In this case, the user terminal may be communicatively connected to the text output device to perform processing.

REFERENCE SIGNS LIST 1 Text output device 10 Control unit 11 Page data output unit 12 Area reception unit 13 Character extraction unit 14 Object identification unit 15 Overlap determination unit 16 Output object determination unit 17 Text output unit 30 Memory unit 31a Program 32 Document file memory unit 33 Text memory unit 36 Input unit 37 Display unit 39 Communication interface unit 41, 51, 62, 65, 68 Designated area 42, 51a, 51b, 61a, 61b, 64a, 64b, 67a, 67b Text box 61, 64, 67 PDF data 63, 66, 69 Editing screen

Claims (7)

A text extraction device that extracts character strings from page data having a plurality of text objects according to positions of embedded characters, comprising:
character extraction means for extracting characters in a specified processing target area from the displayed page data;
an object specifying means for specifying the text object including the character extracted by the character extracting means;
a determination means for determining whether or not there is an overlap between characters in the plurality of text objects identified from the processing target area by the object identification means;
a recognized character acquisition means for acquiring the characters included in the processing target area by optical character recognition;
an output object determination means for calculating a degree of coincidence between characters of a sentence included in each of the plurality of text objects specified from the processing target area and the characters acquired by the recognized character acquisition means, when it is determined that there is an overlap as a result of the determination by the determination means, and determining a sentence included in the text object to be output from the plurality of text objects based on the calculated degree of coincidence;
A text extraction device comprising: 2. The text extraction device according to claim 1,
The output object determination means, when the determination means determines that there is no overlap as a result of the determination, determines the sentences included in each of the plurality of text objects as sentences to be output . In the text extraction device according to claim 1 or 2,
The output object determination means further determines a display mode of the sentence to be output. In the text extraction device according to any one of claims 1 to 3 ,
The text extraction device further comprises a text output means for arranging the text to be output determined by the output object determination means based on meta information corresponding to the text object and outputting the text to an editing screen. In the text extraction device according to any one of claims 1 to 4 ,
an area receiving means for receiving a designation of a designated area including a text in the page data;
The character extraction means extracts the characters from the specified region as the processing target region. In the text extraction device according to any one of claims 1 to 5 ,
The document extraction device, wherein the page data is data in a PDF format. A program for causing a computer to function as the text extraction device according to any one of claims 1 to 6 .

Images