JPH08180131A - Image processing method - Google Patents

Abstract

PURPOSE: To save the trouble of correction, etc., by giving read order to character areas by using the continuity of the character areas found by analyzing images of characters or documents in the character areas when it is judged whether or not the character areas are continuous. CONSTITUTION: To check the continuity of the character areas 21 and 22, the two areas are taken out and while the character area 21 is regarded as a basic character area, the character area 22 is regarded as a compared character area. When the basic character area is determined, plural character areas extracted from an input image are determined in order from an area which is close to the start point that is the right upper point of each character area where a document is longitudinally written or the left upper point when the document is laterally written. The continuity of a candidate character area of the compared character area B is judged and when it is judged that only one character area is continuous, the character area is decided as an area succeeding to the basic character area, but when there are plural continuous areas, the area having the maximum continuity is determined as a succeeding area.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for analyzing an input image in a region unit in an electronic device such as an OCR (optical character recognition) device, a copying machine, a facsimile, and a DTP (desktop publishing). is there.

[0002]

2. Description of the Related Art A character area is extracted from a conventional input image,
As an ordering method for each character area, if each character area is written vertically, order from right to left, and if there are multiple character areas in the vertical direction as well, order them horizontally and then from top to bottom. There is a method of ordering from the relative position of each character area, such as ordering.

[0003]

However, in the above-mentioned prior art, when there are a plurality of articles in the manuscript, such as newspaper articles, and adjacent character areas belong to different articles and are not continuous with each other, Area 2 as in example 2
Areas other than characters exist between areas 1 and 22 and areas 23 and 24 to form a partition, and from the position of each character area, 21, 2
2, 23, 24 or 21, 23, 22, 24?
When it is not possible to distinguish whether or not it is the order, there is a problem that the correct reading order cannot be added as the reading order of each area, and the order determined by the processing device needs to be corrected.

The purpose of the present invention is to determine whether there are a plurality of articles such as newspaper articles in a manuscript, or if the order is 21, 22, 23, 24 from the position as in the example of FIG.
Even if the order of 23, 22, and 24 cannot be distinguished and the correct order cannot be added, the correct reading order is added to reduce the trouble of correction.

[0005]

In order to solve the above-mentioned problems, the present invention stores a document image, stores region information relating to at least two character regions existing in the document image, and stores the region information of the character region. Provided is an image processing method for determining whether or not sentences included in two character areas are continuous.

In order to solve the above-mentioned problems, the present invention preferably determines whether or not the sentences are continuous by analyzing the sentences included in the two character areas.

In order to solve the above-mentioned problems, the present invention is preferably arranged such that the area information relating to the stored character area is extracted by separating the stored original image into areas.

In order to solve the above-mentioned problems, the present invention preferably analyzes the sentence included in the character area with respect to a character obtained by character recognition of image information included in the character area.

In order to solve the above-mentioned problems, the present invention preferably determines whether or not the sentences included in the two character areas are continuous by determining the continuity index.

In order to solve the above problems, the present invention preferably increases the continuity index with respect to a character area existing in the direction indicated by the arrow when an arrow exists at the end of the character area. .

In order to solve the above-mentioned problems, the present invention is preferably such that the continuity index is the character area that is initially staggered when a symbol indicating the end of a sentence is present at the end of the character area. Increase to.

In order to solve the above-mentioned problems, in the present invention, the symbol indicating the end of the sentence is preferably a punctuation mark.

In order to solve the above-mentioned problems, in the present invention, the symbol indicating the end of the sentence is preferably a period.

In order to solve the above-mentioned problems, the present invention preferably uses the certainty as one sentence of the last sentence of the one character region and the first sentence of the other character region to make the continuation. Determine the sex index.

In order to solve the above problems, the present invention preferably determines the continuity index when the end of the character area is not a symbol indicating the end of a sentence.

In order to solve the above-mentioned problems, the present invention preferably has a certainty of the above-mentioned one sentence in that when one character region ends with a noun, the other character region begins with the first character region. When a sentence starts with a particle, it increases the certainty as a sentence.

In order to solve the above-mentioned problems, the present invention preferably has a certainty that the last sentence of one character area contains the subject but does not contain the predicate of the other character. When the first sentence of the area does not include the subject but includes the predicate, the certainty as one sentence is increased.

In order to solve the above-mentioned problems, the present invention is preferably such that the certainty of the one sentence is related to the subject included in the one character region and the predicate included in the other character region. Calculate using.

In order to solve the above-mentioned problems, the present invention preferably judges whether or not the sentence is continuous by using the existence ratio of words or synonyms commonly existing in both character areas.

In order to solve the above-mentioned problems, the present invention preferably determines whether or not the sentences are continuous by using the similarity between the character areas.

In order to solve the above-mentioned problems, the present invention preferably judges the similarity between the character areas with respect to the text expression between the character areas.

In order to solve the above-mentioned problems, the present invention preferably makes the text expressions polite.

In order to solve the above-mentioned problems, the present invention preferably makes the sentence expression a line-end expression.

In order to solve the above-mentioned problems, the present invention preferably makes the text expression a treatment expression.

In order to solve the above-mentioned problems, the present invention preferably judges the similarity between the character areas with respect to the composition ratio for each genre in the character area.

In order to solve the above-mentioned problems, in the present invention, the genre is preferably Chinese characters.

In order to solve the above problems, the present invention preferably defines the genre as hiragana.

In order to solve the above-mentioned problems, in the present invention, the genre is preferably katakana.

In order to solve the above-mentioned problems, in the present invention, the genre is preferably a symbol.

In order to solve the above-mentioned problems, in the present invention, the genre is preferably a numeral.

In order to solve the above problems, in the present invention, the genre is preferably an English character.

In order to solve the above problems, the present invention preferably determines the similarity between the character areas based on the format in the character areas.

In order to solve the above-mentioned problems, the present invention preferably makes the format a font.

In order to solve the above-mentioned problems, the present invention preferably makes the format a character size.

In order to solve the above-mentioned problems, the present invention preferably makes the format a line length.

In order to solve the above-mentioned problems, the present invention preferably has the character pitch.

In order to solve the above-mentioned problems, the present invention preferably makes the format a line pitch.

In order to solve the above-mentioned problems, the present invention is preferably arranged such that the format is the inclination of characters.

In order to solve the above-mentioned problems, the present invention preferably sets the format to be line inclination.

In order to solve the above-mentioned problems, the present invention is preferably such that the format is a space between characters.

In order to solve the above-mentioned problems, the present invention preferably makes the format a space between lines.

In order to solve the above-mentioned problems, the present invention is preferably such that the format is a set direction.

[0043]

[Embodiment] FIG. 18 is a block diagram showing the arrangement of an apparatus according to this embodiment. Reference numeral 1001 denotes a CPU that executes processing of the entire apparatus, and controls judgment and processing according to a control program stored in the ROM 1002. 10
Reference numeral 02 denotes a ROM, which stores a control program of the flowchart described in this embodiment, or data such as predetermined parameters used for processing. 10
A RAM 03 has a working memory area for storing data being processed by the CPU 1001. 100
Reference numeral 4 denotes a scanner that optically reads a document image. The read image data can be stored in the RAM 1003.
A keyboard 1005 is used to input various codes and operator's instructions. A pointing device 1006 can instruct a desired position on the display screen of the display 1007, and can also input a selection or cancellation instruction by clicking a button. 1007
Is a display, which is a CRT or a liquid crystal display. 1
Reference numeral 008 is, for example, an LBP or inkjet printer, 1009 is an external storage device such as FD, 101
Reference numeral 0 is a data bus for exchanging data between these components.

(Embodiment 1) FIG. 1 is a flowchart showing an ordering process of an embodiment according to the present invention.

FIG. 2 shows the scanner 1004 or FD10.
09 is an example of a document image input from a memory such as 09, and this image data is stored in the RAM 1003.

The inputted original image is taken as a histogram in the vertical and horizontal directions of the entire image, and the result is analyzed to extract a character region where a character string or a sentence is fixed to some extent, and is extracted. The position information of the area is stored in the RAM 1003, and when the image is analyzed for each area later, the image information specified from the position information of this area is extracted from the RAM 1003. Two
1, 22, 23, and 24 are extracted character areas.

Reference numeral 25 is a photographic area.

Each processing step in FIG. 1 will be described.

Step S101: Take out two character areas A and B that are to be checked for continuity (check the continuity from A to B), A is the basic character area,
Let B be the comparison character area.

In the example of FIG. 2, the character area following the character area 21 is the character area 22 adjacent to the left of the basic character area 21, or
It cannot be determined from the position whether it is one of the character areas 23 below the basic character area 21. Therefore, it is necessary to check the textual continuity between the character area 21 and the character area 22, and between the character area 21 and the character area 23. Here, in order to check the continuity of the character areas 21 and 22 first,
These two areas are taken out and the character area 21 is set to the basic character area,
The character area 22 is used as a comparison character area. Here, in the determination of the basic character area, among the plurality of character areas extracted from the input image, when the text of each character area is vertical writing, the upper right is set as the starting point, and the area near the starting point is determined in order. If the text in each character area is written horizontally, the upper left corner is set as the starting point, and the areas near the starting point are sequentially determined. Further, the comparison character area is determined as the lower and left vertical writing character areas when the basic character area is vertical writing, and the lower and right horizontal writing character areas when the basic character area is horizontal writing.

Step S102 Next, the continuity C from the basic character area to the comparative character area is obtained. Various methods for obtaining the continuity C will be described in detail later.

Step S103 The continuity C is compared with the threshold value α.

C ≧ α When this expression is satisfied, the process proceeds to step S104. Also,
If not satisfied, the process proceeds to step S105. However, the threshold value α is 1.0 here.

In the example of FIG. 3, the sleep continuity C is C = 1.0,
Since the formula is satisfied, the process proceeds to step S104.

Step S104 It is determined that "continuous".

In the example of FIG. 3, it is determined to be "continuous".

Step S105 It is determined that "not continuous".

Similarly, when the basic character area is the character area 21 and the comparison character area is the character area 23, the continuity is determined.

In this way, the continuity of all candidate character areas of the comparison character area B is judged, and if there is one character area judged to be "continuous", that character area is continuous with the basic character area. If there are a plurality of character areas that are determined to be “continuous”, the continuity C is selected from among them.
The largest area is determined as a continuous area, and if there is no area that is determined to be "continuous", it is determined that the continuous area group in the basic character area is completed.

This completes the ordering process.

By repeating such a process of determining continuity for each character area while updating the basic character area A, all the plurality of character areas extracted from the input original image (or specified as a processing target) are identified. It is possible to judge the continuity (for all the character areas that are displayed), connect the recognition results of the character groups included in each character area according to the determined order, and display the text as the recognition result of the input original image.

Hereinafter, various examples of how to obtain the continuity C in S103 will be described in detail by taking the case where the basic character area A is the area 21 and the comparative character area B is the area 22 as an example.

FIG. 3 shows a character area 2 of the original image shown in FIG.
It is the figure which showed the example which the last character of 1 is an arrow. In the figure, 31 is an arrow.

Reference numerals 32 and 33 are orthogonal coordinate axes whose origin is aligned with the arrow.

Reference numeral 34 is a point at the upper right corner of the character area 22.

35 is the lower right of the arrow, that is, the coordinate axis 32,
It is the fourth quadrant of the orthogonal coordinate system consisting of 33.

FIG. 4 shows the first step S102.
2 is a detailed flowchart of the example of FIG.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S401, 0.0 is substituted for continuity C for initialization.

Next, in step S402, it is determined whether the last character in the basic character area is an arrow. In the case of an arrow, the process proceeds to step S403. If it is not an arrow, step S102 ends. That is, the continuity remains 0.0 and does not change.

In the example of FIG. 3, the basic character area is the character area 2
It is determined as 1 whether the last character is an arrow. The last letter is
As shown by the arrow 31 in FIG. 3, since the arrow points to the lower right, the process proceeds to step S403.

Next, in step S403, it is determined whether or not there is a comparison character area in the direction of the arrow. If there is a comparison character area in the direction of the arrow, the process proceeds to step S404. If not, step S102 ends. That is, the continuity remains 0.0 and does not change.

In the example of FIG. 3, since the arrow 31 is an arrow pointing to the lower right, the upper right corner of the character area 22 is in the fourth quadrant of the orthogonal coordinate system which is composed of the coordinate axes 32 and 33 and whose origin is aligned with the arrow. Since the point 34 (the upper left corner of the character area in the case of horizontal writing) is included, it is determined that the comparative character area 22 exists in the direction of the arrow, and the process proceeds to step S404.

At step S404, the continuity C is set to 1.0.
To increase continuity.

In the example of FIG. 3, the continuity C is 0.0 to 1.0.
To 1.0.

Thus, step S102 is completed.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character area. However, when a punctuation mark or a period exists at the end of the character area, the character is indented first. The continuity with respect to the character area may be increased.

The method of obtaining the continuity C will be described in detail below.

FIG. 5 shows the character area 2 of the original image shown in FIG.
It is the figure which showed the first or last character regarding 1,22,23. In the figure, 51 is the last character and phrase in the character area 21.

Reference numeral 52 is the first indentation portion of the character area 22.

53 is the "fast" of the first character in the character area 23.

FIG. 9 shows the second step S102.
2 is a detailed flowchart of the example of FIG.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S901, 0.0 is substituted for continuity C for initialization.

Next, in step S902, it is determined whether the last character in the basic character area is a punctuation mark or a period. If it is a punctuation mark or a period, the process proceeds to step S903. If it is neither a punctuation mark nor a period, step S102 ends.

In the example of FIG. 5, the case where the basic character area is the character area 21 and the comparison character area is the character area 22 is explained. Since the end of the basic character area 21 ends at the punctuation mark 51, the process proceeds to step S903. .

In step S903, it is determined whether the first character in the comparison character area is indented. If it is indented, the process proceeds to step S904. If not, step S102 ends.

In the example of FIG. 5, since the first character in the comparison character area 22 is the indentation 52, the process proceeds to step S904.

At step S904, the continuity C is set to 1.0.
To increase continuity.

In the example of FIG. 5, the continuity C is 0.0 to 1.0.
To 1.0.

After that, the threshold value is compared with the threshold value in step S103, and the process proceeds to step S104 to determine that the basic character area 21 and the comparative character area 22 are continuous.

Similarly, the case where the basic character area is the character area 21 and the comparison character area is the character area 23 will be described. In this case, the first character in the comparison character area 23 is the character 53, and the character is not indented. Therefore, it is determined to be NO in step S903, step S102 is terminated, and the process proceeds to step S103.

Since the continuity C remains 0.0, the process proceeds to step S105, and the basic character area 21 and the comparison character area 2
It is determined that 3 is not continuous.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character area. However, when the end of the character area does not end with a punctuation mark or a period, The continuity between the character regions may be obtained by using the certainty of the first sentence of the other character regions to be compared with the sentence as one sentence.

Here, the certainty as a sentence is that when the end of the basic character area ends with a noun, the other character areas to be compared have a greater certainty as a sentence when the first sentence starts with a particle. An example will be described.

The method of obtaining the continuity C will be described in detail below.

FIG. 6 shows a character area 2 of the original image shown in FIG.
It is the figure which showed the first or last character regarding 1,22,23. In the figure, 61 is the last character portion of the character area 21 and the noun "Russia". 62 is a character area 22
Is the first letter and particle "ga". 63 is the first character portion of the character area 23.

FIG. 10 is a detailed flowchart of this example regarding step S102.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S1001, 0.0 is substituted for continuity C for initialization.

Next, in step S1002, it is determined whether the end of the basic character area is a punctuation mark or a period. If it is a punctuation mark or a period, step S102 ends. If it is not a punctuation mark or a period, the process proceeds to step S1003.

In the example of FIG. 6, the basic character area is the character area 2
Since it is 1 and the last is neither a punctuation mark nor a period, the process proceeds to step S1003.

In step S1003, it is determined whether the end of the basic character area ends with a noun. If the last is a noun, the process proceeds to step S1004. If it is not a noun, step S102 ends.

In the example of FIG. 6, since the last of the basic character area 21 is the noun 61 “Russia”, the process proceeds to step S1004.

In step S1004, it is determined whether the beginning of the comparison character area starts with a particle. If it starts with a particle, the process proceeds to step S1005. If it does not start with a particle, step S102 ends.

In the example of FIG. 6, the comparison character area is the character area 2.
In the case of 2, the beginning begins with the particle 62 “ga”, and thus the process proceeds to step S1005.

At step S1005, the continuity C is 1.
Add 0 to increase continuity.

In the example of FIG. 6, the continuity C is 0.0 to 1.0.
To 1.0. As a result of the determination in step S103, the process proceeds to step S104, and it is determined that the character areas 21 and 22 are continuous.

Similarly, in the case where the comparison character area is the character area 23, the beginning is the "eye" of the character 63, and since it is not a particle, step S102 is ended and the continuity C is 0.0.
Therefore, it is determined that the character area 21 and the character area 23 are not continuous.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character area. However, when the end of the character area is not terminated by a punctuation mark or a period, it is compared with the last sentence. The continuity between the character regions may be obtained by using the certainty of the first sentence of the other character region as one sentence.

Here, the certainty as a sentence is that when the last sentence of the character area includes the subject but does not include the predicate, the first sentence of the other character area to be compared does not include the subject and includes the predicate. An example of sometimes increasing the certainty as a sentence will be described.

The method of obtaining the continuity C will be described in detail below.

FIG. 7 shows the character area 2 of the original image shown in FIG.
It is the figure which showed the first or last character regarding 1,22,23. In the figure, 71 is the last sentence of the character area 21, "arrangement of related bill". Reference numeral 72 is the first sentence "delayed." In the character area 22. 73 is the first sentence of the character area 23, "The first country is Russia."

FIG. 11 is a detailed flowchart of step S102.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S1101, 0.0 is substituted for continuity C for initialization.

Next, in step S1102, it is determined whether the end of the basic character area is a punctuation mark or a period. If it is a punctuation mark or a period, step S102 ends. If it is not a punctuation mark or a period, the process proceeds to step S1103.

In the example of FIG. 7, the basic character area is the character area 2
Since the last is neither a punctuation mark nor a period, the process advances to step S1103.

In step S1103, it is determined whether the last sentence of the basic character area includes the subject and does not include the predicate.
If the subject is included and the predicate is not included, step S11
Go to 04. If not, step S102 ends.

In the example of FIG. 7, since the last sentence 71 of the basic character area 21 includes the subject but does not include the predicate, step S
Proceed to 1104.

In step S1104, it is determined whether the first sentence of the comparison character area does not include a subject and a predicate. If the subject is not included and the predicate is included, the process proceeds to step S1105. Otherwise, step S102 ends.

In the example of FIG. 7, the comparison character area is the character area 2
In the case of 2, since the first sentence 72 of the comparison character area 22 does not include the subject but the predicate, the process proceeds to step S1105.

At step S1105, the continuity C is set to 1.0.
To increase continuity.

In the example of FIG. 7, the continuity C is 0.0 to 1.0.
To 1.0. As a result of the determination in step S103, the process proceeds to step S104, and it is determined that the character areas 21 and 22 are continuous.

Similarly, when the comparison character area is the character area 23, the first sentence 73 in step S1104 is
Since the subject is included, step S102 ends. The continuity C becomes 0.0, and as a result of the determination in step S103, the process proceeds to step S105, and the character areas 21 and 23
Is determined not to be continuous.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character area. However, when the end of the character area is not terminated by a punctuation mark or a period, it is compared with the last sentence. The continuity between the character regions may be obtained by using the certainty of the first sentence of the other character region as one sentence.

Here, the certainty as a sentence is that, when the last sentence of the character area includes the subject but does not include the predicate, the first sentence of the other character area to be compared does not include the subject and includes the predicate. An example of finding the certainty as a sentence by using the degree of association between the subject and the predicate when increasing the certainty as a sentence will be described.

The method of obtaining the continuity C will be described in detail below.

FIG. 8 shows the character area 2 of the original image shown in FIG.
It is the figure which showed the first or last character regarding 1,22,23. In the figure, reference numeral 81 is the last sentence of the character area 21, "arrangement of related bill". Reference numeral 82 is the first sentence “delay.” In the character area 22. Reference numeral 83 is the first sentence "run." In the character area 23.

FIG. 12 is a detailed flowchart of step S102.

FIG. 13 is a part of the relevance data of the subject and the predicate.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S1201, 0.0 is substituted for continuity C for initialization.

Next, in step S1202, it is determined whether the end of the basic character area is a punctuation mark or a period. If it is a punctuation mark or a period, step S102 ends. If it is not a punctuation mark or a period, the process advances to step S1203.

In the example of FIG. 8, the basic character area is the character area 2
Since the value is 1 and the last is neither a punctuation mark nor a period, the process advances to step S1203.

In step S1203, it is determined whether the last sentence of the basic character area includes the subject and does not include the predicate.
If the subject is included and the predicate is not included, step S12
Go to 04. If not, step S102 ends.

In the example of FIG. 8, since the last sentence 81 of the basic character area 21 includes the subject but does not include the predicate, step S
Proceed to 1204.

In step S1204, it is determined whether the first sentence of the comparison character area does not include the subject and does not include the predicate. If the subject is not included and the predicate is included, the process proceeds to step S1205. Otherwise, step S102 ends.

In the example of FIG. 8, the comparison character area is the character area 2
In the case of 2, since the first sentence 82 of the comparison character area 22 does not include the subject but the predicate, the process proceeds to step S1205.

In step S1205, the continuity is increased by adding the degree of association between the subject and the predicate to the continuity C.

In the example of FIG. 8, the subject is “maintenance” and the predicate is “delayed”, so it can be seen from the relevance data shown in FIG. 13 that the relevance is 1.25. Therefore, the continuity C becomes 1.25 by adding 1.25 to 0.0. As a result of the determination in step S103, the process proceeds to step S104,
It is determined that the character areas 21 and 22 are continuous.

Similarly, when the comparison character area is set to the character area 23, the subject is “maintenance” and the predicate of the first sentence 83 is “run”. Therefore, those relations are shown in the relation degree data shown in FIG. The degree is not listed. If not listed, the relevance is 0.0, so the continuity C remains at 0.0 with 0.0 added. As a result of the determination in step S103, step S1
In step 05, it is determined that the character area 21 and the character area 23 are not continuous.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character areas. However, by using the existence of a word or synonym commonly existing between the character areas, the character areas are separated from each other. May be required to be continuous.

The method of obtaining the continuity C will be described in detail below.

FIG. 15 is a diagram showing all characters in the character areas 21, 22, and 23 of the original image shown in FIG.

FIG. 16 is a detailed flowchart of step S102.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S1601, 0.0 is substituted for continuity C for initialization.

Next, in step S1602, in the comparison character area, the ratio of the same or synonymous words as the words in the basic character area to the total number of words in the comparison character area is calculated and added to C,
Increase continuity.

In the example of FIG. 15, when the comparison character area is the character area 22, the basic character area 2 is within the comparison character area 22.
Taking out the same or synonymous words as 1 and counting the numbers, there are three "economy", one "reform", and one "former Soviet Union" as a synonym for "Russia". Since "Russia" has two words in the basic character area, when counted as two words, the total is 3 + 1 + 2 = 6.

Since the total number of words in the comparison character area 22 is 23, the ratio is 6 ÷ 23 = 0.26.

0.26 is added to the continuity C to obtain 0.26.

Thus, step S102 is completed.

Next, in step S103, the threshold value α is compared with the continuity C. If it is greater than or equal to the threshold value α, step S104
If it is less than, go to step S105. However, the threshold is 0.20 here.

In the example of FIG. 15, since the continuity C is 0.26, the process proceeds to step S104.

In step S104, it is determined that the basic character area and the comparison character area are continuous.

Similarly, the case where the comparison character area is the character area 23 will be described. When the same word as the word in the basic character area 21 or a synonym is taken out and the number is counted, "economy" is 2
There is one “reform”. Therefore, the total is 2 + 1 =
It becomes 3.

Since the total number of units in the comparison character area 23 is 19, the ratio is 3 ÷ 19 = 0.16.

0.16 is added to the continuity C to give 0.16.

Thus, step S102 is completed.

Next, in step S103, the threshold value α is compared with the continuity C. Continuity C is 0.16, so step S1
Go to 05.

In step S105, it is determined that the basic character area and the comparison character area are not continuous.

In the example shown in the flowchart of FIG. 3, the continuity between the character areas is increased by the arrow at the end of the character areas, but it may be obtained by using the similarity between the character areas.

Here, an example will be described in which the similarity between the character areas is obtained by using the similarity of the sentence expression, particularly the similarity regarding politeness.

The method of obtaining the continuity C will be described in detail below.

FIG. 17 is a detailed flowchart of step S102.

Step S102 will be described with reference to the flowchart of FIG.

First, in step S1701, 0.0 is substituted for continuity C for initialization.

Next, in step S1702, the similarity regarding the politeness of the text representation of the basic character area and the comparison character area is calculated and added to C to increase continuity.

For example, it has a dictionary of respected words, humble words, and polite words, and their ratio X existing in sentences in the basic character area is X.
Then, the ratio Y of those existing in the comparison character area is obtained, and the similarity S between the basic character area and the comparison character area is obtained by the following formula.

S = 1.0- (difference between X and Y) (1) Next, the similarity S is added to the continuity C.

If the continuity C thus obtained is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Considering concretely, the ratio of respected words, humble words, and polite words in the basic character area is 0.25, and the ratio of honorific words, humble words, and polite words in the comparative character area is 0.3, which is similar. The degree S is S = 1.0- (0.3-0.25) = 0.95.

By adding the similarity S to the continuity C, the continuity C becomes C = 0.95.

With the above, step S102 is ended, and the process proceeds to step S103.

In step S103, the value is compared with the threshold value α, and if α or more, it proceeds to step S104. If less than
It proceeds to step S105. However, the threshold value α is 0.8 here.

In this example, since the continuity C is not less than the threshold value, the process proceeds to step S104, and it is determined that the basic character area and the comparison character area are continuous.

The flowchart showing the ordering method of this embodiment is the same as that of the first embodiment shown in FIG.

From the above description, the same operation and effect as those of the above-mentioned first embodiment can be obtained.

In the example shown in the flowchart of FIG. 17, the similarity between the character areas is obtained by using the similarity regarding the politeness of the sentence expression, but the similarity indicates the similarity regarding the line end expression. You may use.

For example, from the sentences in the basic character area and the comparison character area, the proportions in which the end-of-line expression is “masamasu” may be calculated, respectively, and the similarity between the character areas may be calculated by the expression (1).

Next, the similarity S is added to the continuity C.

If the obtained continuity C is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, if the ratio of “Damasuma body” in the basic character area is 0.5 and the ratio of “Damasuma body” in the comparison character area is 0.4, the similarity S is S = 1.0- (0.5-0.4) = 0.9.

By adding the similarity S to the continuity C, the continuity C becomes C = 0.9.

Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.8 here.

In the example shown in the flowchart of FIG. 17, the similarity between the character areas is obtained by using the similarity regarding the politeness of the sentence expression, but the similarity indicates the similarity regarding the treatment expression. You may use.

Here, the treatment expression is a linguistic expression that represents the attitude of a speaker who is a person in the topic, such as respect, dearness, and contempt.

For example, the proportion of the treatment expression may be calculated from the sentences in the basic character area and the comparison character area, and the similarity between the character areas may be calculated by the expression (1).

Next, the similarity S is added to the continuity C.

If the obtained continuity C is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, if the proportion of treatment expressions in the basic character area is 0.2 and the proportion of treatment expressions in the comparison character area is 0.3, the similarity S is S = 1.0- (0 .3-0.2) = 0.9.

By adding the similarity S to the continuity C, the continuity C becomes C = 0.9.

Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.8 here.

The flowchart showing the ordering method of this embodiment is the same as that of the first embodiment shown in FIG.

From the above description, the same operation and effect as in the first embodiment described above can be obtained.

In the example shown in the flowchart of FIG. 17, the similarity between the character areas is obtained by using the similarity regarding the politeness of the sentence expression, but the similarity is the kanji in the character area. It is also possible to use the composition ratio for each genre such as hiragana, katakana, symbols, numbers, and letters.

For example, the ratio of Chinese characters to the total number of characters may be calculated from the sentences in the basic character area and the comparison character area, and the similarity between the character areas may be calculated by the equation (1).

Next, the similarity S is added to the continuity C.

If the continuity C thus obtained is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, if the proportion of Chinese characters in the basic character area is 0.4 and the proportion of Chinese characters in the comparative character area is 0.3, the similarity S is S = 1.0- (0.4 -0.3) = 0.9.

By adding the similarity S to the continuity C, the continuity C becomes C = 0.9.

Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.8 here.

Note that other genres may be used instead of the example in which the degree of similarity is obtained using only the ratio of Chinese characters to the total number of characters as described above. Also, several genres may be combined.

The similarity between the character areas as described above is
Instead of an example in which similarity is calculated using the degree of similarity in politeness, the similarity is calculated from the character image in the character area, such as character size, line length, character pitch, line pitch, and character spacing. It is also possible to obtain it by using the gaps and the gaps between the rows.

For example, from the character images in the basic character area and the comparison character area, the average of the character size is calculated, and X,
If the degree of similarity is Y, the degree of similarity S is S = 1.0- (difference between X and Y) / β (2) Next, the degree of similarity S is added to the continuity C. In Expression (2), β is a constant.

If the continuity C thus obtained is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, if the average character size in the basic character area is 64.5 dots and the average character size in the comparative character area is 59.3 dots, the similarity S is S = 1. 0- (64.5-59.3) ÷ 100 = 0.9
It is 5.

The similarity C is added to the continuity C, and the continuity C becomes C = 0.95.

Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.8 and the constant β is 100 here.

Note that line length, character pitch, line pitch, gaps between characters, and gaps between lines may be used instead of the example in which the similarity between character regions is obtained using the character size as described above. Of course good. Also, some of them may be combined.

The similarity between the character areas as described above is
Instead of an example in which the degree of similarity is calculated by using the degree of politeness in the sentence expression, the degree of similarity may be obtained by using the font difference obtained from the character image in the character area.

For example, from the character images in the basic character area and the comparison character area, the ratio of the number of characters in Mincho typeface to the total number of characters may be obtained, and the similarity between the character areas may be obtained by the equation (1).

Next, the similarity S is added to the continuity C.

If the continuity C thus obtained is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, the ratio of Mincho typeface in the basic character region is 0.9, and the ratio of Mincho typeface in the comparative character region is 0.
When the similarity is 91, the similarity S is S = 1.0- (0.91-0.9) = 0.99.

Adding the similarity S to the continuity C, the continuity C becomes C = 0.99.

Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.9 here.

Note that instead of the example of obtaining the similarity between the character areas using the ratio of the number of characters in Mincho type to the total number of characters as described above, for example, the font used in the basic character area is a Gothic BBB type. Since the font used for the comparison character area is a standard width Gothic font, it is determined that the similarity is 0, that is, the continuity C is C = 0.0, and the basic character area and the comparison character area are not continuous. You may.

Note that instead of the example in which the similarity between character regions is obtained by using the ratio of the number of characters in Mincho type to the total number of characters as in the above example, it goes without saying that Gothic type and textbook type are used instead of Mincho type. The font may be italic, Hosyo, or Taichung.

The similarity between the character areas as described above is
Instead of an example in which the degree of similarity is calculated using the degree of politeness in the text expression, the degree of similarity may be obtained using the difference in the inclination of the character or line obtained from the character image in the character area.

For example, the inclination angles may be respectively obtained from the character images in the basic character area and the comparison character area, and the similarity between the character areas may be obtained by the equation (2).

Next, the calculated similarity S is added to the continuity C.

If the continuity C thus obtained is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

Specifically, if the inclination angle of the basic character area is 0.1 degrees and the inclination angle of the comparison character area is 1.0 degrees, the similarity S is S = 1.0- (1.0-0 .1) /90=0.99.

Adding the similarity S to the continuity C, the continuity C becomes C = 0.99. Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, here, the threshold value α is 0.9 and the constant β is 90.

The similarity between the character areas as described above is
Instead of an example in which the degree of similarity is calculated using the degree of politeness in the text expression, the degree of similarity may be obtained using the difference in the set direction obtained from the character image in the character area.

For example, the ratio of the number of vertically written characters to the total number of characters may be calculated from the character images in the basic character area and the comparison character area, and the degree of similarity between the character areas may be calculated by the equation (1).

Next, the calculated similarity S is added to the continuity C.

If the obtained continuity C is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous.

More specifically, the ratio of vertical writing in the basic character area is 0.9, and the ratio of vertical writing in the comparison character area is 0.
When the similarity is 91, the similarity S is S = 1.0- (0.91-0.9) = 0.99.

By adding the similarity S to the continuity C, the continuity C becomes C = 0.99. Since this is larger than the threshold value α, it is determined that the basic character area and the comparison character area are continuous. However, the threshold value α is 0.9 here.

Note that, instead of the example in which the ratio of the number of vertically written characters to the total number of characters is used to obtain the similarity between the character regions as described above, for example, the set direction of the basic character region is vertical writing, Since the set direction of the areas is horizontal writing, the similarity is 0, that is, the continuity C is C = 0.0, and it may be determined that the basic character area and the comparison character area are not continuous.

Note that the ratio of the number of vertically written lines to the total number of lines may be used instead of the example in which the similarity between character regions is obtained by using the ratio of the number of vertically written characters to the total number of characters as described above. .

(Embodiment 2) In the first embodiment, it is determined whether or not two character areas are consecutive, but in the present embodiment, the continuity of the case where the target area is three or more areas is determined. The determination will be described.

FIG. 14 is a flowchart showing the ordering method of this embodiment.

A method of determining whether or not the character areas are continuous in this embodiment will be described with reference to FIG. Step S102
1 is the same as that of FIG. 1, and the method of determining the continuity C described in detail in the previous embodiment is also used in this embodiment.

First, in step S1401, a basic character area which is a base to be checked for continuity is extracted.

Explaining the example of FIG. 5, the basic character area is the character area 21.

Next, in step S1402, one or more comparison character areas that are continuous or are compared with the basic character area are extracted.

In the example of FIG. 5, the character areas 22 and 23 are comparison character areas.

In step S1403, one comparison character area is extracted.

In the example of FIG. 5, first, the character area 22 is taken out as a comparison character area.

At step S102, the continuity C from the basic character area to the comparative character area is obtained. Details of step S102 are as shown in FIG. 9, and are exactly the same as those shown in the second embodiment.

In the example of FIG. 5, regarding the character area 22,
The continuity C is 1.0.

Next, in step S1404, it is determined whether the continuity C has been obtained for all the comparison character areas. If all the continuity is obtained, the process proceeds to step S1405. If any remain, the process returns to step S1403 and step S140.
One region in which the continuity C of the regions obtained in 2 is not obtained is taken out, and the process for obtaining continuity is continued.

In the example of FIG. 5, since the character area 23 remains, the process returns to step S1403 to continue the processing. When the continuity is calculated similarly to the character area 22, the continuity C is 0.0 as described in the second embodiment. Now that the continuity is obtained for all the comparison character areas, step S
Proceed to 1405.

In step S1405, it is determined that the comparison character area having the largest continuity C is continuous.

In the example of FIG. 5, the continuity C to the comparison character area 22 is 1.0 and the continuity C to the comparison character area 23 is 0.0.
Therefore, it is determined that the character area continuous with the basic character area 21 is the character area 22.

[0250]

As described above, according to the present invention, when determining whether or not two or more character areas are continuous, a character obtained by analyzing a character or a sentence or an image in the character area is determined. By adding the reading order to the text area by using the continuity of the area, even if there are multiple articles in the manuscript such as newspaper articles or if the order cannot be set correctly from the position, the reading order is correct. Can be attached, and the effect of reducing the trouble such as correction can be achieved.

[Brief description of drawings]

FIG. 1 is a flowchart illustrating an ordering process according to a first exemplary embodiment.

FIG. 2 is a diagram showing an example of a document image.

FIG. 3 is an exemplary view when the last character in a character area 21 of the document image shown in FIG. 2 is an arrow.

FIG. 4 is a detailed flowchart of step S102.

FIG. 5 is a character area 21, 22, 2 of the original image shown in FIG.
The figure which showed the first and the last character regarding 3.

FIG. 6 is a character area 21, 22, 2 of the original image shown in FIG.
The figure which showed the first and the last character regarding 3.

FIG. 7 is a diagram showing the character areas 21, 22, and 2 of the original image shown in FIG.
The figure which showed the first and the last character regarding 3.

FIG. 8 is a character area 21, 22, 2 of the original image shown in FIG.
The figure which showed the first and the last character regarding 3.

FIG. 9 is a detailed flowchart of step S102.

FIG. 10 is a detailed flowchart of step S102.

FIG. 11 is a detailed flowchart of step S102.

FIG. 12 is a detailed flowchart of step S102.

FIG. 13 is a diagram showing a part of relevance data of a subject and a predicate.

FIG. 14 is a flowchart showing an ordering process according to the second embodiment.

15 is a diagram showing the character areas 21 and 22 of the original image shown in FIG.
The figure which showed all the characters regarding 23.

FIG. 16 is a detailed flowchart of step S102.

FIG. 17 is a detailed flowchart of step S102.

FIG. 18 is a block diagram showing the configuration of an apparatus according to this embodiment.

Claims (38)

[Claims] 1. A document image is stored, region information relating to at least two character regions existing in the document image is stored, and sentences included in two character regions of the character region are continuous. An image processing method characterized by determining whether or not. 2. The image processing method according to claim 1, wherein the determination as to whether the sentences are continuous is made by analyzing the sentences included in the two character areas. 3. The image processing method according to claim 1, wherein the area information relating to the stored character area is information obtained by separating the stored document image into areas. 4. The analysis of sentences included in the character area is performed by
The image processing method according to claim 2, wherein the image information included in the character area is analyzed for a character obtained by character recognition. 5. The image processing method according to claim 1, wherein whether or not the sentences included in the two character areas are continuous is determined by an index of continuity. 6. The image processing according to claim 5, wherein when the arrow exists at the end of the character area, the continuity index is increased with respect to the character area existing in the direction indicated by the arrow. Method. 7. The continuity index is set to be much larger in a character area that is staggered first when a symbol indicating the end of a sentence is present at the end of the character area. The image processing method described in. 8. The image processing method according to claim 7, wherein the symbol indicating the end of the sentence is a punctuation mark. 9. The image processing method according to claim 7, wherein the symbol indicating the end of the sentence is a period. 10. The index of continuity is determined by using the certainty of the last sentence of the one character region and the first sentence of the other character region as one sentence. Item 6. The image processing method according to Item 5. 11. The image processing method according to claim 10, wherein the continuity index is determined when the end of the character area is not a symbol indicating the end of a sentence. 12. The certainty as one sentence is,
12. The likelihood as one sentence is increased when the first sentence of the other character region starts with a particle when the end of the one character region ends with a noun. Image processing method. 13. The certainty as the one sentence is,
When the last sentence of one character area contains a subject but does not contain a predicate, when the first sentence of the other character area contains no subject and contains a predicate, it increases the certainty as a sentence. The image processing method according to claim 11, which is characterized in that. 14. The certainty as the one sentence is,
The image processing method according to claim 13, wherein the subject included in the one character area and the predicate included in the other character area are used to obtain the object. 15. The determination as to whether the sentence is continuous is made by
The image processing method according to claim 1, wherein the determination is performed by using the existence ratio of words or synonyms that commonly exist in both character areas. 16. The determination as to whether or not the sentence is continuous,
The image processing method according to claim 1, wherein the image area is obtained by using the similarity between the character areas. 17. The image processing method according to claim 16, wherein the similarity between the character areas is determined with respect to a text expression between the character areas. 18. The image processing method according to claim 17, wherein the text expression is polite. 19. The image processing method according to claim 17, wherein the text expression is a line ending expression. 20. The image processing method according to claim 17, wherein the text expression is a treatment expression. 21. The image processing method according to claim 16, wherein the similarity between the character areas is determined based on a composition ratio for each genre in the character area. 22. The image processing method according to claim 21, wherein the genre is Chinese characters. 23. The image processing method according to claim 21, wherein the genre is hiragana. 24. The image processing method according to claim 21, wherein the genre is katakana. 25. The image processing method according to claim 21, wherein the genre is a symbol. 26. The image processing method according to claim 21, wherein the genre is a number. 27. The image processing method according to claim 21, wherein the genre is an alphabetic character. 28. The similarity between the character areas is determined based on a format in the character area.
6. The image processing method according to item 6. 29. The image processing method according to claim 28, wherein the format is a font. 30. The image processing method according to claim 28, wherein the format is a character size. 31. The image processing method according to claim 28, wherein the format is a line length. 32. The image processing method according to claim 28, wherein the format is a character pitch. 33. The image processing method according to claim 28, wherein the format is a line pitch. 34. The image processing method according to claim 28, wherein the format is an inclination of a character. 35. The image processing method according to claim 28, wherein the format is line inclination. 36. The image processing method according to claim 28, wherein the format is a space between characters. 37. The image processing method according to claim 28, wherein the format is a space between lines. 38. The image processing method according to claim 28, wherein the format is a set direction.