JP3278221B2 - Image processing method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method capable of inputting image information of an OCR (optical character recognition device), a copying machine, a facsimile or the like, and dividing the image information into regions for each attribute. It concerns the device.

[0002]

2. Description of the Related Art Conventionally, when a plurality of headings exist in image information, the order in which the document is sequentially arranged from right to left for a vertically written document and from top to bottom for a horizontally written document is as follows. there were.

Conventionally, in an area dividing process of an input image,
The classification of the table and other regions was simply obtained from the ratio of the area of the circumscribed rectangle to the number of pixels.

[0004]

However, according to the above-described conventional technique of heading ordering, in the case of a manuscript having a heading surrounded by the main body such as a newspaper article, the heading is recognized in the main body. As a result, there is a problem that, unlike the original order, the flow of text in the text is disturbed.

[0005]

[0006]

According to a first aspect of the present invention, there is provided an image processing method, comprising: extracting a radiation component from a predetermined point in a region divided for each attribute in input image information; Stretching, detecting related information between the radiation component and the input image information, and determining the order of the area including the predetermined point with respect to the other divided area according to the detected related information. It is characterized by the following.

According to another aspect of the present invention, there is provided an image processing apparatus comprising: a line segment radiating unit for extending a radiation segment from a predetermined point in a region divided for each attribute in input image information; A related information detecting means for detecting related information between the radiation component and the input image information, and, according to the detected related information, an area including the predetermined point with respect to the other divided area. Region order determining means for determining the order.

[0008]

[0009]

[0010]

【Example】

(Embodiment 1) FIG. 1 is a block diagram showing a configuration of an image processing apparatus of the present embodiment.

In FIG. 1, reference numeral 101 denotes an image data input unit. Reference numeral 102 denotes an arithmetic processing unit (CPU) that controls the apparatus and calculates each unit according to the control program stored in the storage unit 103. A storage unit 103 stores a control program for processing shown in a flowchart described later and various data, and includes a ROM and a RAM. 104
Reference numeral denotes an output unit that outputs a calculation processing result, an image processing result, and image data. An area dividing unit 105 performs the entire area dividing process. An image thinning unit 106 thins out an input image. Reference numeral 107 denotes a labeling unit that labels the thinned pixels and simultaneously creates initial rectangular data. 1
Reference numeral 08 denotes an attribute detection unit that detects attributes in the area, such as a separator, a table, and a graphic. A combination direction detection unit 109 detects a composition direction of a sentence such as vertical writing or horizontal writing. 1
Reference numeral 10 denotes a headings detection unit that detects headings. 11
Reference numeral 1 denotes an ordering unit for ordering paragraphs and headings of a sentence.

FIG. 2 is a flowchart showing image processing in the image processing apparatus of the present embodiment. A control program for executing this processing is stored in the storage unit 103, and the arithmetic processing unit 1
02.

Step S201 First, an original image is input from the image input unit 101.

Step S202 Next, in the image thinning section 106, the vertical
A logical sum of dots and horizontal n dots is taken to newly thin out m × n pixels to one pixel.

Here, if even one dot has a black pixel in the m × n image of the original image, the thinned image becomes black.

Step S203 Next, in the labeling unit 107, a label is added to the black pixels of the thinned image one line at a time, and the same label is assigned to pixels that are continuous vertically, horizontally, and diagonally, and at the same time, a rectangle is formed. Perform the following processing.

Referring to FIG. 3 as an example, the first detected pixel A is labeled 1, and the coordinates (Xa, Xa,
Assuming that Ya) is the start point and end point of the rectangle, the number of pixels is 1, and the same label 1 as the pixel is added to the rectangle label for distinguishing the rectangle, and the above data is stored in the storage unit 103 as rectangular data (FIG. 4). Remember.

Next, a label 2 is assigned to a pixel B having no continuous pixels in the left direction (of course, there is no continuous pixel from the top because it is the first row), and the coordinates (Xb, Yb) of this pixel B are rectangular. , The number of pixels is 1, and the same label 2 as a pixel is added to a rectangular label for distinguishing a rectangle, and the above data is stored in the storage unit 103 as rectangular data (FIG. 4).

When the labeling of the first line is completed as described above, the process moves to the second line.

The first pixel C of the second row is the pixel A of label 1
And the same pixel label 1 as the pixel A is added, and the number of pixels is added to the rectangular data of the rectangular label 1 by adding 1 of the pixel C to a total of 2 pixels, and the rectangular label remains 1 In the rectangular coordinates, only the end point is updated from (Xa, Ya) to (Xa, Yc) (the coordinates of the start point do not change).

Since the next pixel D is continuous with the pixel C from the left, the same label 1 as that of the pixel C is added, and the number of pixels is added to the rectangular data of the rectangular label 1 by 1 to make a total of 3 pixels, and the rectangular label becomes The rectangular coordinates are updated at (Xa, Yc) from (Xa, Yc) to (Xd, Yc) with the rectangular coordinates unchanged (the Y coordinate of the end point does not change).

At this time, the pixel D is also obliquely continuous with the pixel B and continues from the pixel C. Therefore, the label of the pixel B is changed from the label 2 to the same label 1 as the pixel D, and the rectangular label 1 is changed. The number of pixels of the rectangle label 2 is added to the rectangle data of 4 to make a total of 4 pixels, the rectangle label remains unchanged, and only the end point of the rectangle coordinates includes all the pixels A, B, C, and D (( Xd, Yc) is updated to (Xb, Yd). The rectangular data of the rectangular label 2 is invalidated by setting the rectangular label to 0.

When the second line is completed as described above, the process moves to the third line.

Since the first pixel E in the third row is obliquely continuous with the pixel C, the same pixel label 1 as that of the pixel C is added.
The pixel becomes a pixel, the rectangle label remains unchanged, and the rectangular coordinates are updated from (Xa, Ya) to (Xe, Ya) at the start point and from (Xb, Yd) to (Xb, Ye) at the end point.

In the same manner, labeling and rectangular shaping are performed on all thinned pixels.

Step S204 Next, in the attribute detecting unit 108, after the labeling and the rectification of the rectangle, the attributes of each rectangle such as a rectangle corresponding to the text, a rectangle corresponding to a figure or a photograph, a table, and a rectangle corresponding to a separator are determined. , The width W of the rectangle, the height H, the area S, the number of pixels with respect to the area, that is, the pixel density D (these are easily obtained by calculating using the number of pixels of the rectangular data, the start point, the end point, and the coordinates). Distinguish.

The details of the process of detecting the attribute in step S204 are shown in the flowchart of FIG. 5 and will be described here.

When the rectangular search is completed in the labeling process (step S203) (step S502), the process proceeds to the following process using each data. The width W of the separator is equal to or less than the threshold Tw1 and the height H is equal to or more than twice the threshold Tw of the width W (step S503), or the width W is greater than the threshold Tw1 and the height H is equal to or more than the threshold Tw3 of the width W (step S504). ), The rectangular label is unified as -3 as the vertically long separator, and the pixel labels constituting the rectangle are held as they are separately from the rectangular label (step S51).
4).

In the case where the width and height are exchanged, the rectangular label is changed to -3 as a horizontally long separator in the same manner as above (steps S505 and S506).

Next, as shown in FIG.
If it is d1 or less (step S507), the rectangular label is changed to -3 (step S514) by regarding it as a key-shaped deformable separator.

When the area S is larger than the threshold value Ts1 (step S508), if the pixel density D is smaller than the threshold value Td2 (step S509 in FIG. 5), it is regarded as a table and the rectangular label is changed to -4 (step S515). , Pixel density D
Is greater than or equal to the threshold value Td2, the rectangular label is changed to -5 assuming that it is a figure or a photograph (step S516 in FIG. 5).

When the area S is equal to or smaller than the threshold value Ts1 and equal to or larger than the threshold value Ts2 (step S510), the pixel density D is equal to or larger than the threshold value Td3 (step S511), and the width W and the height H are equal to or larger than the threshold value Tw4. And the pixel density D is Td5
At this time (step S513), the rectangle is regarded as a figure or a photograph, and the rectangle label is changed to -5.

Further, when the area S is equal to or smaller than the threshold value Ts1 and equal to or larger than the threshold value Ts2 (step S510), a rectangle whose pixel density D is smaller than the threshold value Td4 (step S512) is regarded as a table, and the rectangle label is changed to -4 (step S510). S515).

As described above, a rectangle corresponding to a figure, a photograph, a table, or the like, a rectangle corresponding to a separator, or the like is detected.
With the remaining rectangle as the text, the rectangle label is made the same as the pixel label as it is (step S517).

Step S205 Next, in the combination direction detecting unit 109, in the case of a horizontally written sentence,
In the rectangle remaining as the text, pixels thinned out in the horizontal direction tend to be a horizontally long rectangle, and in the case of a vertically written text, the rectangle remaining as the text tends to be vertically long.

Then, the average value of the width and height of the rectangle of the text is calculated (step S701 in FIG. 7). If the average width w is larger than the average height h, it is regarded as a chapter with a lot of horizontal writing and the average height h
Is the character size of one character (step S70 in FIG. 7).
2. Step S703). On the other hand, if the average height h is larger than the average width w, it is regarded as a sentence with much vertical writing and the average width w is set to 1
The character size is set to the character size (steps S702 and S710 in FIG. 7).

Next, a histogram is taken in the direction opposite to the set direction (steps S704 and S711 in FIG. 7),
Based on the shape of the marginal distribution, a place that is equal to or smaller than the threshold value Tk is set as a paragraph break (steps S705 and 8, FIG. S712 in FIG. 7), and a projection is taken for each paragraph in the same direction as the grouping direction (step S708 in FIG. S715), the length of the continuation of the black pixels is determined as the character size of the character in the paragraph, and the length of the continuation of the white pixels is detected as the line spacing from the shape of the peripheral distribution (steps S709 and S71 in FIG. 7).
6).

The horizontal writing text or the vertical writing text is stored in the storage unit 103.

Step S206 Next, the headings detection unit 110 detects headings based on the set direction and the character size.

Taking the case of vertical writing as shown in FIG. 9 as an example, a rectangle in which the width W of the rectangle is larger than the character size w by a threshold value Tm1 or more from the body rectangle is detected as a candidate for a heading rectangle. Nine steps S905).

Further, the height H of the rectangle is Tm of the character size w.
If it is smaller than twice (step S906 in FIG. 9),
It is assumed that the horizontally written characters are continuous, and is set as a heading (step S908 in FIG. 9).

In addition, since the rectangles which are headed based on the size of the rectangles as described above may include those in which the characters of the body are combined, the distance between the body rectangle and the heading rectangle is equal to the threshold T1. Closer one (step S907 in FIG. 9)
Now, the headline is corrected to the text (step S90 in FIG. 9).
9).

Further, among the text rectangles remaining after the above-described processing, a rectangle having no text and a heading class within the range of the character size w is set as an isolated rectangle as a heading (steps S910 to S913 in FIG. 9).

Step S207 Next, in the ordering section 111, headings and body paragraphs are ordered.

Hereinafter, the ordering will be described in detail. FIG. 10 is a detailed flowchart of step S207. First, it is checked whether or not there is a headline surrounded by the text, and if there is, the order is assigned first (step S1001). Step S1001 will be described later in detail.

Next, it is determined whether the sentence is a vertical writing sentence or a horizontal writing sentence (step S1002). This uses the result of the above-described group direction detection (step S205). In the case of vertical writing, headings / paragraphs located at substantially the same position in the horizontal direction as the topmost region among unordered headings / paragraphs are collected into the same group (step S1003). When the headings / paragraphs belonging to the same height are determined, the headings / paragraphs are arranged in order from the right in the headings / paragraphs (step S1004).
After all headings / paragraphs are ordered in this way,
After step S207, the process proceeds to step S208 (step S1005). If there is any headline / paragraph that has not been ordered yet, the procedure returns to step S1003 and the order is similarly set (step S1005).

Similarly, if it is determined in step S1002 that horizontal writing is to be performed, heading / paragraphs located at substantially the same position in the vertical direction as the leftmost region among unordered heading / paragraphs are collected and the same. The group is set (step S1006). When the headings / paragraphs belonging to the same group are determined, the headings / paragraphs are arranged in order from the top and are ordered (step S1007). When all headings and paragraphs are ordered in this way, step S207 is terminated,
The process proceeds to step S208 (step S1008). If there is a headline / paragraph not yet ordered, step S
Returning to step 1006, the order is similarly set (step S
1008).

Here, a description will be given using a practical example. FIG. 14 shows an input document 1401. FIG. 15 shows a state in which the headline / text paragraph is extracted from the input document 1401 by the area dividing means (steps S202 to S206). In FIG. 15, reference numeral 1501 denotes the entire heading / body paragraph corresponding to the input document 1401. Also, 1502 and 1503 indicated by solid lines indicate headings, and 1504 to 1507 indicated by dotted lines indicate body paragraphs. FIG. 16 shows a case where the heading 150
No. 3 is determined to be surrounded by the text, and the first order is shown.

Next, vertical writing is determined in step S1002, and the flow advances to step S1003. In step S1003, the top one is extracted from among the unordered headings and body paragraphs. In FIG. 16, the main text paragraph 1504 is located at the top. Next, a heading / body paragraph belonging to the same group as the body paragraph 1504 is extracted. First, paragraph 1504 and heading 150
It is determined whether 2 belongs to the same group. FIG. 19 illustrates this. In FIG. 19, H1502, H1
504 indicates the height of the heading 1502 and the main body paragraph 1504, respectively, and Ho indicates the main body paragraph 1504 and the heading 1502.
In the horizontal direction. In this example, Ho =
H1502. Based on these, the same group determination is performed as follows. When (horizontal overlapping length of two regions) ÷ (height of region having smaller height of two regions)> (threshold value γ), the two regions belong to the same group. However, the threshold value γ is a value of 0.0 to 1.0. Here γ =
0.8.

In the example shown in FIG. 19, since Ho ÷ H1502> γ, the main text paragraph 1504 and the heading 1502 are determined to be the same group. Similarly, when the main body paragraph 1504 and the main body paragraphs 1505 to 1507 are determined, it is found that they are not in the same group, and the step S1003 ends.

Next, in step S1004, step S1
003 The heading 1502 and the body paragraph 1504 determined to be in the same group are sequentially ordered from the right. By comparing the points in the upper right corner, the heading 1502 on the rightmost side is numbered 2, and then the main paragraph 1504 is numbered 3. The comparison is not limited to the point at the upper right corner, but may be the center of gravity or the like, as long as it can be determined which area is on the right. FIG. 17 shows the order in which the heading 1502 is numbered 2 and the body paragraph 1504 is numbered 3 from the right in step S1004. Since the body paragraphs 1505 to 1507 are not ordered, the process returns to step S1003 in step S1005 to continue the processing.

In step S1003, paragraph 1505 of the main text
1507 belong to the same group, and in step S1004, the body paragraph 1505 is numbered 4, the body paragraph 1506 is numbered 5, and the body paragraph 1507 is numbered 6 in order from the right.

Since all headings and body paragraphs have been ordered, the flow advances from step S1005 to the next step S208. FIG. 18 shows a state in which step S207 is completed and all headings and body paragraphs are ordered.

Next, step S1001 will be described in detail. FIG. 11 is a flowchart showing the processing in step S1001.

First, in step S1101, attention is paid to one heading area, and its center of gravity is determined. Input manuscript 1
In the example of 401, as shown in FIG. 15, there are two heading areas, heading 1502 and heading 1503. here,
First, look at the heading 1503. Since the region is rectangular, the center of gravity is at the point where the width and height are each divided into two equal parts. This is shown as a center of gravity 2001 in FIG.

Next, in step S1102, the center of gravity 2001
Extend the line segment from top to bottom and right and left, and check whether each line intersects with the main paragraph. In FIG. 20, the center of gravity 200
A line segment extending upward from 1 is indicated by a line segment 2002. Similarly, the downward direction is line segment 2003, and the leftward direction is line segment 200.
4. The right direction is indicated by a line segment 2005. As is apparent from FIG. 20, the line segment 2002 intersects the paragraph 1504 of the main text.
Similarly, the line segment 2003 is composed of the main text paragraph 1506 and the line segment 200.
5 is body paragraph 1505, and line segment 2004 is body paragraph 15.
07.

In step S1103, since the line segment extended from the center of gravity intersects the main text paragraph in at least three directions, up, down, left, and right, the flow advances to step S1104. If the line segment extending from the center of gravity only intersects with the main text paragraph in two directions or less in the upper, lower, left and right directions, the process proceeds to step S1105.

In step S1104, an order is assigned to the headline of interest. Here, since no heading has yet been assigned, the heading 1503 is assigned the first order.

Next, in step S1105, it is determined whether all headings have been checked. If headings remain,
Returning to step S1101, the same processing is performed. After checking all the headings, the process proceeds to step S1002. In the example of the input document 1401, since the headline 1502 has not been checked yet, the process returns to step S1101.

The center of gravity is obtained in exactly the same way as in the case of the heading 1503 (step S1101), and a line segment is extended from the center of gravity in the up, down, left, and right directions, and it is checked whether each line intersects with the main paragraph (step S1102). As is clear from FIG. 15, since there is no body paragraph above or to the right of the heading 1502, the upward and rightward line segments extending from the center of gravity of the heading 1502 do not intersect with the body paragraph. Therefore, the process proceeds from step S1103 to step S1105. That is,
Headings 1502 are not ordered. Step S110
In step 5, all the headlines are checked, so that all the processing in step S1001 is completed and step S100
Proceed to 2. FIG. 16 shows the results up to this stage.

As described above, all headings and body paragraphs are ordered in step S207. FIG. 18 shows an example of the input document 1401 that has been completed up to step S207.

Step S208 Finally, the output unit 104 outputs the rectangular data of the various regions obtained as described above together with the image data.

FIG. 12 shows that in the ordering processing of the headings surrounded by the body, the center of gravity is determined by determining whether the above-described example is the heading surrounded by the body (flowchart of FIG. 11).
It is a flowchart which shows the process which used the arbitrary point in an attention heading instead of using 001. Hereinafter, description will be made based on the flowchart shown in FIG.

First, in step S1201, attention is paid to one heading area, and an arbitrary point within the attention heading is obtained. Here, a point located α% of the height and β% of the width from the upper left corner is determined. Where α is 0 to 10
0%, β takes a value of 0 to 100%, and is determined in advance in RAM
Holding. In the example of the input document 1401, as shown in FIG. 15, there are two heading areas of a heading 1502 and a heading 1503. Here, the heading 1503 is first focused. An arbitrary point in the headline 1503 is set as a point of interest. Here, an arbitrary point, that is, α% of the height from the upper left corner,
I explained how to find the point located at β% of the width,
FIG. In FIG. 21, H is the height of the heading 1503, Hα is α% of the height from the upper left corner, and W is the heading 153.
03, and Wβ represents β% of the width from the upper left corner.
In this way, the point of interest 2101 is determined. Here, α
Is 40% and β is 25%.

Next, in step S1202, the point of interest 210
Extend lines from top to bottom, left and right, and check whether each line intersects the main text paragraph. In FIG. 22, attention point 2
A line segment extending upward from 101 is indicated by a line segment 2201. Similarly, the downward direction is line segment 2202, and the left direction is line segment 2
203, the right direction is indicated by a line segment 2204. As is apparent from FIG. 22, the line segment 2201 intersects with the paragraph 1504 of the main text. Similarly, the line segment 2202 is composed of the main text paragraph 1506 and the line segment 2.
204 intersects with the main text paragraph 1505, and the line segment 2203 intersects with the main text paragraph 1507.

In step S1203, the attention point 2
Since the line segment extended from 101 intersects with the main text paragraph in three or more directions of up, down, left, and right, the process proceeds to step S1204. If the line segment extending from the point of interest 2101 is 2
If only the direction or less intersects the body paragraph, step S1
Proceed to 205.

In step S1204, the order is assigned to the headline of interest. Here, since no heading has yet been assigned, the heading 1503 is assigned the first order.

Next, it is determined in step S1205 whether all headings have been checked. If a heading remains, the process returns to step S1201 to perform the same processing. After checking all headings, the process proceeds to S1002. In the example of the input document 1401, since the headline 1502 has not been checked yet, the process returns to step S1201.

A point of interest is obtained in exactly the same way as in the case of the heading 1503 (step S1201), and a line segment is extended from the point of interest in the up, down, left, and right directions to check whether each intersects a main paragraph (step S1202). As is clear from FIG. 15, since there is no body paragraph above or to the right of the heading 1502, the upward and rightward line segments extending from the center of gravity of the heading 1502 do not intersect with the body paragraph. Therefore, the process proceeds from step S1203 to step S1205. That is, the heading 1502 is not ordered. Step S1
In step 205, all the headlines are checked, so that all the processing in step S1001 ends, and step S1
Proceed to 002. FIG. 1 shows the results up to this stage.
6.

As described above, in step S207, all headings and body paragraphs are ordered. In the example of FIG. 14, FIG.
It is.

Steps other than step S1001 are the same as in the first embodiment. The configuration of the image processing apparatus according to the second embodiment is the same as that of the first embodiment shown in FIG. According to the above description, the same operation and effect as those of the first embodiment can be obtained.

FIG. 13 is a flowchart showing step S1001 in FIG.
To the ordering processing of the headings surrounded by the text, the processing contents of steps S1304 and S1305 are further added.
10 is a flowchart showing a process of examining the degree of overlap of the noted heading with the upper, lower, left, and right body paragraphs, and ordering the noted heading if there is a body paragraph in which the degree of overlap is equal to or greater than a threshold value in three or more directions. .

The following is a description based on the flowchart shown in FIG.

First, in step S1301, attention is paid to one heading area, and its center of gravity is obtained. Input manuscript 1
In the example of 401, as shown in FIG. 15, there are two heading areas, heading 1502 and heading 1503. here,
First, look at the heading 1503. Since the region is rectangular, the center of gravity is at the point where the width and height are each divided into two equal parts. This is shown as a center of gravity 2001 in FIG.

Next, in step S1302, the center of gravity 2001
Extend the line segment from top to bottom and right and left, and check whether each line intersects with the main paragraph. In FIG. 20, the center of gravity 200
A line segment extending upward from 1 is indicated by a line segment 2002. Similarly, the downward direction is line segment 2003, and the leftward direction is line segment 200.
4. The right direction is indicated by a line segment 2005. As is apparent from FIG. 20, the line segment 2002 intersects the paragraph 1504 of the main text.
Similarly, the line segment 2003 is composed of the main text paragraph 1506 and the line segment 200.
5 is body paragraph 1505, and line segment 2004 is body paragraph 15.
07.

In step S1303, since the line segment extended from the center of gravity intersects the main text paragraph in at least three directions, up, down, left, and right, the flow advances to step S1304. If the line segment extending from the center of gravity intersects the main text paragraph only in two directions or less in the up, down, left, and right directions, the process proceeds to step S1307.

In step S1304, for the text paragraphs positioned above and below the heading, the degree of overlap with the heading in the vertical direction is checked, and similarly, for the text paragraph positioned horizontally, the degree of overlap with the heading in the horizontal direction is determined. Find out. When the degree of overlap with the main text paragraph 1504 above the heading 1503 is examined, as shown in FIG.
The overlap between the width W1503 of the third paragraph and the width W1504 of the main paragraph 1504 is indicated by Wo. In this example, Wo = W1
503. In step S1305, based on these, the degree of overlap is determined as follows. When (the vertical overlapping length of the two regions) ÷ (the width of the smaller region of the two regions)> (threshold γ), the degree of overlap is equal to or greater than the threshold. However,
The threshold value γ is a value between 0.0 and 1.0. Here, γ = 0.
8 is assumed.

In the example shown in FIG. 23, since Wo ÷ W1503> γ, the overlap between the main paragraph 1504 and the heading 1503 is equal to or larger than the threshold.

Similarly, the heading 1503 and the main paragraph 1506 below the heading 1503 have an overlapping degree equal to or larger than the threshold. Similarly, when the degree of overlap with the main text paragraph 1507 on the left of the heading 1503 is checked, as shown in FIG. 24, the height of the heading 1503 is H1503, and the height of the main text paragraph 1507 is H150.
7. The overlap between heading 1503 and body paragraph 1507 is H
Indicated by o. In this example, Ho = H1503. In step S1305, based on these, the degree of overlap is determined as follows. When (the overlapping length of the two regions in the horizontal direction) ÷ (the height of the region having the smaller height of the two regions)> (threshold value γ), the degree of overlap is equal to or greater than the threshold value. However,
The threshold value γ is a value between 0.0 and 1.0. Here, it is set to 0.8.

In the example shown in FIG. 24, Ho ÷ H1503> γ, so that the main paragraph 1507 and the heading 1503 have an overlap degree equal to or greater than the threshold. Similarly, the heading 1503 and the text paragraph 1505 to the right of the heading 1503 have a degree of overlap equal to or greater than the threshold.

As described above, in the example of FIG. 14, there are text paragraphs in which the degree of overlap is greater than or equal to the threshold value in three or more directions.
Proceed to step S1306.

In step S1306, the order is assigned to the headline of interest. Here, since no order has been assigned to any of the headings, the first order is assigned to the heading 150.
Attach to 3. Next, it is determined whether all headings have been checked. If a heading remains, the process returns to S1301 to perform the same processing. After checking all headings, S1
Proceed to 002.

In the example of the input document 1401, the heading 150
2 has not yet been checked, so step S1301
Return to The center of gravity is obtained in exactly the same way as in the case of the heading 1503 (step S1301), and a line segment is extended from the center of gravity in the up, down, left, and right directions, and it is checked whether each line intersects with the main text paragraph (step S1302). As is clear from FIG. 15, there is no body paragraph above or to the right of the heading 1502, and therefore the upward and rightward line segments extending from the center of gravity of the heading 1502 do not intersect with the body paragraph. Therefore, step S1
The process proceeds from step 303 to step S1307. That is, the heading 1502 is not ordered.

In step S1307, all headlines have been checked, so that all the processing in step S1001 ends, and the flow advances to step S1002. FIG. 16 shows the results up to this paragraph.

As described above, in step S207, all headings and body paragraphs are ordered. In the example of FIG. 14, FIG.
It is.

(Embodiment 2) In this embodiment, an example will be described in which radiation is extracted from a rectangle whose attribute is determined to be a table and a table area is determined from the number of intersections with a table frame. The configuration of the device of the present embodiment is the same as that of the first embodiment.

FIG. 25 is a flowchart showing image processing in the image processing apparatus of this embodiment. A control program for executing this processing is stored in the storage unit 103.
In the flowchart, the same processing steps as those in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

Step S209 Next, the table area determination section 112 determines the table area. FIG. 28 is a flowchart illustrating in detail the table area determination processing in step S209.

FIG. 26 shows step S209 in the case of a table.
2601 is a rectangular area, 2602 is a table frame, 2603 is a character in the contents of the table, etc., 2604 is the center of gravity (point of interest) of the rectangular area 2601, and 2605 to 2608 extend vertically, horizontally, and respectively from the point of interest 2604. Represents a line segment.

FIG. 27 is a view for explaining step S209 in the case of a deformed separator.
Reference numeral 2702 denotes a deformed separator, 2703 denotes characters in the contents of the deformed separator, etc., 2704 denotes the center of gravity (point of interest) of the rectangular area 2701, and 2705 to 2708 denote the points of interest 27, respectively.
It represents a line segment extending from top, bottom, left, and right.

The case of a table will be described below with reference to the flowchart of FIG. 28 and FIG.

In this embodiment, the center of gravity is set at an arbitrary point of interest in a rectangular area determined to be a table, and a total of four line segments are extended one by one in the vertical and horizontal directions. If there is one (or more) point of interest such that at least one of the line segments extending vertically and horizontally intersects the table frame three or more times, the rectangular area is determined to be a table, and so on. If not, a table determination means for determining the deformation separator is used.

First, in step S2801, the center of gravity of a rectangular area is obtained and set as a point of interest.

The point where the width and height are each halved is the center of gravity, and the point of interest 2604 is obtained.

Next, in step S2802, the point of interest 260
Lines are extended in the vertical and horizontal directions from 4, and the number of times each intersects the table frame is counted. In FIG. 26, attention point 26
A line segment extending upward from 04 is a line segment 2605, similarly, a line segment 2606 is on the lower side, a line segment 2607 is on the left, and a line segment 2608 is on the right.

The number of crossings with the table frame is 26
05 three times, line segment 2606 three times, line segment 2607 two
Times, the line segment 2608 is once.

In step S2803, it is checked whether or not there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally. Since the line segments 2605 and 2606 intersect the table frame three or more times, the flow advances to step S2804. If there is no line segment that crosses three or more times, the process proceeds to step S2805.

In step S2804, the rectangular area is determined as the table area, and the flow advances to step S210.

Next, the flowchart of FIG.
Will be described with reference to FIG.

First, in step S2801, the center of gravity of the rectangular area is obtained and set as a point of interest. The point where the width and height are each halved is the center of gravity, and an attention point 2704 is obtained.

Next, in step S2802, the point of interest 270
The line segment is extended in the up, down, left, and right directions from 4, and how many times each intersects with the front frame (deformed separator) is counted. In FIG. 27, a line segment extending upward from the point of interest 2704 is a line segment 2
705, similarly, a line segment 2706 on the lower side, a line segment 2707 on the left side,
The right is a line segment 2708.

The number of crossings with the table frame is 27
05 once, line 2706 once, line 2707 1
Times, the line segment 2708 is one time.

In step S2803, it is determined whether or not there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally. Since no line segment intersects the table frame three or more times, the process proceeds to step S2805.

In step S2805, the rectangular area is determined as a deformed separator area, and the flow advances to step S205.

Step S210 Finally, the rectangular data of the various regions obtained as described above is output from the output unit 104 together with the image data.

FIG. 31 is a flowchart describing in detail an example in which a plurality of points of interest are taken in a rectangular area in the table area determination processing in step S209.

FIG. 29 shows a step S20 for a table.
9, 2901 is a rectangular area, 2902 is a table frame, 2903 is a character in the table, etc., 2904 to 2906
Denotes an arbitrary point of interest in the rectangular area, and 2907 to 2912 denote lines of interest extending from the point of interest 2904, the point of interest 2905, and the point of interest 2906 to the top, bottom, left and right.

FIG. 30 is a view for explaining step S209 in the case of a deformed separator.
Reference numeral 3002 denotes a deformed separator, 3003 denotes a character or the like in the deformed separator, 3004 to 3006 denote an arbitrary point of interest in a rectangular area, 3007 to 3012 denote points of interest 3004, a point of interest 3005, and lines extending vertically and horizontally from the point of interest 3006 Represents minutes.

The case of a table will be described below with reference to the flowchart of FIG. 31 and FIG.

In the present embodiment, three arbitrary points of interest in the rectangular area determined to be a table are taken, and a total of 12 points are considered for each point of interest. I will stretch them one by one. If there are two or more points of interest such that at least one of the line segments extending vertically and horizontally intersects the table frame three or more times, the rectangular area is determined to be a table,
If it is not the case, it is defined as a table determination means for determining the deformation separator.

First, in step S3101, three points of interest are obtained in a rectangular area. From the point at the upper left corner, the point where the width and height are each reduced to そ れ ぞ れ, the point where each is reduced to 、, and the point where each is reduced to ４ are noted points 2904, 2905, and 2906. I do.

Next, in step S3102, (table area fixed attention point) = 0 is set.

In step S3103, a line segment is extended in the up, down, left, and right directions from the point of interest 2904, and how many times each intersects the table frame is counted. In FIG. 29, the attention point 2904
A line segment extending up and down from line 2907 is a line segment and a line segment extending right and left is line segment 2910.

The number of crossings with the table frame is two for the upward line segment, four for the downward line segment, two for the leftward line segment, and one for the rightward line segment.

In step S3104, it is determined whether or not there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally. Since the downward line segment intersects the table frame three or more times, the process proceeds to step S3105. If no line segment intersects three or more times, the process proceeds to step S3106.

In step S3105, 1 is added to the number of table region fixed attention points, and the flow advances to step S3106. (Table area fixed attention points) = 0 + 1 = 1

In step S3106, it is checked whether all attention points have been processed. Attention point 2905, Attention point 2
Since 906 remains, the process returns to step S3103.
Attention point 2905, attention point 2906 and attention point 29
Processing is performed in the same manner as in step S04. As is clear from FIG. 29, when the processing is completed for all the points of interest, (table area fixed number of points of interest) = 3. Since the processing has been completed for all the points of interest, the flow advances to step S3107.

In step S3107, it is checked whether or not (table region fixed attention point) is 2 or more. Since it is 2 or more, the process proceeds to step S3108.

In step S3108, the rectangular area is determined as the table area, and the flow advances to step S205.

Next, the flowchart of FIG. 31 and the flowchart of FIG.
Will be described with reference to FIG.

First, in step S3101, three points of interest are obtained in a rectangular area. From the point at the upper left corner, the point where the width and the height are respectively reduced to そ れ ぞ れ, the point where the width is reduced to 点, and the point where the height is reduced to ４ are noted points 3004, 3005, and 3006. I do.

Next, in step S3102, (table area fixed point of interest) = 0 is set.

In step S3103, a line segment is extended from the point of interest 3004 in the up, down, left, and right directions, and the number of times each intersects the table frame is counted. In FIG. 31, attention point 3004
A line segment extending upward and downward is a line segment 3007, and a line segment extending left and right is a line segment 3010.

The number of crossings with the table frame is one for the upward line segment, one for the downward line segment, one for the leftward line segment, and one for the rightward line segment.

In step S3104, it is determined whether or not there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally. Since neither the upper, lower, left, or right direction line segment intersects three times or more, the process proceeds to step S3106.

In step S3106, it is checked whether all attention points have been processed. Attention point 3005, Attention point 3
Since 006 remains, the process returns to step S3103.
Attention point 305 for attention point 3005 and attention point 3006
Processing is performed in the same manner as in step S04. As is clear from FIG. 30, when the processing is completed for all the points of interest, (table area fixed number of points of interest) = 0. Since the processing has been completed for all the points of interest, the flow advances to step S3107.

In step S3107, it is checked whether or not (table area fixed attention point) is 2 or more. Since it is not two or more, the process proceeds to step S3109.

In step S3109, the rectangular area is determined as the deformed separator area, and the flow advances to step S210.

FIG. 34 is a flowchart for explaining in detail an example in which a plurality of line segments extending up, down, left, and right are taken in the determination processing of the table area in step S209.

FIG. 32 shows step S209 in the case of a table.
Is a rectangular area, 3202 is a table frame, 3203 is a character in the table, etc., 3204 is a center of gravity (point of interest) in the rectangular area, and 3205 to 3207 are points of interest 32
Line segments extending upward from 04, 3208 to 3210 are line segments extending downward from the point of interest 3204, 3211 to
3213 is a line segment extending leftward from the point of interest 3204;
Reference numerals 3214 to 3216 denote line segments extending rightward from the attention point 3204.

FIG. 33 is a view for explaining step S209 in the case of a deformed separator.
3302 is a deformed separator, 3303 is a character in the contents of the deformed separator, etc., 3304 is a center of gravity (point of interest) in a rectangular area, 3305 to 3307 are line segments extending upward from the point of interest 3304, and 3308 to 3310 are points of interest 3304
The line segments 331 1 to 313 extending downward from, the line segments extending leftward from the point of interest 3304, 3314 to 33
Reference numeral 16 denotes a line segment extending rightward from the point of interest 3304.

The case of a table will be described below with reference to the flowchart of FIG. 34 and FIG.

In this embodiment, the center of gravity is set at an arbitrary point of interest in a rectangular area determined as a table, and a total of 12 line segments are extended three by three in the vertical and horizontal directions. If one (or more) point of interest that intersects the table frame three or more times exists among the line segments extended vertically and horizontally, the rectangular area is determined to be a table, and if not, a deformed separator is determined. Table determination means to be used.

First, in step S3401, the center of gravity of the rectangular area is obtained and set as a point of interest.

A point where the width and the height are each halved is the center of gravity, and an attention point 3204 is obtained.

Next, in step S3402, the point of interest 320
A total of 12 line segments are extended from 3 to 4 in the up, down, left, and right directions, and the number of times each intersects the table frame is counted. As is apparent from FIG. 32, the number of times of intersection with the table frame is represented by the line segment 3205, the line segment 3206, and the line segment 320, respectively.
7 three times, line segment 3208, line segment 3209, line segment 3210
Three times, line 3211, line 3212, line 3213 twice, line 3214 twice, line 3215, line 321
6 is once.

In step S3403, it is checked whether there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally.

Line segments 3205, 3206, 320
7, the line segment 3208, the line segment 3209, and the line segment 3210 intersect with the table frame three or more times, and the process advances to step S3404. If there is no line segment that crosses three or more times, step S
Proceed to 3405.

In step S3404, the rectangular area is determined as the table area, and the flow advances to step S210.

Next, the case of a modified separator will be described with reference to the flowchart of FIG. 34 and FIG.

First, in step S3401, the center of gravity of the rectangular area is obtained and set as a point of interest.

The point where the width and height are each halved is the center of gravity, and the point of interest 3304 is obtained.

Next, in step S3402, the point of interest 330
A total of 12 line segments are extended from 3 to 4 in the up, down, left, and right directions, and the number of times each intersects the table frame is counted. As is apparent from FIG. 33, the number of times of intersection with the table frame is represented by line segment 3305, line segment 3306, and line segment 330, respectively.
7 once, line segment 3308, line segment 3309, line segment 3310
Once, segment 3311, segment 3312, segment 3313 once, segment 3314, segment 3315, segment 3316 is one
Times.

In step S3403, it is checked whether there is a line segment that intersects the table frame at least three times among the line segments extended vertically and horizontally. Since there is no line segment that intersects three or more times, step S340
Go to 5.

In step S3405, the rectangular area is determined as a deformed separator area, and the flow advances to step S210.

FIG. 36 is a flow chart for explaining in detail the table area determination processing in step S209, and is an example in which the point of interest is obtained in a region other than the table frame in the area rectangle (step S1701).

FIG. 35 is a diagram for explaining step S209 in the case of a table, where 3501 is a rectangular area, 3502 is the center of gravity of the rectangular area 3501, 3503 is 8 neighborhoods of the center of gravity 3502, 3
504 is an upper left point among eight neighborhoods of the center of gravity 3502, 35
05 is a point directly above and attention point of the eight neighborhoods of the center of gravity 3502, 3506 to 3509 are line segments extending vertically, horizontally, and from the attention point 3505, black circles are black pixels belonging to the table frame,
White circles represent pixels other than the table frame.

Also, black pixels constituting characters and the like in the table are omitted.

The case of a table will be described below with reference to the flowchart of FIG. 36 and FIG.

In this embodiment, an arbitrary point of interest that does not belong to the table frame in the rectangular area determined to be a table is taken, and a total of four line segments are extended one by one in the vertical and horizontal directions. . If there is one or more points of interest where at least one of the line segments extending vertically and horizontally intersects the table frame three or more times, the rectangular area is determined to be a table, and not so. The case is defined as table determination means for determining the deformation separator.

First, in step S3601, a point of interest other than the table frame is obtained in the rectangular area.

Here, the center of gravity is first determined as a point of interest, and it is checked whether or not the center of gravity belongs to the table frame. If they do not belong, the center of gravity is taken as the point of interest. If the center of gravity belongs to the table frame, the vicinity of 8 of the center of gravity is checked clockwise from the upper left, and points not belonging to the table frame are set as points of interest. If it is not near 8, the outside will be checked once.

First, the center of gravity is determined.

The place where the width and height are each halved is the center of gravity, and the center of gravity 3502 is obtained.

Next, it is checked whether the center of gravity does not belong to the table frame. The center of gravity 3502 belongs to the table frame as is apparent from FIG. Since it belongs to the table frame, the points near 8 are examined. First, it is checked whether or not the upper left point 3504 of the center of gravity 3502 does not belong to the table frame. The neighborhood point 3504 belongs to the table frame as is apparent from FIG. Then, the next eight points are examined clockwise. The next point is the neighborhood point 3505.

As is clear from FIG. 35, since the neighboring point 3505 does not belong to the table frame, the neighboring point 3505 is set as a point of interest, and the flow advances to step S2802.

Next, in step S2802, the point of interest 350
Lines are extended in the vertical and horizontal directions from 5, and the number of times each intersects the table frame is counted. In FIG. 35, a point of interest 35
A line segment extending upward from 05 is a line segment 3506, similarly, a line segment 3507 is at the bottom, a line segment 3508 is at the left, and a line segment 3509 is at the right.

The number of crossings with the table frame is 35
06 twice, line segment 3507 three times, line segment 3508 2
Times, the line segment 3509 is one time.

In step S2803, it is determined whether or not there is a line segment that intersects the table frame three or more times among the line segments extended vertically and horizontally. Since the line segment 3507 intersects the table frame three or more times,
The process proceeds to step S2805. If there is no line segment that crosses three or more times, the process advances to step S2806.

In step S2805, the rectangular area is determined as the table area, and the flow advances to step S210.

If the processing has proceeded to step S2806, the rectangular area is determined as the deformed separator area, and the processing proceeds to step S210.

It is to be noted that, at the time of image input in step S201, it is determined whether or not the image is multi-valued, and the image is converted into a binary image.
Even if the input image is a multi-valued image such as a color image, the region division processing can be performed.

When inputting an image in step S201,
If the number of pixels of the input image is large enough to require a great deal of processing time, the processing can be speeded up by image thinning-out processing in which an image of vertical m dots and horizontal n dots is made one pixel.

Further, in the detection of the separator in step S203 and the like, the rectangular labels are distinguished based on the difference between the vertical and horizontal of the separator, the difference in the threshold value when obtaining the figure or the table, etc. It is also possible to do.

When the table area is determined in step S209, the table area determination processing can be performed without equally distributing the number of line segments extending in the up, down, left, and right directions.

When the table area is determined in step S209, when there are a plurality of points of interest, the table area determination processing can be performed without equally distributing the number of line segments extending in the up, down, left, and right directions for each point of interest.

(Embodiment 3) In this embodiment, an example will be described in which radiation is extracted from a rectangular area determined to be a table and the table area is determined from the number of times of intersection with the table frame and the direction. The configuration of the device of the present embodiment is the same as that of the first embodiment.

The flowchart showing the image processing in the image processing apparatus of this embodiment is the same as the flowchart of FIG. 25 described in the second embodiment.

Step S209 Next, the table area is determined. FIG. 37 shows step S209.
9 is a flowchart illustrating in detail a table area determination process. The same processing steps as those in FIG. 28 of the second embodiment are denoted by the same reference numerals, and description thereof will be omitted.

In the present embodiment, the center of gravity is set at an arbitrary point of interest in a rectangular area determined to be a table, and a total of four line segments are extended one by one in the vertical and horizontal directions. In addition, one point of interest is such that at least one of the vertically extended line segments intersects the table frame at least twice and at least one of the horizontally extended line segments intersects the table frame at least two times ( If there is), the rectangular area is determined to be a table, and if not, a table determining means is determined to determine a deformed separator.

In step S3703, it is checked whether at least one of the vertically extended lines intersects the table frame at least twice. Since both the line segment 2605 and the line segment 2606 intersect the table frame at least twice, the process advances to step S3704. if,
If there is no line segment that intersects at least twice, the process proceeds to step S3706.

In step S3704, it is checked whether at least one of the line segments extended left and right intersects the table frame at least twice. Since the line segment 2605 intersects the table frame two or more times, the flow advances to step S3705. If there is no line segment that intersects twice or more, the process proceeds to step S3706.

In step S3705, the rectangular area is determined as the table area, and the flow advances to step S210.

Next, the case of a deformed separator will be described.

In step S3703, it is checked whether at least one of the vertically extended lines intersects the table frame at least twice. Since neither the line segment 2705 nor the line segment 2706 intersects the table frame twice or more, the process proceeds to step S3706.

In step S3706, the rectangular area is determined as a deformed separator area, and the flow advances to step S210.

Step S210 Finally, the rectangular data of the various regions obtained as described above is output from the output unit 104 together with the image data.

FIG. 38 is a flowchart for explaining in detail an example in which a plurality of points of interest are taken in a rectangular area in the determination processing of the table area in step S209.

In the processing of FIG. 38, the same processing steps as those in the flowchart of FIG. 31 in the second embodiment are denoted by the same reference numerals, and description thereof will be omitted.

The case of a table will be described below with reference to the flowchart of FIG. 38 and FIG.

In this processing, three arbitrary points of interest in the rectangular area determined to be a table are taken, and a total of 12 points are considered for each point of interest. I will lengthen it. In addition, at least two points of interest such that at least one of the vertically extended lines intersects the table frame at least twice and at least one of the horizontally extended lines intersects the table frame at least twice. If it exists, the rectangular area is determined to be a table, and if not, the table is determined by a table determining means for determining a deformed separator.

In step S3804, it is determined whether at least one of the vertically extended lines intersects the table frame at least twice. Since the upper and lower sides intersect with the table frame at least twice, the process proceeds to step S3805. If there is no line segment that crosses twice or more, the process proceeds to step S3106.

In step S3805, it is checked whether at least one of the line segments extended left and right intersects the table frame at least twice. Since the left direction line segment intersects the table frame more than once, the process proceeds to step S3806. If there is no line segment that crosses twice or more, the process proceeds to step S3106.

In step S3806, one is added to the table area fixed attention point. (Table area fixed attention points) = 0 + 1 = 1

In step S3106, it is checked whether all attention points have been processed. Attention point 2905, Attention point 2
Since 906 remains, the process returns to step S3103.
Attention point 2905, attention point 2906 and attention point 29
Processing is performed in the same manner as in step S04. As is clear from FIG. 29, when the processing is completed for all the points of interest, (table area fixed number of points of interest) = 3. Since the processing has been completed for all the points of interest, the flow advances to step S3107.

In step S3107, it is checked whether or not (table area fixed attention point) is 2 or more. Since it is 2 or more, the process proceeds to step S3108.

In step S3108, the rectangular area is determined as the table area, and the flow advances to step S210.

Next, the case of a deformed separator will be described.

In step S3804, it is determined whether at least one of the vertically extended lines intersects the table frame at least twice. Since the upper and lower sides do not intersect at least twice, step S3
Proceed to 106.

In step S3106, it is checked whether all attention points have been processed. Attention point 3005, Attention point 3
Since 006 remains, the process returns to step S3103.
Attention point 305 for attention point 3005 and attention point 3006
Processing is performed in the same manner as in step S04. As is clear from FIG. 30, when the processing is completed for all the points of interest, (table area fixed number of points of interest) = 0. Since the processing has been completed for all the points of interest, the flow advances to step S3107.

In step S3107, it is checked whether (table area fixed attention point) is 2 or more. Since it is not two or more, the process proceeds to step S3109.

In step S3109, the rectangular area is determined as a deformed separator area, and the flow advances to step S210.

FIG. 39 is a flowchart for explaining in detail an example in which a plurality of line segments extending in the up, down, left, and right directions are obtained in the table area determination processing in step S209.

In the processing of FIG. 39, the same processing steps as those in the flowchart of FIG. 34 in the second embodiment are denoted by the same reference numerals, and description thereof will be omitted.

In step S3903, it is checked whether there is a line segment that intersects the table frame more than once among the line segments that are extended up and down.
Since all the line segments intersect the table frame twice or more, the process proceeds to step S3904. If there is no line segment that intersects at least twice, the process advances to step S3906.

In step S3904, it is checked whether there is a line segment that intersects the table frame more than once among the line segments extended to the left and right.
Line 3211, Line 3212, Line 3213, Line 32
Since step 14 intersects the table frame more than once, step S3
Proceed to 905. If there is no line segment that crosses more than once,
Proceed to step S3906.

In step S3905, the rectangular area is determined as a table area, and the flow advances to step S210.

Next, the case of a deformed separator will be described.

In step S3903, it is checked whether or not there is a line segment that intersects the table frame at least twice among the line segments that are extended vertically.
Since there is no line segment that crosses twice or more, the process proceeds to step S3906.

In step S3906, the rectangular area is determined as a deformed separator area, and the flow advances to step S210.

FIG. 40 is a flowchart for explaining in detail the table area determination processing in step S209, and is an example in which a point of interest is obtained in a region other than the table frame in the area rectangle (step S3601).

In the processing shown in FIG. 40, FIG.
The same processing steps as those shown in the flowchart of FIG.

In this embodiment, an arbitrary point of interest that does not belong to the table frame in the rectangular area determined to be a table is taken, and a total of four line segments are extended one by one in the up, down, left, and right directions. . Also, at least one point of interest is such that at least one of the vertically extended lines intersects the table frame at least twice and at least one of the horizontally extended lines intersects the table frame at least twice. If it exists, the rectangular area is determined to be a table, and if not, a table determining means for determining a deformed separator.

In step S4003, it is checked whether at least one of the vertically extended line segments intersects the table frame at least twice. Since both the line segments 3506 and 3507 intersect the table frame at least twice, the process advances to step S4004. if,
If there is no line segment that intersects at least twice, the process proceeds to step S4006.

In step S4004, it is checked whether at least one of the line segments extended left and right intersects the table frame at least twice. Since the line segment 3507 intersects the table frame twice or more, the process proceeds to step S4005. If there is no line segment that crosses twice or more, the process proceeds to step S4006.

In step S4005, the rectangular area is determined as the table area, and the flow advances to step S210.

If the processing has proceeded to step S906, the rectangular area is determined as a deformed separator area, and the processing proceeds to step S210.

It should be noted that, at the time of image input in step S201, it is determined whether or not the image is multi-valued, and the image is converted into a binary image.
Even if the input image is a multi-valued image such as a color image, the region division processing can be performed.

When inputting an image in step S201,
If the number of pixels of the input image is large enough to require a great deal of processing time, the processing can be speeded up by image thinning-out processing in which an image of vertical m dots and horizontal n dots is made one pixel.

Further, in the detection of the separator in step S203 and the like, the rectangular labels are distinguished based on the difference between the vertical and horizontal directions of the separator, the difference in the threshold for obtaining the figure or the table, etc. It is also possible to do.

When the table area is determined in step S209, the table area determination processing can be performed without equally distributing the number of line segments extending in the up, down, left, and right directions.

Further, when the table area is determined in step S209, if there are a plurality of points of interest, the table area determination processing can be performed without equally distributing the number of line segments extending in the up, down, left, and right directions for each point of interest.

At the time of the last output in step S210, it is possible to output only the image data of the necessary area by referring to the rectangular data of various areas. The time can be further reduced.

[0214]

As described above, according to the present invention, a radiation component is extracted from a heading and any one of the top, bottom, left and right of the heading is obtained.
If there is a body paragraph in more than the direction, the heading is placed before the body paragraph, so that even in the case of a document such as a newspaper article where the heading may be surrounded by the body, the correct ordering can be performed. Since the flow of the sentence is not disturbed, there is no need for the user to make corrections, and the processing procedure can be simplified and the processing time can be shortened.

[0215]

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating image processing in the image processing apparatus according to the first embodiment.

FIG. 3 is a diagram illustrating an example of a labeling process according to the first embodiment.

FIG. 4 is a diagram showing a rectangular data structure according to the first embodiment;

FIG. 5 is a flowchart illustrating processing of a separator and the like detection unit according to the first embodiment.

FIG. 6 is a diagram illustrating separation of attributes (density D and area S) of a rectangle (region) according to the first embodiment.

FIG. 7 is a flowchart illustrating processing performed by a group direction detecting unit according to the first embodiment.

FIG. 8 is a diagram illustrating an example of paragraph detection processing according to the first embodiment.

FIG. 9 is a flowchart illustrating processing of a headings detection unit according to the first embodiment.

FIG. 10 is a flowchart illustrating processing of an ordering unit according to the first embodiment.

FIG. 11 is a flowchart illustrating a process of assigning an order to a headline surrounded by a text of the ordering unit according to the first embodiment.

FIG. 12 is a flowchart illustrating a process of assigning an order to a heading surrounded by a text in the ordering unit according to the first embodiment.

FIG. 13 is a flowchart illustrating a process of assigning an order to a headline surrounded by a text of the ordering unit according to the first embodiment.

FIG. 14 is a diagram illustrating an input document according to the first embodiment.

FIG. 15 is a diagram showing a heading / text paragraph extracted by the area dividing means according to the first embodiment.

FIG. 16 is a diagram showing a state in which the ordering unit of the first embodiment has ordered the headings surrounded by the main text paragraphs.

FIG. 17 is a diagram showing a state in which a heading and a part of a text paragraph are ordered by the ordering means of the first embodiment.

FIG. 18 is a diagram showing a state where all headings and body paragraphs are ordered by the ordering means of the first embodiment.

FIG. 19 is a diagram illustrating how the paragraphs overlap in the first embodiment.

FIG. 20 is a diagram illustrating a state in which it is determined whether a main body paragraph exists in the up, down, left, and right directions of the noted headline in the first embodiment.

FIG. 21 is a diagram illustrating a process of obtaining a point of interest located at α% of height and β% of width from the upper left corner of the target heading of Example 1.

FIG. 22 is a diagram illustrating a state in which it is determined whether a main body paragraph exists in the up, down, left, and right directions of the noted headline in the first embodiment.

FIG. 23 is a diagram showing how the paragraphs in the vertical direction overlap in the first embodiment.

FIG. 24 is a diagram showing how horizontal paragraphs overlap in the first embodiment.

FIG. 25 is a flowchart illustrating image processing in the image processing apparatuses according to the second and third embodiments.

FIG. 26 is a diagram showing a table and a rectangle indicating a table area according to the second embodiment;

FIG. 27 is a diagram illustrating a modified separator according to the second embodiment and rectangles indicating regions of the modified separator.

FIG. 28 is a flowchart illustrating a processing example of a table area determination unit according to the second embodiment.

FIG. 29 is a diagram showing a table and rectangles indicating table areas according to the second embodiment;

FIG. 30 is a view showing rectangles indicating regions of the deformed separator and the deformed separator according to the second embodiment.

FIG. 31 is a flowchart illustrating a processing example of a table area determining unit according to the second embodiment;

FIG. 32 is a diagram illustrating a table and rectangles indicating table areas according to the second embodiment;

FIG. 33 is a view showing rectangles indicating deformed separators and regions of deformed separators according to the second embodiment.

FIG. 34 is a flowchart illustrating a processing example of a table area determination unit according to the second embodiment.

FIG. 35 is a diagram illustrating a table and rectangles indicating table areas according to the second embodiment;

FIG. 36 is a flowchart illustrating a processing example of a table area determining unit according to the second embodiment;

FIG. 37 is a flowchart illustrating a processing example of a table area determining unit according to the third embodiment;

FIG. 38 is a flowchart illustrating a processing example of a table area determining unit according to the third embodiment;

FIG. 39 is a flowchart illustrating a processing example of a table area determining unit according to the third embodiment;

FIG. 40 is a flowchart illustrating a processing example of a table area determination unit according to the third embodiment;

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Akihiro Matsumoto 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (56) References JP-A-59-3692 (JP, A) JP-A-63 -213093 (JP, A) JP-A-2-122380 (JP, A) JP-A-6-215184 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06K 9/20 H04N 1/40

Claims (16)

(57) [Claims] 1. A method according to claim 1, further comprising: extending a radiation portion from a predetermined point in a region divided for each attribute in the input image information; detecting relevant information between the radiation portion and the input image information; And determining the order of the area including the predetermined point with respect to the other divided area according to related information. 2. The image processing method according to claim 1, wherein the predetermined point is a center of gravity of the area. 3. The method according to claim 1, wherein the predetermined point is a point determined based on a height and a width of the area.
The image processing method according to 1. 4. The apparatus according to claim 1, wherein the related information includes intersection information between the divided area other than the area including the predetermined point and the radiation. The image processing method described in the above. 5. The image processing method according to claim 4, wherein the related information includes overlapping information of a region including the predetermined point and a region that intersects with the radiation. 6. The related information includes: a number of radiations in a region where the region intersects with the radiation component where the degree of overlap between the region including the predetermined point and the region that intersects the radiation component is equal to or greater than a predetermined threshold. The image processing method according to claim 5, wherein 7. The image processing method according to claim 1, wherein the area including the predetermined point is a heading area, and the other divided area is a text area. 8. After the order of the heading areas is determined, in the case of a vertically written sentence, the heading area and the body area for which the order is not determined are located at substantially the same horizontal position in order from the top in the vertical direction. Assuming that the regions are the same group, the order of each region is determined in order from the right in the group, and in the case of horizontal writing, the regions located at approximately the same position in the vertical direction from the left are the same group and 8. The image processing method according to claim 7, wherein the order of each area is determined in order from. 9. A line segment radiating means for extending a radiation component from a predetermined point in a region divided for each attribute in input image information, and detecting related information between the radiation component and the input image information. A related information detecting unit that determines the order of an area including the predetermined point with respect to the other divided area according to the detected related information. Processing equipment. 10. The image processing apparatus according to claim 9, wherein the predetermined point is a center of gravity of the area. 11. The image processing apparatus according to claim 9, wherein the predetermined point is a point determined based on a height and a width of the area. 12. The apparatus according to claim 9, wherein the related information includes intersection information between the divided area other than the area including the predetermined point and the radiation. An image processing apparatus as described in the above. 13. The image processing apparatus according to claim 12, wherein the related information includes overlap information of a region including the predetermined point and a region that intersects with the radiation. 14. The related information includes: a number of radiations in an area where the area intersects with the radiation where the degree of overlap between the area including the predetermined point and the area intersecting the radiation is equal to or greater than a predetermined threshold. The image processing apparatus according to claim 13, wherein: 15. The apparatus according to claim 9, wherein the area including the predetermined point is a heading area, and the other divided area is a text area. 16. The area order determination means, after determining the order of the heading area, further sets a heading area and a body area whose order is not determined as a target area, and in the case of a vertically written sentence, from the top. Regions located at substantially the same position in the horizontal direction in order are determined to be the same group, and the order of each region is determined in order from the right in the group. 16. The image processing apparatus according to claim 15, wherein the order of each area is determined as a group from the top in the group.