JP3380024B2 - Document image processing method and document image processing apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a document image processing method and a document image processing apparatus.

[0002]

2. Description of the Related Art Conventionally, when a document is reproduced, the document image is directly reproduced and the document image is not deformed. Therefore, when annotating or commenting on the printed (reproduced) text, there is no choice but to write it in the limited space between the lines of the text, making it difficult to annotate or comment, etc.
There is also the problem that it is difficult to read this.

[0003]

In order to avoid such a problem, it is possible to enlarge the document, but in this case, it is necessary to make a copy on a sheet having a size larger than that of the original document. This causes inconveniences such as being bulky. Furthermore, although the white space can be expanded by enlarging the document, the image data itself also expands.

The present invention physically increases the white space between text objects in a document image without changing the text size of the document image or the relative spatial placement of the text within the text area. It is an object of the present invention to provide a document image processing method and device capable of

[0005]

According to one aspect of the present invention, first, rectangles that are boundaries of a range of document image data are defined, and these rectangles are C ₁ X ×.
Map to a virtual sheet having C ₂ Y dimensions (C ₁ , C ₂ : constant). The new mapped rectangles then have the same dimensions as before, but are placed farther apart from each other.

According to another aspect of the present invention, the above-mentioned mapping is performed so that a logical reading order is preserved by assigning an order number to a rectangular group organized as a text block (document block). Place the new rectangle on the physical document image page.
If the new mapped block overlaps a physical page boundary, split this block in two at that page boundary and renumber all subsequent blocks (renumbering). ).

According to still another aspect of the present invention, first,
Map blocks to physical pages by mapping to image areas. The text blocks are then mapped, starting from the top left of the page. The next block is mapped just to the right of the last mapped block.

As described above, according to the present invention, the white space can be expanded without changing (saving) the image data size (size), the paper size, and the logical reading order. Other features and advantages of the invention are described below.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing an outline of white space enlargement processing according to an embodiment of the present invention. In this process, first, the digital representation of the physical document is created as a bit string (step 2). A document scanner can be used for this purpose. Each bit corresponds to one pixel of the scanned image and represents whether or not there is a white space at that position. The image pixel data obtained in step 2 is compressed in step 4 in order to minimize the total amount of system memory.

In most documents, words and images are separated from each other both vertically and horizontally by areas of white space. Thus, a word, text, or pictorial image can be viewed as a run of predominantly black pixels separated by a spread of white pixels. The boundaries of these two regions define the edges of the rectangular regions containing the word or image. In step 6, the operations required to reclassify the document image pixel data using the relationships described above are performed.

Each of the short areas of the text (document), noise and image also has its own characteristic. For example, word rectangles tend to have smaller dimensions and smaller aspect ratios than picture image rectangles. In step 8, these relationships are used to identify a text block from the entire set of short regions of the document.

At the end of step 8, the coordinates are determined for each of the document image rectangular areas. Given a document image consisting of X pixels in the horizontal direction and Y pixels in the vertical direction, in step 10, an image of size X × Y is obtained by multiplying the coordinate values of each rectangular area. Map to an image of size C ₁ X × C ₂ Y. The resulting image is C ₁ C ₂ times larger, but the size of the original rectangular area remains the same.

At this point, the white space is expanded,
As described above, in this case, it may not be possible to fit the text on one page. The following steps 12-14 are optional steps to reorder and preserve the logical reading order of the document.
In step 12, the rectangular areas are grouped into image blocks and text blocks in a logical order. In step 14, the ordered rectangular areas are mapped to one or more suitable pages to ensure a logical reading order.

FIG. 2 is a diagram showing a hardware configuration suitable for reproducing an image and expanding a white space according to the preferred embodiment of the present invention. This hardware configuration may be included in a stand-alone type device such as a copying machine, or may be a part of a local network or a wide area network. In the document image processing system, the scanning device 20 is connected to the CPU 22. The scanning device 2
As 0, another scanning device known to those skilled in the art may be used. CPU 22 executes commands for processing document image data in accordance with the present invention.

A memory 24 is connected to the CPU 22. Here, as the memory 24, any type of random access memory can be used. Document image data is stored in the memory 24. In this case, the memory 24 may have separate memories 24a and 24b which respectively function as a source memory and a target memory, as described later. A ROM (not shown) may be connected to the CPU 22 in order to store commands and data executed by the CPU 22.

A user interface 26 is connected to the CPU 22. The operator of the document image processing apparatus uses this user interface 2
6, the desired features can be specified for the completed document, in which case the CPU 22 can execute a subset of the commands necessary to perform processing based on these features. For example, the operator of the device can specify that many collated copies be made, or that the completed document be sent to a particular destination.

FIGS. 3 to 5 are flowcharts of the white space expanding method according to the present invention. In the flowchart of FIG. 3, first, the CCD scanner and other scanning devices 1
To scan a document to create a digital representation of the document image (step 201). As a result, each scan line is digitally represented as a bit string corresponding to each pixel of the image. In the preferred embodiment, the scan lines extend from left to right. However, the scan direction may be reset so that the scan direction is associated with the direction in which the document to be scanned is normally read. For example, Arabic text may be scanned from right to left.

Next, the bit map representation is compressed.
(Step 202). This data compression, as described below,
It is used to extract rectangular regions. The compression technique in this embodiment can reduce the actual amount of data used to represent a document by a factor of 4, and the amount of processed data by a factor of 32. In this compression technique, a logical OR operation is used to combine four horizontal scan lines into one compressed scan line.
In addition, the number of scan lines selected as described above,
That is, the four lines are based on experience, and by selecting the four scan lines, it becomes possible to process a document having a low resolution such as a 6-point typeface. It is also possible to select other numbers.

The compression technique of step 202 comprises two processes: vertical compression and horizontal compression. If one or more black pixels are present at the same position in four vertically adjacent scan lines, the pixels of one scan line obtained by compressing the black pixels are expressed as black pixels. It When there is no black pixel at the same position in the group of four scan lines, the pixel of one scan line obtained as a result of compression is expressed as a white pixel.

FIG. 6 is a diagram showing compression of scan lines in this embodiment. FIG. 6 shows four scan lines 300 from the original uncompressed bitmap representation.
Through 303 are shown respectively. Scan line 3
For each of 00 to 303, 2 bytes (3
04,305; 306,307; 308,309; 31
0, 311) are given. Also, the bytes (312, 313) resulting from the vertical compression are shown.
Each of the bytes 304 to 311 and the bytes 312 and 313 obtained as a result of the compression are composed of 8 bits.

After the image data is compressed in the vertical direction, it is compressed in the horizontal direction. In this case, as shown in line 314 of FIG.
If 2 or 313 contains black pixel data, then this segment has all 1 byte of pixel data represented as black, while if one segment does not contain black pixel data, then that segment is All 1-byte pixel data is represented as white. The compression technique described above can reduce the amount of system memory. In the present invention, compression techniques other than the above may be used. Alternatively, the data compression process may be omitted.

After compressing the image in step 202 of FIG. 3, in steps 203 and 204 the data is organized into rectangular areas by analyzing the document image. Here, the rectangular area defines the boundary between the text, the pictorial image, and the noise. In order to obtain the rectangular area data, in step 203, the run length is first extracted by the run length extraction process for each block of continuous black pixels.

In the definition of run length, the first element is
The location of the black pixel where the transition from white to black occurs is identified, and the next element identifies the location where the transition from black to white occurs. Each compressed scanline can have one or more run lengths. By searching a series of consecutive black pixels along the compressed scanline, a "run length" set consisting of run length records is obtained for each compressed scanline. In this process, first, when the logical value "0" represents a white pixel, a continuous logical pixel is identified by obtaining a logical value other than "0" and examining the byte value of the scan line. . The first black pixel position in such processing is set as the start value for the run length. Next, the next white pixel is searched by obtaining a pixel having a logical value of "0" and examining the subsequent byte value of the scan line. This pixel is set as the end value of the run length. In this way
All "run lengths" are extracted for one scan line. Once all run lengths have been extracted for a given scanline, these tuples are labeled as the runlength tuples that lie in the top nth scanline of the compressed bitmap representation. .

Run length classification is used to initially classify the associated extracted rectangular regions. The classification rule for a document scanned at 300 dpi resolution is:
It is based on heuristic data and looks like this: 1. If run length ≦ 2 pixels, SHORT (“short”) is assigned to the run length type. 2. If the run length is> 60 pixels,
Assign LONG (“long”) to the run length type. 3.
If 60 pixels ≧ run length> 2 pixels, MEDIUM (“intermediate”) is assigned to the run length type.

FIG. 7 is a diagram showing scan lines and run lengths of pixels. It should be noted that in FIG. 7, one byte is represented as its corresponding pixel value for simplification. For example, pixel 405 represents a non-zero byte value (ie, a black pixel), while pixel 406 represents a byte value of “0” (ie, a white pixel). The scan line 401 has a portion 402 composed of a series of continuous black pixels. The address of pixel 403 indicates the beginning of its run length, and the address of pixel 404 indicates the end of its run length. Here, assuming that the pixel 403 is at the address “312” and the pixel 404 is at the address “440”, the threshold value for the long run length is “100”.
, The resulting run length record has a start value of "312", an end value of "440", and a run length flag value for long run lengths.

When run lengths are extracted based on compressed scanlines, rectangular areas are constructed that represent the features of the document. These rectangular areas represent the boundaries of consecutive black pixels in both the horizontal and vertical directions of the document image. The run length is one-dimensional, whereas the rectangular area is two-dimensional.

At any point in the process, only two sets of records describing the run lengths of the two compressed scanlines are used and stored in memory. First set of records
Code describes the run length of the current scan line, and the second set of records describes the run length of the past scan line. Past scanline information is
Used to extract a rectangular area. Prior to reading a new set of compressed scanline records, the current set of records is copied to a memory location to hold as a record of past scanlines. The record describing the new scanline is then read into the memory location holding the record describing the current scanline and processed accordingly.

The relationship between the current compressed scan line and the past compressed scan line determines whether the run length of the current compressed scan line is assigned to an existing rectangular area or a new rectangular area is created. . When the first compressed scanline is processed, each run length gives
One rectangular area is defined. When processing new compressed scanlines one after another, the run length is either associated with the rectangular area of the existing scanline, or
Used to define the boundaries of a new rectangular area. If the run length part exists within the boundary of the rectangular area,
The run length is associated with the existing rectangular area. When all the pixels adjacent to the rectangular area in the current compressed scan line are white, one rectangular area is completed,
No further expansion. In other words, one rectangular area is complete when the run length of the current compressed scanline no longer lies within the rectangular area boundary. If a run length is partially within the bounds of the rectangular area, a new rectangular area is created. In such a method, overlapping rectangular areas may be generated. Such overlapping rectangular areas are further processed in the next step.

In FIG. 8, from the run length of the current compressed scan line and the run length of the past compressed scan line,
The construction of a rectangular area is shown. In FIG. 8, the past compressed scan line 501 and the current compressed scan line 502 each include a plurality of run lengths. The past compressed scan line 501 has run lengths 503 to 509. On the other hand, the current compressed scan line 502 has run lengths 510-517. In addition, as shown in FIG.
01 and scan line 502 are bit-aligned. Therefore, the leftmost bit of the scan line 501 corresponds to the leftmost bit of the scan line 502.

Further, FIG. 8 shows rectangular areas 520 to 525 defined in the past. In order for the run lengths 510 to 517 to be added to an existing rectangular area, the relationship between the run lengths 510 to 517 and the run lengths 503 to 509 needs to be: That is, the start point of one run length in the current scan line must have continuity with the run length in the past scan line. For example, in FIG. 8, the start point of the run length 510 is the run length 503, 50 of the compressed scan line 501.
Since it is continuous with 4, the run length 510 is added to the existing rectangular area 520. On the other hand, the run length 515 has no continuity with the run length in the scan line 501 in the past, so that a new rectangular area 522 is created. Further, the run length 508 in the past scan line 501 is not continuous with the run length in the current scan line, and thus the rectangular area 524 is completed.

In FIG. 8, the run length of the scan line 501 is added to the existing rectangular area as follows. That is, run lengths 503 to 504 are added to rectangular area 520, run lengths 505 are added to rectangular area 521, run lengths 506 to 507 are added to rectangular area 523, and run lengths are added. The length 509 is added to the rectangular area 525. Also, depending on the run length 508,
A rectangular area 524 is created. Scan line 502
With respect to, the run lengths 510 and 511 are rectangular areas 520.
In addition, the run lengths 512 to 514 are added to the rectangular area 52.
1, the run length 516 is added to the rectangular area 523, and the run length 517 is added to the rectangular area 525.
Further, as described above, the run length 515 newly creates the rectangular area 522.

As the rectangular areas are constructed, counting is continued for the different types of run lengths contained in each rectangular area. When one rectangular area is defined,
The initial classification of the rectangular area is performed, and the rectangular area is classified into one of four types: vertical line “VL”, horizontal line “HL”, image (picture image) “IMG”, and unknown “UNKNOWN”. To be done. The following general rules are used for classifying rectangular areas. Rule 1. "All run lengths are type LONG", and
If the height of the rectangular area is less than or equal to the threshold of the run length type SHORT, then the rectangular area is classified as an HL type. Rule 2. If "the run lengths are all of the type SHORT" and "the height of the rectangular area is larger than the threshold of the run length of the type SHORT"), the rectangular area is classified as the VL type. Rule 3. "Is the run length type LONG?", Or "the width of the rectangular area is less than or equal to the threshold of the run length type LONG" and "the height of the rectangular area is greater than the image height rectangular area threshold value". , The rectangular area is classified as an IMG type. Rule 4. All other remaining rectangular areas are UNKNOWN
Classify as.

Rule 1 identifies horizontal lines, rule 2 identifies vertical lines, rule 3 identifies rectangular image areas, and rule 4 provides a default classification of "unknown". For a document image of 300 dpi, the threshold of run length type SHORT is set to 2 pixels, and the threshold of the image height rectangular area is set to 8 pixels.
It was defined as 2 pixels. The classification rules above are derived from the known parameters that a typical document contains. These parameters may be modified depending on the resolution of the bitmap representation of the document and / or may be tailored to the document by analyzing the distribution of the size of the rectangular areas.

At the end of the process of step 204 of FIG. 3, a list of rectangular areas describing all basic objects of the document image is created and an initial classification is made. At this stage, some text (document) is vertical line type or UNKN
OWN misclassified as "unknown" type segment. For example, the letters "I", "l", "1" are often misclassified.

Therefore, in step 205, the following rules are used to test the initial classification result, made as described above, to make it more accurate. Rule 1 ': Correct misclassified document, i.e. 1 (1), l (el), or I (eye) classified as vertical lines. If "rectangular area type is VL" and "rectangular area height is smaller than" unknown "rectangular area height threshold", the rectangular area is classified as UNKNOWN type. Rule 2 ': Reallocate rectangular areas based on font size. For rectangular areas larger than the maximum font size, these are images. (2 of the height of the rectangular area
If (double) is larger than (threshold for image height), the rectangular area is classified as an IMG type. Rule 3 ': Allocate image regions based on the assumption that "words" tend to be longer than they are higher. If ((4 times the height of the rectangular area) + (width of the rectangular area)) is larger than (4 times the threshold of the height of the image), the rectangular area is classified as an IMG type. Rule 4 ': This Rule 4'provides a criterion for defining a horizontal line, and is based on the assumption that long horizontal lines tend to be thicker than short horizontal lines that separate text blocks or columns. . If the (width of the rectangular area) as a ratio of (four times the height of the rectangular area) is larger than the (threshold value for the width of the horizontal line), the rectangular area is classified as the HL type. Rule 5 ': Rule 5'provides a criterion for distinguishing horizontal lines from long lines of small (eg 6 point) font text. If ((width of rectangular area) as a ratio of (height of rectangular area)) is larger than (threshold for the ratio of width to height of horizontal line), the rectangular area is classified as HL type.

For the image of 300 dpi, the above threshold values are as follows. That is, the height threshold value for the “unknown” rectangular area is “5” and the image height threshold value is “8”.
2 ", the horizontal line width threshold value is" 77 ", and the horizontal line width height ratio threshold value is" 15 ".

At the end of step 205, the pictorial image area,
Accurately classify vertical and horizontal lines. All other documents are classified as UNKNOWN. These U
The NKNOWN rectangular area represents text in the document or text-like noise. This process in the present invention includes 1992.
April 6, 2014, "" Segmentation of Text Picture and L
skew detection and skew correction processing, short region merging processing, as described in US patent application Ser. No. 07 / 864,423 filed under the name "ines of a Document Image". And / or the processing of block ordering can be added and included at will. Skew detection and skew correction are useful in defining column edges. However, in one embodiment of the present invention, in the processing of step 207 of FIG.
Detect rectangular areas that can be merged into blocks that can be classified as lines of text. Such merged blocks are classified as type "CHAR". Therefore, the group of merged UNKNOWN rectangular areas is a text block.

In this embodiment, two rectangular areas are merged when the rectangular areas are of the same type and are within a predetermined horizontal merge threshold and vertical merge threshold. It Rectangular areas of type "IMG" are not merged. Such a merge threshold is determined according to the image resolution and the average height of the rectangular area.
The following table shows such merge thresholds for documents with a resolution of 300 dpi.

[0039]

[Table 1]

The merge process includes a horizontal merge process and a vertical merge process. In the horizontal merge process,
Text (document) rectangles in adjacent but different columns must not be merged. Skew angle detection can be used to define column edges when included as part of step 206. During the horizontal merging process, a rectangular area classified as “UNKNOWN” and having a boundary length smaller than the noise length threshold is removed as noise. The rest of the horizontally merged rectangular areas are classified as text (ie, type "CHAR").

The vertical merging process includes the steps of extracting horizontal lines of text and vertically merging them. Rectangular area of text and type "IMG", "HL"
Alternatively, if there is an overlap with the “VL” rectangular area, the processing for these rectangular areas is postponed until later processing.

If the rectangular area is inside the rectangular area of the image, the merge process is performed using the same merge parameters as described above. Rectangular areas that overlap the "IMG" rectangular area are not merged. In the next step 209, a process of logically ordering the blocks of text is performed. Although step 209 is a process that can be arbitrarily selected, by performing step 209, it becomes easy to map the text blocks onto the page in the logical reading order after expanding the white space. The process of step 209 may be performed before expanding the white space or may be performed after expanding the white space. Regarding the processing of this step 209, as in steps 201 to 210, ““ Se
gmentation of Text Picture and Lines of a Document
US patent application No. 07 filed under the name "Image""
/ 864,423.

The ordering method of step 209 counts the number of "upper" and "left" text blocks of the block under consideration. It should be noted that the terms “upper side” and “left side” here relate to the geometrical layout of the document.

The block ordering method of this embodiment will be described with reference to FIGS. 9 to 11. Note that FIG.
6 is a flowchart showing a process for determining the logical order of various text blocks of a document image. 10 is a diagram showing a document image, and FIG. 11 is a diagram showing a result value table used to calculate the logical order of text blocks. Referring to FIG. 9, first, a "TAU" value is assigned to each text block (each document block) of the document (step 1101). The "TAU" values are assigned sequentially from the top to the bottom of the document image, starting with "1". That is, "TAU" is assigned in the same order as the text blocks are constructed. Referring to FIG. 10, the text block 1
Document image 120 having 201 to 1207
0 is shown. Further, FIG. 10 shows a rectangular area 1208 of the image. It should be noted that no "TAU value" is assigned to the rectangular area 1208 of the image. When examining a document sequentially from left to right and from top to bottom, the top left text block is block 1201 and is therefore assigned a "TAU" value of "1". Also, the next text block is block 1202. Therefore, this text block 1202 contains "T" of "2".
The AU "value is assigned. Such processing continues until the text block 1207 is assigned the" TAU "value of" 7 ". Result value table 122 of FIG.
At 0, the "TAU" value for each of the text blocks 1201 to 1207 is shown. That is, "T
All the "TAU" values for each text block are shown in line "AU" 1210. The "TAU" values that order the blocks are called the geometric order of the text blocks.

When the value of "TAU" is set, the next step is to generate the value of "MU" for each text block (step 1102). The value of "MU" is first used in determining the logical order of the text blocks. In determining the "MU" value for each text block, the total number of blocks above or to the left of a given block also includes that given block. Referring to FIG. 11, row 121 of result value table 1220
1 shows the “MU” value obtained for the text block of the document image 1200. For example, the "MU" value for text block 1204 is
It is "4". "M for text block 1204
The U ”value is“ 4 ”because the text block 120
3,1201 and 1202 are above or to the left of the text block 1204. The "MU" value of a text block gives a logical ordering when ordered from left to right and top to bottom.

In general, the top-to-bottom geometric order is weighted by the value of "MU", taking into account the left-to-right position of the page. This results in values that give an order of top / left to bottom / right. However, if the text block is aligned from top to bottom in the document, it will preferentially reach the bottom of the text column before proceeding to the next text block to the right. Such priority processing is possible by calculating the value "PSI" for each text block.

Referring to FIG. 9, for each text block of the document image, the "PSI" value is calculated by summing the number of text blocks to the left of the given block (step 1103).
As mentioned above, the "PSI" value provides a means of ordering text when the text block is in a column format. Referring to FIG. 11, the row 1212 of the result value table 1220 shows the obtained "PS".
I "values are shown. For example, text block 12
05 has a "PSI" value of "5". Text block 1205 has a "PSI" value of "5" because blocks 1201, 1203, 1204, 1206, 1207
Is to the left of text block 1205.

Referring again to FIG. 9, the next step 11
In 04, the original "PSI" value is multiplied by the number of text blocks to weight the "PSI" value (step 11).
04). This weighting process allows the text blocks to be logically ordered more accurately.
Row 1213 of the result value table 1220 shows the weighted "PSI" values.

A weighted "PSI" for each text block to determine the final logical order.
The value is added to the "MU" value (step 1105). The resulting value is a very good approximation of the logical order of the text blocks on the document.
In FIG. 11, this processing result is shown in the row 1214 of the table 1220. Referring to FIG. 9, then
It is determined whether or not there is the same weighted "PSI" value and "MU" value (step 1106). This block ordering does not provide useful information, since multiple text blocks have the same logical ordering value if they have the same value. If the "MU" values are not the same, the text block ordering process ends. If they have the same "MU" value, the geometric order of the text blocks is considered (step 1107). Note that, as described above, the geometrical order is the "TA" calculated first.
U "value.

Referring again to FIG. 11, it is clear that no text block has the same "MU" value. Therefore, the result of ordering the text blocks of the document image 1200 is 1203, 1201,1.
204, 1206, 1207, 1202, 1205. This document is a column-type format, as found in newspapers and magazines. Once the blocks are ordered, these text blocks can be used in a character recognition program to logically order the characters on the document page.

Finally, in order to apply the above criteria for detecting a text block as "upper, left", the text block must be clearly above and to the left of any position of the text block in question. I won't. However, the "left side" reference may be that the half or more of one text block is located on the left side of the text block in question in the horizontal direction.

Once the block ordering is complete, the segmented text block information needs to be created, eg, for character recognition, ie in a usable form. When the image representation is compressed, it is necessary to create the actual coordinate address corresponding to the block. This is done by rescaling the image representation to the dimensions of the original uncompressed document image.

When the coordinates of the text block of the document image are determined at least in steps 201 to 204, and optionally in steps 205 to 211, steps 212 to 1290 of FIGS. To do.

Since the dimensions of the document image in the horizontal direction X and the vertical direction Y are known by the previous steps,
In step 212, this document image is C ₁
Map to a larger virtual sheet of dimensions X × C ₂ Y. The memory 24 has steps 201 to 204.
, And optionally the coordinate values (x ₀ , y ₀ ), for each of the rectangular regions defined in steps 205 through 211;
(x _n, y _m) are stored. More specifically, in the process of step 212, each of the rectangular regions from the X × Y document space is converted into C
Map to a space of ₁ X x C ₂ Y.

[0055]

[Number 1] _{x 02 → c 1 x 0 x} n2 → c 1 x 0 + (x n -x 0) y 02 → c 2 y 0 y m2 → c 2 y 0 + (y m -y 0)

In the general form, the above conversion formula is expressed as the following formula.

[0057]

## EQU2 ## x _i → c ₁ x ₀ + (x _i −x ₀ ) y _j → c ₂ y ₀ + (y _j −y ₀ )

In other words, the rectangular areas that define the boundaries of the image data in the document image are identified, and the first set of coordinate values (x ₀ , y ₀ ), (x _n , y) is identified in each rectangular area. _m ) is assigned to each of the rectangular regions, x ₀₂ → c ₁ x ₀ x _n2 → c ₁ x ₀ + (x _n −x ₀ ) y ₀₂ → c ₂ y ₀ y _m2 → c ₂ y ₀ + (y The second set of coordinate values (x ₀₂ , y ₀₂ ), (x _n2 , y) represented by _m −y ₀ ).
_m2 ) is assigned to change the white space in the document image.

12 and 13 are diagrams for explaining the mapping process of step 212. Referring to FIG. 12, the document image 1220 includes text rectangular areas 1222 to 1230 and a picture image rectangular area 1232.
And are included. The document image 1220 is
It has dimensions X and Y. Further, the document image 1240 of FIG. 13 corresponds to the document image generated by the coordinate mapping process. It should be noted that in this conversion, the size (size) of the rectangular area itself does not change. That is, it should be noted that the conversion from the document image 1220 to the document image 1240 functions to expand only the white space between the text areas. In the example of FIG. 13, the constant C ₁ is set to “1” and the constant C ₂ is “1”.
Is set to a value greater than. Therefore, in the case of this example, the white space is expanded only in the vertical direction.

The process of step 212 of FIG. 4 further produces a set of document image records. The first set of document records corresponds to the original document rectangular area data. The first set of document records is stored in the target memory, eg, memory 24a in FIG. Also, the second set of document records corresponds to the transformed document rectangular area of step 212 and associated coordinate data. The second set of document records is stored in the target memory, eg memory 24b of FIG.

At the end of step 212, storage in the target memory is done and the image 1240 is transformed into a real physical form with dimensions of at least C ₁ X × C ₂ Y. However, the physical representation of the expanded document image may exceed the size of current physical media. For example, if an operator wants to print a document with enlarged white space on a paper of the same size as the original document, the new mapped document no longer fits. In addition, the expanded document may have out-of-order columns.

FIG. 14 shows two text columns 125.
One document image 125 with 2,1254
0 is shown. FIG. 15 shows the result of the white space enlargement processing performed on the document image 1250. In FIG. 15, columns 1252 are arranged as columns 1252a and 1252b on different pages. The columns 1254 are arranged as columns 1254a and 1254b on different pages, but the positional relationship (order relationship) with the adjacent columns 1252a and 1252b is maintained. This ordering of the column information allows the reader to read column 1252a on page 1256, column 1252b on page 1258, and then return to page 1256 to read column 1254a. Thus, in the preferred embodiment of the present invention, the logical reading order and the continuity of the text columns can be preserved even if the white space is expanded. Steps 1280 through 129 of FIG.
This processing is shown in 0.

In step 209 of FIG. 3, the target areas are sequentially ordered. The region of interest defines the coordinates of a set of rectangular regions contained within its boundaries. In step 212 of FIG. 4, the coordinate values of the text rectangular area and the target area are mapped to new coordinates, and in step 1270 of FIG. 4, the end coordinate value of each target area is compared with the coordinate value of the physical page boundary. . Physical page boundaries do not necessarily have to match actual page edges. For example, if a margin of 1 cm is desired for one sheet of A4 size paper, a physical boundary can be set at the coordinate of 1 cm from the bottom edge of the page.

If the coordinate value of the physical area exceeds the coordinate value of the physical page boundary, in step 1272, the target area is divided into two target areas at the boundary of the physical area. Therefore, the resulting end coordinates of the first portion of the target area match the boundaries of the physical area, and the start coordinates of the second portion of the target area match the boundaries of the physical area.

In this manner, the processing of step 1272 creates a new list of target areas and coordinate values associated therewith. In step 1274, the newly created second target area is inserted after the first target area in the list of target areas. Then, the sequence numbers of all the target areas following this second target area in the list are stepped (incremented).

Since the target area includes the picture target area as well as the text target area, step 1276 is executed.
Then, the picture image area is mapped to the enlarged new coordinates determined in step 212.

In the initial state, no text area is mapped. In order to map the text area that has not yet been mapped, step 1278 determines the height and width of the area. Then, in steps 1280 to 1282, the text area is mapped in order from the upper left portion of the page area. That is, the picture image area previously mapped to the location where you are trying to map the text area,
Alternatively, if the text area does not exist, the text area is mapped to this position and the position is determined.

On the other hand, when the picture image area or the text area is already mapped in the upper left portion of the document page, the current text area is mapped to the lower side or the right side of this area. Step 1284 checks that the ending coordinates of the current text area do not cross page boundaries and attempts to map the current text area below the previously mapped area. That is, if the ending coordinates of the current text area do not exceed the page boundary, the current text area is below the previously mapped area and has the same X as the previously mapped area.
Map to coordinates.

On the other hand, when the end coordinates of the current text area exceed the page boundary, the current text area is mapped to the right side of the previously mapped area by the processing of steps 1286 to 1288. That is, the X coordinate value and the Y coordinate value of the current area mapped to this position are checked to see if the page boundary is exceeded. If the page boundary is not exceeded, then in step 1288, the current position is set at this position. Map the area of. If the current area is mapped to the right of the existing area, then Y
Give the coordinates.

On the other hand, if the current area cannot be mapped below or to the right of the previously mapped area, then in step 1290 it is determined that the document image page is full, Go to a new page. The image area is allocated to the new page in the same manner as described above.

By performing area mapping by the processing of steps 1270 to 1290, it is possible to reliably maintain (save) the logical reading order.
That is, the region is mapped below or to the right of the previously mapped region and not to the left or above the previously mapped region. In this way, the logical reading order can be preserved.

As a concrete example, a copying machine will be described.

FIG. 16 shows the first page 1 of a multi-page document to be copied by a copying machine using the present invention.
299 is shown. The document image of FIG. 16 includes two text columns 1301 and 1302. In a copier, the operator uses the input device to
You can select the desired document characteristics. These features include, for example, the number of copies and collation, in addition to the selection of the white space expansion feature.

When the desired playback feature is selected, the CCD scanner scans page 1299 to
99 features are obtained as a pixel image. FIG. 17 shows the image representation as one set of extracted rectangular areas.

Each rectangular area in FIG. 17 corresponds to one word, a series of words, or one image. That is, each area 1501 to 1503 is directly associated with the area of FIG. For example, the area 1501 is shown in FIG.
Corresponds to the word "buildings" at the end of the column.

FIG. 18 shows an example in which "UNKNOWN" type rectangular areas are merged to form a text block, and the text blocks thus formed are ordered. Here, the order is indicated by the integer value in the upper left portion of the text block. In this example, the text block corresponds to two columns of the document image. Therefore, when the document image is reduced to a text file, such as in character recognition, the text appears in that file in the order shown above.

FIG. 19 shows FIG. 18 after expanding the white space.
It is a figure which shows the mapping of the text block of. In this example, C ₁ is “1” and C ₂ is “2”.
Other values are possible for C ₁ and C ₂ , but the above values “1” and “2” are 8.5 × 11 inches (21.6 × 2).
It was found to be suitable for 7.9 cm) paper. Therefore, in this example, the text block 1 originally located at the coordinates (0,1), (10,40) has the coordinates (0,0),
Mapped to (10,80), block 200
It becomes 0.

Further, FIG. 19 shows the coordinates of the physical page boundary 2001 of the document image.
As shown in FIG. 19, both text blocks overlap at this boundary. Therefore, text block 2
000 and 2002 are divided at the junction of page boundaries into four text blocks 2004 to 2007. By creating new blocks 2005 and 2007, all blocks are re-ordered as shown.

FIG. 20 shows the mapping of text blocks to physical pages. The first numbered text block 2004 maps to the top left of the first page. That is, the first text block 2004 is mapped to the upper left portion of the first page because it does not interfere with the previously mapped pictorial image area in the upper left portion of the first page.

Next, in the present invention, the text block 2005 is mapped to the lower side of the first text block. However, this second text block does not fit in this area because it crosses a physical page boundary. Therefore, the present invention maps block 2005 to the right of block 2004. Where block 2
005 has the same Y coordinate value as the Y coordinate of the previously mapped block 2004.

An attempt is then made to map block 2006 just below block 2005. However, in this case, the block 2006 cannot be arranged at this position because it crosses the physical page boundary. Therefore, an attempt is made to map block 2006 to the right of previously mapped block 2005 from the same Y coordinate as block 2005. However, this mapping crosses horizontal physical page boundaries. Therefore, for block 2006,
We need to map this to a new page.

Page 2010 is the next physical page. Block 2006 is mapped to the top left of page 2010. The process described above continues for block 2007. The completed document is shown in FIG.

Although the preferred embodiment of the present invention has been described above, various modifications will be apparent to those skilled in the art. For example, in the above-described processing, the black pixels of the image are focused and the processing is performed on the black pixels, but the same processing may be performed by focusing on the white pixels. Further, the physical image may be a sheet of paper, a display monitor or another visual representation of a document image, and the physical image should be broadly construed.

Further, in the above-mentioned specific examples, the case where the present invention is applied to the copying machine has been described, but the present invention is not limited to the copying machine and can be applied to other devices and other uses.

[0085]

As described above, according to the present invention, the rectangular area that defines the boundary of the image data in the document image is identified, and the rectangular area is set to the area of C ₁ X × C ₂ Y. By mapping to expand the white space in the document image with X and Y dimensions, you can change the size of the text in the document image and the relative spatial arrangement of the text in the text area. Without,
The white space between the text objects of the document image can be physically increased.

[Brief description of drawings]

FIG. 1 is a flowchart showing an embodiment of document image processing according to the present invention.

FIG. 2 is a diagram showing a hardware configuration suitable for reproducing an image and expanding a white space according to the present invention.

FIG. 3 is a flowchart showing an example of white space expansion processing according to the present invention.

FIG. 4 is a flowchart showing an example of white space enlargement processing according to the present invention.

FIG. 5 is a flowchart showing an example of white space expansion processing according to the present invention.

FIG. 6 is a diagram showing an example of compression of scan lines.

FIG. 7 is a diagram showing scan lines and run lengths.

FIG. 8 is a diagram for explaining a method of forming a rectangular area from a past compressed scan line and a current compressed scan line.

FIG. 9 is a flowchart for explaining text ordering.

FIG. 10 is a diagram showing an example of a document image.

FIG. 11 shows a result value table used to calculate the logical order of blocks.

FIG. 12 is a diagram for explaining a mapping process according to the present invention.

FIG. 13 is a diagram for explaining a mapping process according to the present invention.

FIG. 14 is a diagram showing an example of a document image.

FIG. 15 is a diagram showing a result of performing white space expansion processing on the document image of FIG.

FIG. 16 is a diagram showing an example of a document image.

17 is a diagram showing a rectangular area extracted from the document image of FIG.

FIG. 18 is a diagram illustrating an example of ordered text blocks.

FIG. 19 is a diagram showing mapping of a text block after expanding a white space with respect to the text block of FIG. 18;

FIG. 20 is a diagram showing mapping of text blocks to physical pages.

[Explanation of symbols]

20 scanning device 22 CPU 24 memory 25 User interface

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Koichi Ejiri 1-3-6 Nakamagome, Ota-ku, Tokyo Inside Ricoh Co., Ltd. (56) Reference JP-A-2-58158 (JP, A) JP JP-A-4-364584 (JP, A) JP-A-1-131966 (JP, A) JP-A-5-48872 (JP, A) JP-A-4-177976 (JP, A) JP-A-2-143379 (JP , A) JP 61-60066 (JP, A) JP 1-183784 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 11/60 H04N 1/387 G06F 17/21

Claims (9)

(57) [Claims] 1. A document Ime - Ime in di - delimiting Jideta identified rectangular region, a first set of coordinate values in each of the rectangular area _{(x 0, y 0),} (x n, y m) assignment, to each of the rectangular _{regions, x 02 = c 1 x 0} x n2 = c 1 x 0 + (x n -x 0) y 02 = c 2 y 0 y m2 = c 2 y 0 + (y m - The second set of coordinate values (x ₀₂ , y ₀₂ ), (x _n2 , y) represented by y ₀ ).
_m2 ) to change the white space in the document image to see if each second set of coordinate values fits within a physical page with X, Y dimensions. For an image area having a second set of coordinate values that exceeds the page, divide it into an image area that fits within the physical page, and include the new image area obtained by the division, with respect to the image area. A method of processing a document image according to claim 1, wherein the image area is re-ordered to obtain a second sequence number and the image area is mapped to the physical page with the second sequence number. 2. The document image processing method according to claim 1 , wherein the document image has n image areas, and the n image areas have a picture image image area and a text image area. In this case, the mapping of the image area comprises: (a) mapping the pictorial image area to the physical page; and (b) mapping the first text image area to the upper left part of the physical page. Then, in step (c), when the text image area A _i fits within the physical page, the text image area A _i is mapped below the text image area A _i-1 , and in step (d) the text image area A _i. although itself falls within physical pages, over in the process of step (c), the text image area a _i is the boundary of the physical page If maps the text image area A _i in a portion of the physical page that is adjacent to the right edge of the text image area A _i-1, in the process of step (e) step (d), the text image area A A document image processing method characterized in that, when _i exceeds the boundary of the physical page, the text image area is mapped to a new physical page. 3. A medium representation consisting of a plurality of scan lines is given to a run length extracting / classifying means, a run length is extracted from each scan line of the medium representation, and based on the length of the run length, Each of the run lengths is “short”, “middle”,
A run length record is created by classifying the run length as one of “long”, and a part of the medium is represented from the run length information, and a set of coordinate values (x ₀ , y ₀ , x _n , y _n ), and each of the rectangular regions has x ₀₂ = c ₁ x ₀ x _n2 = c ₁ x ₀ + (x _n −x ₀ ) y ₀₂ = c ₂ y ₀ y _m2 = c a second set of coordinate values represented by _{2 y 0 + (y m -y} 0) (x 02, y 02), (x n2, y
_A method of processing a document image, characterized in that _m2 ) is assigned. 4. The document image processing method according to claim 3 , wherein each of the rectangular areas is an "image",
Categorizing as "vertical line", "horizontal line", or "unknown" type, merging "unknown" type rectangular area into at least one text block, and assigning a sequence number to each of the text blocks, The second of each of the image areas
Check whether the set of coordinate values fits within a physical page having X, Y dimensions, and for image areas having a second set of coordinate values beyond the physical page, set this to the physical page. A method for processing a document image, characterized by dividing the image area to fit inside. 5. The document image processing method according to claim 4 , wherein the image areas are re-ordered to obtain a second sequence number including the new image areas obtained by the division. A method for processing a document image according to claim 2, wherein an area is mapped to the physical page with the second sequence number. 6. The method of processing a document image according to claim 5 , wherein the document image has n image areas, and the n image areas have a picture image image area and a text image area. In this case, the mapping of the image area comprises: (a) mapping the pictorial image area to the physical page; and (b) mapping the first text image area to the upper left part of the physical page. Then, in step (c), when the text image area A _i fits within the physical page, the text image area A _i is mapped below the text image area A _i-1 , and in step (d) the text image area A _i. although itself falls within physical pages, over in the process of step (c), the text image area a _i is the boundary of the physical page If maps the text image area A _i in a portion of the physical page that is adjacent to the right edge of the text image area A _i-1, in the process of step (e) step (d), the text image area A A document image processing method characterized in that, when _i exceeds the boundary of the physical page, the text image area is mapped to a new physical page. 7. A means for identifying a rectangular area that defines a boundary of image data in a document image, and a first set of coordinate values (x ₀ , y ₀ ), (x _n , y) for each rectangular area.
_m ), and for each of the rectangular regions, x ₀₂ = c ₁ x ₀ x _n2 = c ₁ x ₀ + (x _n −x ₀ ) y ₀₂ = c ₂ y ₀ y _{m 2} = c ₂ y ₀ + The second set of coordinate values (x ₀₂ , y ₀₂ ), (x _n2 , y) represented by (y _m −y ₀ ).
and means for assigning a _{m @ 2),} a document Ime - changing the white space in the di, respectively
The second set of coordinate values is a physical page with X and Y dimensions.
Check if it fits inside and cross the physical page
For image areas with a second set of coordinate values,
Divided into image regions that fit the physical page within the di, before
Including the new image area obtained by notation division,
The image area is reordered and the second order
Get the ordinal number and place the image area in the second sequence number
A document image processing device, characterized in that it is adapted to map to said physical page . 8. The document image processing apparatus according to claim 7 , further comprising means for scanning the document image. 9. The document image processing apparatus according to claim 7 , further comprising means for outputting a document image.