JP2000293671A - Method and device for image processing and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To analyze an inputted image and to acquire information which makes it possible to perform document editing by extracting a character string area from inputted image information and deciding whether the character string area is a character string area of a text according to the layout information of the extracted character string area. SOLUTION: Image data are read from an image reader (S11-1). The read image data are converted into a PAF(page analysis format) by using an OCR engine (S11-2). For instance, the read image data are subjected to area division, and a block shown by a rectangular frame are extracted. The separation processing of blocks for a text area from the other area blocks is performed (S11-3). Here, text blocks other than a block for a frame area are separated as blocks for a text area. Preparation processing of a text format is performed (S11-4). The format prepared here is the same as a document format prepared by normal word processor software, etc.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for analyzing an input image to obtain information that can be used for document editing.

[0002] The present invention relates to a technique for converting an image into text.

[0003]

2. Description of the Related Art In recent years, image reading apparatuses, OCR (optical character recognition, Optical Character R).
Due to the spread of “ecognition” and the like, opportunities to read a character from a paper manuscript, convert it into a character code, create a document file, and further edit the document file have been increased.

[0004]

However, the image data read from the image reading device is divided into regions, and the document file created by OCR is converted into a frame in which each divided region is converted into a frame. I have. There is no document connection between frames. It was that. Therefore, when performing editing, it is necessary to perform editing only in the area for each area, which is very inefficient.

[0005]

According to the present invention, a character string region is extracted from input image information, and the character string region is converted into a text according to the layout information of the extracted character string region. An image processing method, apparatus, and storage medium for determining whether or not a character string area is included.

In order to solve the above-mentioned problem, the present invention preferably stores the area determined to be the character string area of the text and the area determined not to be the character string area of the text separately.

[0007] In order to solve the above problem, according to the present invention, preferably, the storage stores feature information extracted from each area.

[0008] In order to solve the above-mentioned problems, according to the present invention, the storage is preferably performed by adding identification information capable of identifying each of the storages.

In order to solve the above-mentioned problem, in the present invention, preferably, the layout information of the character string area has a positional relationship with different areas.

In order to solve the above problem, in the present invention, preferably, the layout information of the character string area is the number of lines of the character string included in the character string area.

In order to solve the above problem, the present invention extracts a plurality of image regions from input image information, classifies the extracted image regions into a text region and other regions, and according to the classification. Provided are an image processing method, an apparatus, and a storage medium that store information extracted from each area separately into a first storage area and a second storage area.

In order to solve the above-mentioned problems, the present invention is preferably arranged such that the image information is an image read by a scanner.

[0013] In order to solve the above-mentioned problems, the present invention is preferably arranged such that the image information is an image input at another terminal connected via a line.

In order to solve the above-mentioned problem, the present invention is preferably arranged such that the image information is an image read by a scanner at another terminal connected via a line.

In order to solve the above-mentioned problems, the present invention analyzes input image information, creates a format for determining a character arrangement, and recognizes the input image information according to the created format. A document processing method, apparatus, and storage medium for arranging characters are provided.

In order to solve the above-mentioned problems, the present invention is preferably arranged such that the image information to be analyzed is a text string region extracted from an input image.

In order to solve the above-mentioned problem, the present invention preferably sets the image information to be analyzed to be one of a plurality of regions obtained by dividing input image information into regions.

In order to solve the above-mentioned problems, the present invention is preferably arranged such that the format defines an arrangement of character strings in an area extracted from an input image.

In order to solve the above-mentioned problems, the present invention preferably displays a document in which the characters are arranged on a display screen, and performs editing on the display screen.

In order to solve the above problems, the present invention preferably displays a document on which the characters are arranged on a display screen.

In order to solve the above problems, the present invention preferably prints a document in which the characters are arranged by a printer.

In order to solve the above problem, according to the present invention, preferably, the characters to be arranged are fonts.

In order to solve the above problems, the present invention preferably edits a document in which the characters are arranged, and rearranges the edited document according to a format in which the characters are arranged.

[0024] In order to solve the above problem, the present invention preferably provides that the format includes column information.

In order to solve the above-mentioned problem, the present invention preferably provides that the form includes a paper size.

In order to solve the above-mentioned problem, according to the present invention, preferably, the format includes margin information.

In order to solve the above-mentioned problem, the present invention preferably provides that the form includes an assembling direction.

In order to solve the above problem, the present invention preferably provides that the format includes a character size.

In order to solve the above problem, according to the present invention, preferably, the format includes information on the number of characters per line.

In order to solve the above problem, according to the present invention, preferably, the format includes information on the number of lines per area.

In order to solve the above-mentioned problems, the present invention preferably provides that the format includes typeface information.

In order to solve the above-mentioned problem, the present invention extracts a plurality of image areas from input image information, selects a frame area from the extracted plurality of image areas, and outputs image information of the selected frame area. Create a frame format from.

In order to solve the above-mentioned problem, according to the present invention, preferably, the frame region is arranged so as to overlap another region.

In order to solve the above-mentioned problem, according to the present invention, preferably, the frame region is a text region in a set direction different from the set direction of characters in the body region.

[0035]

FIG. 1 is a system block diagram of an apparatus according to the present invention.

The system bus 1 enables data to be exchanged between components described below. CPU (Cen
The true processing unit 2 executes various processes according to the present invention in accordance with a control program stored in the PMEM 3 described later. The processing shown in the flowchart described later is also executed by the CPU 2. The PMEM (memory) 3 is a memory for appropriately reading, reading, and storing control programs for various processes according to the present invention from the hard disk 10 as appropriate. The control program for the processing shown in the flowchart described later is
It is stored in M3. The PMEM 3 also functions as a text memory, and stores text data input from the keyboard 12 and text data read from the external storage medium 9 or the hard disk 10 under the control of the external storage control unit 8. The communication control unit 4 controls input / output data at the communication port 5. The communication port 5 is connected to a communication port 7 of another device on the network via a communication line 6 such as a LAN or a WAN under the control of the communication control unit 4 to transmit and receive data. Transmission of print data to a printer connected to the network and input of data from a scanner also connected to the network are also performed through the communication port 5.

The external storage control unit 8 reads data from a memory for data files, for example, a hard disk (HD) 10 or an external storage medium (for example, a floppy disk, MO, CD-R, etc.) that is removable from the apparatus. And control the writing of data. The input control unit 11 inputs data from input devices such as a keyboard 12 and a mouse 13. Here, the mouse 13 only needs to function as a coordinate input unit for inputting the coordinates of the position indicated on the display screen of the CRT 16, and may be a tablet, a touch panel, or the like. Furthermore, at the designated coordinate position,
Any device can be used as long as a selection instruction can be made by pressing a button, tapping, or the like. By operating the mouse, the designated position displayed by the mouse cursor is set to a desired position,
For example, a command represented by a command icon can be input by pressing the button while positioning the command icon on the command menu. Instructions for editing and drawing positions can also be made with the mouse.
13 is possible. The operator operates the keyboard 12 to input a character code and various operation commands.

Video image memory (VRAM) 14
Is a memory for holding an image to be displayed on a display such as a CRT, for example. Under the control of the display output control unit 15, display data is developed and written into bitmap data.
Displayed on RT16. The display is not limited to the CRT, but may be a liquid crystal display. The printer control unit 17 controls output of data to the connected printer 18. The printer 18 functions as a unit for printing an image, and is an LBP, an inkjet printer, or the like. The image input device controller 1A controls the connected image input device 1B. The image input device 1B may be a scanner that optically reads a placed document or a film scanner.
Alternatively, it may execute a function of reading image data from a memory storing the image data. In FIG. 1, the printer control unit 17 and the printer 18 and the image input device 1B and the image input device control unit 1A are described as different configurations.
Of course, these may be physically separate components or one component.

It is to be noted that the present invention may, of course, be realized by a single device, or may be configured by a system including a plurality of devices connected via a network. An example of this network system is shown in FIG. 28 and will be described below.

FIG. 28 is a block diagram of a network system in which a network board (NB) 101 for connecting a printer to a network is connected to a printer 102 having an open architecture. The NB 101 connects to a local area network (LAN) 100, for example, an Ethernet interface 10Base-2 having a coaxial connector or a 10Base- having an RJ-45.
It is connected via a LAN interface such as T.

PC 103, PC 104, PC 111, P
A plurality of personal computers (PCs) such as C112 and PC115 are connected to the LAN, and under the control of the network operating system, these PCs
It can communicate with B101 and function as each device connected to the network. Also, for example, the PC 103 can be designated to be used as a network device management PC.
The print processing in the printer 105 locally connected to the printer 102 or the PC 104 may be controlled.

A file server PC is connected to the LAN 100.
104, and accesses the file server 106 via the LAN 100 in accordance with an instruction input on the display screen of the PC 104, and has a large capacity (for example, 10 billion bytes).
The management of reading data from a file stored in the network disk 107 and writing and storing the data is managed. The file server PC 104 serves as a file management unit to receive data files between LAN members,
It performs storage, queuing, caching, and transmission. For example, a data file created by the PC 104 itself or the PC 103 is sent to the file server 106 under the control of the file server PC 104, and the file server 106 arranges these data files in order, and according to a command from the print server 104, The arranged data file is transmitted to the printer 110.

The scanner server 115 controls the locally connected scanner 117 and the remote scanner 110 to input an image. The copying machine 118 is connected to the LAN 11 via the image processing unit 119.
0, and also functions as a scanner or printer under the control of a PC connected via the network 110.

PC 103 and PC 104 are respectively
Generation of data file and L of generated data file
It comprises a normal PC capable of transmitting to the AN 100, receiving a file from the LAN 100, and displaying and processing such a file. Although FIG. 28 illustrates a personal computer device, other computer devices that are suitable for executing network software may be used. Usually LA
LANs, such as N100 and LAN 110, provide services to somewhat local user groups, such as user groups on one floor or multiple connected floors in one building. A wide area network (WAN) may be created as a user moves away from another user, such as in another prefecture. WAN is
Basically, it is an aggregate formed by connecting several LANs with a high-speed digital line such as an integrated high-speed service digital network (ISDN) telephone line. Therefore, as shown in FIG.
And the LAN 120 via the backbone 140 to form a WAN. These connections are simple electrical connections with several buses. Each LAN includes a dedicated PC and, although not necessarily, usually includes a file server and a print server.
Devices connected to the LAN 100, the LAN 110, and the LAN 120 communicate with other LANs via the WAN connection.
Can access the functions of the device.

FIG. 2 shows an image input device 1 in the device (2-1).
FIG. 2 is a perspective view of a system to which an image reading device (2-2) as an example of B is connected.

Next, a file format, PA, which plays an important role in a “method of separating a text region from other regions” described later.
F (Page Analysis Format) will be described.

The PAF is a method of dividing image data into a single rectangular area having a same attribute according to an image characteristic obtained by analyzing the image data, and for each divided block (rectangular area). Is a file format in which data (including data for specifying a block area and data representing characteristics of an image of the block) is stored, and the type of the block (text block, image block, line block, table) Block) is different and holds necessary data suitable for each type. For example, when the image data shown in FIG. 3 is input from the image input device 1B, the area division as shown in FIG. 4 is performed. For example, text blocks such as 4-1 and 4-2 (blocks all of which are determined to be text), image blocks such as 4-3, line blocks such as 4-4, and 4-5 The area is divided into the following table blocks (table blocks) and the like, and the type of each block is determined.

The structure of the PAF will be described in more detail with reference to FIG. This PAF data is obtained by analyzing image data input from the image input device
Is stored in 5-1 is the entire PAF, having a “header” area at the beginning, followed by “block management data”, and “in-block data” managed by the block management data, and a memory area. As shown in 5-2, in the “header” area, the file size of the entire file, the number of blocks held indicating the total number of blocks extracted from the entire file, and the width / height of the read image (image) are displayed. It holds the number of pixels, and also holds what DPI (dot / inch) size image data was read.

The "block management data" area stores, for each block, layout information indicating the position of the block in the read image and attribute information of the image of the block. That is, the PAF data is header 5
The number of block management data areas is equal to the number of blocks within -1. Distance from the left of the paper, distance from the top of the paper,
In addition, the table holds the horizontal / vertical size of the area of each block in the number of pixels, and further has a table indicating what kind of data the block holds. The data up to this point is the data that is commonly stored in any type of block, and the following differs depending on the type of block. For example, in the case of “text block management data” shown in 5-3, an offset indicating the location of the data in the block is provided, and the language information of the text (text in the block is mainly Japanese or English) ), And the direction of the text (horizontal or vertical). In the case of "image block management data" shown in 5-4, an offset indicating the location of the image data in the block is provided. In the case of "line block management data" shown in 5-5, the start and end positions of the line are set. It has a table showing each (X, Y) position shown, and the type and thickness of the line.

The "data in block" area differs depending on the type of data in each block. For example, in the case of the “data in a text block” area, an area for storing text data as a result of character recognition of image data of a block specified by the associated block dissociation data, and a character code string; Layout information indicating the arrangement of characters, lines, and the like obtained when each character image is extracted. As shown in 5-6, it has OCR management data at the beginning, and then holds the line information area from the first line to the last line, and further holds the character information area from the first character to the last character of the last line of the first line are doing. The OCR management data 5-7 includes the total number of lines in the block, an average line spacing value (a value obtained by adding each line interval and dividing by the number of lines),
And the horizontal / vertical size of the block in the number of pixels. The line information 5-8 is an area for storing character layout information and the like for each line, and includes the total number of characters existing in the line,
Holds the indentation value up to the first character, the average character spacing value (the sum of each character spacing and dividing by the number of characters), and a flag indicating whether there is a line feed at the end of the line. I have. The determination as to whether or not a line break exists is made based on the distance from the end of the line to the end of the block area. The character information 5-9 includes, for each character, a position (distance from the left / top of the block) where the character image of the character is extracted, a character width, a character height, a font type,
Holds the font style, font size, and character code resulting from OCR. The resulting character code may be the one having the highest similarity at the time of the recognition processing, or may be the code of a plurality of candidate characters.

Next, a process for separating the image data read from the image reading apparatus 2-2 into a text region and other regions will be described with reference to FIGS.
4, description will be given according to the flowcharts of FIGS.

First, in step 11-1, image data is read from the image reading device 2-2. Further, in step 11-2, the read image data is
Convert to PAF (Page Analysis Format) using engine. For example, the image data obtained when a document as shown in FIG. 6 is placed on the image reading device 2-2 and read is divided into regions, and blocks as shown by rectangular frames in FIG. 7 are extracted. Blocks 7-1, 7-2, 7-3, 7-4, and 7-5 are text blocks, that is, areas including only character strings, and the set direction is horizontal writing. The block shown in 7-6 is also a text block, but the set direction is vertical writing. If the line blocks shown in 7-7 and 7-8 are line blocks, a determination is made in the area division processing.
Is created and stored in the HD 10.

After the conversion into the PAF, in step 11-3, "separation processing between text blocks and other blocks" is executed in order to discriminate the text area blocks from other area blocks.

The flowchart shown in FIG. 12 shows a "separation process between a text block and another block" (step 11-).
It shows the detailed processing of 3). Step 12-1
First, all the font sizes of all the text blocks (the font sizes of the character information shown in 5-9) are examined, the font size having the largest number is obtained, and the font size is stored in the HD 10 as the most font size information. Then, in step 12-2, "header area text block extraction processing" is executed to extract a header area text block.

The flowchart shown in FIG. 13 shows the "extraction process of the header area text block" (step 12-).
This shows the detailed processing of 2), and is performed in step 13-1.
At the Y position (5
The text block having the smallest “distance from the paper of the“ text block management data ”indicated by -3) is acquired.
If there is a text block at the same Y position, a small block at the X position (the distance from the left of the sheet of “text block management data” shown in 5-3) is acquired. Further, in step 13-2, the composition direction of the text block (the composition direction of the “text block management data” shown in 5-3) is checked to determine whether or not the text is written horizontally. If it is not horizontal writing, it is determined that the block is not a header area block, and the “header area text block extraction processing” ends. In the case of horizontal writing, in step 13-3, the number of lines in the block (the number of lines of the OCR management data shown in 5-7) is examined, and it is determined whether or not the block has only one line. If it is a block of two or more lines, it is determined that the block is not a header area block, and the “header area text block extraction processing” ends. If the block has only one line, in step 13-4, the average font size of the block is compared with the maximum font size obtained and stored in step 12-1 of FIG. Is determined to be 1.5 times or less. If not less than 1.5 times, it is determined that the block is not a header area block,
The “text block extraction processing for header area” ends. If it is less than 1.5 times, the text block is determined to be a header area text block,
The identification information of the block is stored in the HD 10 as a text block for the header area. Thus, the first header area text block (the first element of the header area text block) is obtained. Next, step 13-
In step 6, a text block next to the Y position (the distance from the sheet of the “text block management data” shown in 5-3) is acquired (if there is a block at the same Y position, a small block at the X position) is acquired. Then, in step 13-7, it is determined whether the block exists in the range of two lines of the text block stored in step 13-5. For example, FIG.
It is assumed that the text block shown in 8-1 is the first element of the text block for the header area and the step 13-5
The range of the two lines is the size obtained by doubling the block vertical size 5-7 of the text block 8-1 by the distance 5-4 from the paper of the text block 8-1.
And the range defined by the line segments 8-2 and 8-3. Therefore, the text block shown in 8-4 is within the range, and the text block shown in 8-5 is outside the range. That is, in step 13-7,
A block in which the block acquired in step 13-6 is completely included in the determined range is determined to be “within the range”. If it is determined in step 13-6 that the obtained text block is out of the range, it is determined that the text block is a header area text block and an undetermined block no longer exists. Processing ”ends. If it is determined in step 13-7 that the value is within the range of two lines of the first element of the header area text block, in step 13-8, the text block combination direction ("text block" shown in 5-3) is set. Investigate the direction of management data) and determine whether it is horizontal writing. If it is not horizontal writing, it is determined that the block is not a header area block, and the process returns to step 13-6 to acquire the next target text block. In the case of horizontal writing, in step 13-9, the number of lines in the block (the number of lines of the OCR management data shown in 5-7) is examined to determine whether the block has only one line. If the block is a block of two or more lines, it is determined that the block is not a header area block, and the process returns to step 13-6 again to acquire the next target text block. If the block has only one line, in step 13-10, the average font size of the header area text block stored in step 13-5 and the average font size of the text block are compared. Determine if it is within 20%. If the size difference is not within 20%, the process returns to step 13-6 again to acquire the next target text block. If the size difference is within 20%, in step 13-11, the text block is set
The identification information of the block is stored in D10.

With the above processing, the extraction processing of the header area text block for obtaining one or a plurality of header area text blocks (step 12-2) is completed.

Next, in step 12-3 of FIG.
A "footer area text block extraction process" for extracting a footer area text block is executed.

The flowchart shown in FIG. 14 shows the "texter area extraction processing" (step 12-).
This shows the detailed processing of 3), and is step 14-1.
At the Y position (5
The text block with the largest “distance from the paper of“ text block management data ”indicated by -3) is acquired.
If there is a text block at the same Y position, a small block at the X position (the distance from the left of the sheet of “text block management data” shown in 5-3) is acquired. Further, in step 14-2, the composition direction of the text block (the composition direction of the “text block management data” shown in 5-3) is checked to determine whether or not it is horizontal writing. If it is not horizontal writing, it is determined that the block is not a footer area block, and the “footer area text block extraction processing” ends. In the case of horizontal writing, in step 14-3, the number of lines in the block (the number of lines of the OCR management data shown in 5-7) is examined, and it is determined whether the block has only one line. If the block is more than one line,
It is determined that the block is not a footer area block, and the “footer area text block extraction processing” ends. If the block is a block of only one line, in step 14-4, the average font size of the block is compared with the maximum font size obtained and stored in step 12-1 of FIG. 1.
It is determined whether it is 5 times or less. If it is not 1.5 times or less, it is determined that the block is not a footer area block, and the “footer area text block extraction processing” ends.
If it is 1.5 times or less, the text block is determined to be a footer area text block, and H
D10 stores the identification information of the block as a text block for the footer area. Thus, the first footer area text block (the first element of the footer area text block) is obtained. Next, step 14-6
, A text block next to the Y position (the distance from the sheet of “text block management data” shown in 5-3) (the smaller block at the X position if there is a block at the same Y position) is acquired. . However, step 14-1 or the previous step 14
If there is a block whose processing is suspended at the same Y position in -6, the block of that block is acquired.
In step 14-7, it is determined whether the block exists in the range of two lines of the text block stored in step 14-6. The method of obtaining the range for two rows is similar to step 13-7, except that the starting point for defining the range is the distance obtained by adding the block vertical size to the distance of the first element block from the sheet. different. For example, the text block shown at 8-6 in FIG.
In the case of the first element of the footer area text block, the range of the two lines is the line segment indicated by 8-7 and 8-8. Therefore, the text blocks shown in 8-9 and 8-10 are within the range, and the text blocks shown in 8-11 are outside the range. If it is determined in step 14-6 that the acquired text block is out of the range,
It is determined that there is no more undetermined block for the footer area text block, and the “footer area text block extraction process” ends. Step 14-
In step 7, if it is determined that the value is within the range of two lines of the first element of the footer area text block, the process proceeds to step 14-.
In step 8, the grouping direction of the text block (the grouping direction of the "text block management data" shown in 5-3) is checked to determine whether or not it is horizontal writing. If it is not horizontal writing, it is determined that the block is not a footer area block, and the process returns to step 14-6 again to acquire the next target text block. If it is horizontal writing, step 14-9
In step (1), the number of lines in the block (the number of lines of the OCR management data shown in 5-7) is examined to determine whether or not the block has only one line. If the block is two or more rows, it is determined that the block is not a footer area block, and step 14-6 is performed again.
To get the next target text block.
If the block is only one line, step 14-1
At 0, the average font size of the footer area text block stored in step 14-5 is compared with the average font size of the text block, and it is determined whether the size difference is within 20%. If the size difference is not within 20%, the process returns to step 14-6 again to acquire the next target text block. If the size difference is within 20%, in step 14-11, the text block is set as the footer area text block, and the identification information of the block is stored in the HD 10. With the above processing, the extraction processing (step 12-3) of the footer area text block obtained by one or more footer area text blocks is completed.

Next, in step 12-4 of FIG.
A "frame area block extraction process" for extracting a frame area block is executed. The flowchart shown in FIG. 15 is described as “frame area block extraction processing” (step 12
4) shows the detailed processing of the above.

In step 15-1, a main set direction is determined from all the text blocks other than the header area text block and the footer area text block which have been obtained and stored, and is set as the main area set direction. The main group direction is obtained as follows. The respective areas (determined from the block horizontal / vertical sizes of the text block management data shown in 5-3) of all the text blocks other than the header area text block and the footer area text block are shown in respective set directions (shown in 5-3). The sum is separately calculated for each text block management data), and the larger area is set as the main set direction.

Next, in step 15-2, among the blocks in the PAF, the Y position (5-3) is set in all the text blocks other than the text block for the header area and the text block for the footer area which have already been obtained and stored. The block with the smallest “distance from the paper of the“ text block management data ”shown) is acquired. If there is a block at the same Y position, the X position (5-
The smaller blocks (distance from the left of the sheet of “block management data” shown in 3, 5-4, and 5-5) are acquired. If there is no corresponding block, step 15
In step -3, this is determined, and the "frame area block extraction process" ends. If the corresponding block exists, in step 15-4, it is determined whether the type of the block (the type of the block of “block management data” shown in 5-3, 5-4, and 5-5) is an image. Judge,
If the block is an image block, the block is extracted as an image frame and stored in the HD 10 together with the identification information of the block in step 15-5. If it is determined that the block is not an image block, it is determined in step 15-6 whether the block type (block type of “block management data” shown in 5-3, 5-4, and 5-5) is a table. Is determined, and if it is a table block, step 15-
In step 7, the information is extracted as a table frame and stored in the HD 10 together with the identification information of the block. If it is determined that the block is not a table block, it is determined in step 15-8 whether the block type (block type of “block management data” shown in 5-3, 5-4, and 5-5) is a line. And if it is a line block,
In step 15-9, the data is extracted as line data and stored in the HD 10 together with the identification information of the block. If it is determined that the block is not a line block, step 15-
10, the type of block (5-3, 5-
It is determined whether the “block management data” shown in 4, 5-5 is a text block, and if it is a text block, in step 15-11,
Execute "text block check processing".

The flowchart shown in FIG. 16 shows the detailed processing of the "text block check processing" (steps 15-12).

In step 16-1, it is determined whether or not the text block extends over the header area using the block management data. If the text block does extend over the header area, the text block cannot be a text area. Is returned (step 16-6). If it does not straddle the header area, it is determined in step 16-2 whether it straddles the footer area. If it straddles, it cannot return to the body area, and it is returned that it has been extracted as a text frame ( Step 16-6). If the text block does not extend over the footer area, in step 16-3, the composition direction obtained and stored in step 15-1 is compared with the composition direction of the text block. Therefore, the fact that the text frame has been extracted is returned (step 16-6). If the orientations are the same, in step 16-4, it is determined whether the largest font size of the text block is at least twice the largest font size of the entire text block obtained and stored in step 12-1. If it is twice or more, it returns that it has been extracted as a text frame (step 16-6). If it is not twice or more, it returns that it has been extracted as a text region (step 16-).
5).

In step 15-12, the data returned in step 16-5 or step 16-6 in the "text block check process" is judged, and if the data indicates that the frame is a text frame, step 15-
In step 13, the text block acquired in step 15-2 is extracted as a text frame, and the HD 10 stores the block identification information.

By the above-described processing, storage of the blocks classified into the blocks for the text area and the blocks for other areas is completed, and the creation of the text format in step 11-4 in FIG. 11 is started.

The result of executing the “separation process between text region and other region” (step 11-3) described with reference to the flowcharts of FIGS. 12 to 16 is performed on the image data shown in FIG. This will be specifically described.

In step 11-2, when "block discrimination between text block and other blocks" is executed for each block (FIG. 7) obtained by converting the image data read into PAF using the OCR engine, first, In step 12-2, the text blocks shown in 7-1 and 7-2 in FIG. 7 are classified as the header area text blocks.
Subsequently, in step 12-3, the text blocks indicated by 7-3, 7-4, and 7-5 in FIG. 7 are classified as the footer area text blocks. And step 1
In 2-4, as a block for the frame area, 7- in FIG.
The text block shown in FIG. 6 and the line blocks shown in 7-7 and 7-8 are classified, and the text blocks shown in FIG. 9 other than these classified blocks are classified for the body region. As a result, each of the 18 areas shown in FIG. 9 is an area that is classified as a text area block.

In the header area text block extraction processing shown in the flow chart of FIG. 13, one of the criteria for determining whether or not the block to be determined is the header area text is step 13-4. In the above, it was determined whether or not the average font size of the block was 1.5 times or less the maximum font size of all text blocks, but the average font size of the block was compared with a numerical value specified by the user. Is also good. FIG. 17 shows a flowchart of the header area text block extraction process in this case. Step 13-
In step 17-4, it is determined whether or not the average font size of the block is equal to or smaller than the point size specified by the user. If it is not smaller than the designated point size, it is determined that the block is not a header area block, and the “header area text block extraction processing” ends. If it is determined that the text block is smaller than the designated point size, it is determined that the text block is a header area text block, and the identification information of the block is stored in the HD 10 as a header area text block. In the flowchart of FIG. 17, the processing other than step 17-4 is the same as the processing shown in the flowchart of FIG. 13, and the same steps are denoted by the same step numbers.

In accordance with the header area text block extraction processing, the footer area text block extraction is replaced with a comparison step with a numerical value designated by the user. This processing is shown in the flowchart of FIG. 18, and step 18-4 is executed instead of step 14-4 in the flowchart of FIG. In step 18-4, it is determined whether or not the average font size of the block is equal to or smaller than the point size specified by the user. If it is not smaller than the point size designated by the user, it is determined that the block is not a footer area block, and the “footer area text block extraction process” ends. If it is smaller than the point size specified by the user, the text block is determined to be a footer area block, and the identification information of the block is stored in the HD 10 as a footer area text block. In the flowchart of FIG.
The processing other than Step 18-4 is the same as the processing shown in the flowchart of FIG. 14, and the same steps are denoted by the same step numbers. Steps 17-4 and 18
The point size designated by the user used in -4 is data stored in advance in the HD 10 as a parameter of the user designated point size before these processes are executed, and is input from the keyboard 12 or the mouse 13. It is a numerical value.

Next, step 1 in the "text block check process" shown in the flowchart of FIG.
In 6-4, a description will be given of a process using a numerical value designated by a user instead of a process using a parameter of twice as one of the criteria for determining whether or not to extract a text region. FIG. 19 is a flowchart showing this processing. In step 19-4, the maximum font size of the text block is determined in step 12-
It is determined whether or not a value obtained by multiplying the maximum font size of the entire text block obtained and stored in step 1 by the magnification designated by the user is equal to or larger than the value. Here, if it is determined that the size is equal to or more than the specified size, the fact that the text area has been extracted is returned (step 16-6). If it is determined that the size is not more than the specified size, the fact that the text area has been extracted is returned. (Step 16-5).

In the flow chart of FIG. 19, the processing other than step 19-4 is the same as the processing shown in the flow chart of FIG. 16, and the same steps are denoted by the same step numbers. The magnification specified by the user used in step 19-4 is the value of HD10 before these processes are executed.
Is data stored in advance as a parameter of a user-specified magnification, and is a numerical value input from the keyboard 12 or the mouse 13.

Here, the process of creating the text format in step 11-4 will be described. Note that the format created here is the same as the format of a document created by ordinary word processing software, etc., and defines the layout of the preceding document from the set position. When a page break is inserted in the editing operation, the layout of the next page is made the same. That is, regardless of the area size when the format is created, the control target area expands and contracts in accordance with subsequent insertion or deletion of characters.

The flowchart shown in FIG. 20 is a detailed processing step of “text body format creation processing” in step 11-4.

In step 20-1, the standard paper size (A) larger and closest to the image input from the scanner as a processing target represented by the image width information and the image height information of the header area 5-2 of the PAF in FIG.
3, A4, A5, B4, B5, etc., in which the identification information of the paper type and the length and width
0 has been registered. ) Is determined as the paper size of the text format and stored in the HD 10. At this time, the difference between the image width and the determined width of the standard paper and the difference between the image height and the height of the standard paper are also calculated and stored in the HD 10.

Next, in step 20-2, the left, right, top and bottom margins are determined by setting the minimum rectangle enclosing the text block group (FIG. 9) remaining for the text area as the text area (FIG. 10, frame 9-1). At this time, in consideration of the difference between the image width stored in step 20-1 and the determined standard paper width and the difference between the image height and the determined standard paper height, left, right, top and bottom margin values are calculated and stored in the HD 10. . For example,
The rectangle shown at 9-1 in FIG. 10 is the minimum rectangle that wraps the text block group for the text region, and thus this rectangular region is defined as the text region. And 9-2, 9-3, 9-4,
9-5 shows the left, right, top and bottom margins, respectively.
As the left margin value determined as the text format, a left margin + (standard paper width−image width) / 2, which is the width from the left end of the input image shown in 9-2 to the text area, is set.
Similarly, the right margin value is set to right margin + (standard paper width−image width) / 2, which is the width from the right end to the body area in the input image shown in 9-3. Further, as the upper margin value, an upper margin + (standard paper height−image height) / 2, which is the height from the upper end to the body area in the input image shown in 9-4, is set. Similarly, for the lower margin value, lower margin + (standard paper height−image height) / 2, which is the height from the lower end to the body area in the input image shown in 9-5, is set.

Next, in step 20-3, the main set direction stored in step 15-1 in FIG. 15 is stored in the HD 10 as the set direction of the text format. Further, in step 20-4, the font size of all the text blocks for the body area (the font size of the character information shown in 5-9) is checked, and the font size with the largest number is obtained.
The text size is stored in the HD 10 as the character size. Then, in step 20-5, the average character spacing of all the text blocks for the body area (the average character spacing of the line information shown in 5-8) is checked, and the average character spacing with the largest number is determined. Character size of the space and text format (step 20
-4 is stored in the HD 10), and a quotient obtained by dividing the width of the body area by the value obtained by adding the value to the number of characters / line is stored in the HD 10. Further, in step 20-6, the font is set to the Mincho font and stored in the HD 10. In step 20-7, the average line spacing of all the text blocks for the body area (the average line spacing of the OCR management data shown in 5-7) is set. ),
The average line spacing with the largest number is obtained, and the quotient obtained by dividing the height of the body area by the value obtained by adding the obtained average line space and the character size of the body format (stored in the HD 10 in step 20-4) is obtained by dividing the number of lines /
The page is set and stored in the HD 10.

Next, in step 20-8, the column relation of the text format is set. First, the number of columns is determined by dividing the text block group for the body area into rectangles that can include the largest number of text blocks without overlapping other text blocks and without overlapping other rectangles. The number of the smallest rectangles at the Y position that indicates the minimum Y coordinate value is determined and stored in the DH 10 as the number of steps in the text format. In the case of a plurality of stages, the width of each rectangle is set to the stage width, and the space between the rectangles is set between the stages. If line data exists between rectangles, it is set as a ruled line between steps. For example, 10-3, 1 shown by a dotted frame in FIG.
The text blocks for the body area determined by the above-described processing are present in three rectangles 0-4 and 10-5 and divided by each rectangle. Since the rectangle shown in 10-3 is the smallest rectangle at the Y position, in this case, one column is set. In this case, since the column is a single column, the column width is set to the same value as the width of the text region.

Next, in step 17-9, the setting for changing the middle set is performed. If the same Y-position determination as in step 20-8 is performed in a rectangle other than the rectangle set in step 20-8 and it is determined that the column changes from the middle of the page, the column change is performed. The resulting Y position, new number of steps, step width, and step spacing are set. For example, FIG.
In the case of 2, since the rectangle 10-3 is set in columns in step 20-8, when the Y positions of the remaining rectangles 10-4 and 10-5 are determined, the Y positions of these two rectangles are the same. From the two rectangles, it is determined that a column format that should be changed from the middle of the page should be set, and the Y position of the rectangle shown in 10-4 is set as the changed position. 2 rectangles in position
Since there are two steps, the number of steps is set to two steps, the step width is set to the width of each rectangle, and the space between steps is set to the space between rectangles. Also, 10-6,
The line data shown in 10-7 is stored in step 15 in FIG.
-9, stored in the memory, 10-
When the line block management data (shown at 5-5 in FIG. 5) of the line data shown at 6, 10-7 is examined, the values of the start position X / end position X are the same, and the line type and thickness are the same. For this reason, it is set as one ruled line between steps.

In the process of creating a text format from a text region text block group described above with reference to the flowchart of FIG. 20, for example, a header / footer format may be determined from a text block for a header / footer region and included in the text format. Alternatively, the page number may be set in the text format from the bearded numbers in the text block for the header / footer area.
Hereinafter, the process of setting the header, footer, and page number in the text format will be described with reference to the flowchart of the second example of the "text format creation process" shown in FIG.

However, since the processing steps from step 20-1 to step 20-9 are the same as the processing steps described in the flowchart of FIG. 20, the same step numbers are given in the flowchart of FIG. Omitted.

Next, at step 21-10, FIG.
The header format is determined by defining the minimum rectangle that wraps the header area text blocks extracted and stored in the HD 10 in the “header area text block extraction process” shown in FIG. At this time, in consideration of the difference between the image width stored in step 20-1 and the determined standard paper width and the difference between the image height and the determined standard paper height, the upper left margin value of the header area is determined. Remember. For example, the rectangle indicated by the dotted frame in 10-1 in FIG. 22 is the minimum rectangle that wraps the text block group for the header area, and thus this rectangular area is used as the header area. And 10-
8 and 10-9 are the upper left margins, respectively, so the left margin value is set to the left margin shown in 10-8 + (standard paper width−image width) / 2, and the upper margin value is , 10-
The upper margin + (standard paper height−image height) / 2 shown in FIG. 9 is set. In addition, the width and height of the minimum rectangle enclosing the header area text block are set as the size of the header area, and stored in the HD 10.

The font size of the header format is determined by examining the font size of all the text blocks in the header area (the font size of the character information shown in 5-9), finding the font size with the largest number, In the HD 10. For the number of characters / line, the average character spacing (average character spacing of the line information shown in 5-8) of all the text blocks for the header area is investigated, and the average character spacing with the largest number is determined. The quotient obtained by dividing the width of the header area by the value obtained by adding the space and the character size of the header format is set to the number of characters / line, and stored in the HD 10.

Next, in step 21-11, FIG.
The footer area is determined by setting the smallest rectangle that wraps the footer area text block group extracted and stored in the “footer area text block extraction processing” shown in FIG. At this time, in consideration of the difference between the image width stored in step 20-1 and the determined standard paper width and the difference between the image height and the determined standard paper height, the upper left margin value of the footer area is determined. Remember. For example, the rectangle indicated by a dotted frame in 10-2 in FIG. 22 is the minimum rectangle that wraps the text block group for the footer area, and thus this rectangular area is set as the footer area. And 10-10, 10-
11 shows a left margin and a lower margin, respectively. Therefore, a left margin shown in 10-10 + (standard paper width−image width) / 2 is set as a left margin value, and a lower margin value is 10
Lower margin shown at -11 + (standard paper height-image height) / 2
Set. In addition, the width and height of the minimum rectangle enclosing the footer area text block are set in the size of the footer area, and stored in the HD 10.

The font size of the footer format is determined by examining the font size of all the text blocks in the footer area (the font size of the character information shown in 5-9), finding the font size with the largest number, In the HD 10. For the number of characters / line, the average character spacing (average character spacing of the line information shown in 5-8) of all the text blocks for the footer area is investigated, and the average character spacing with the largest number is determined. A quotient obtained by dividing the width of the footer area by a value obtained by adding the space and the character size of the footer format is set to the number of characters / line, and stored in the HD 10.

Next, in step 21-12, it is checked whether a number with a "whisker" exists in the header area and the footer area. That is, "-1-", "(1)", "<
If there is a number between the beards, such as 1>"
Type of beard, existing area (header area or footer area), existing position (position from upper left of each area)
Is set to the page number of the text format and stored in the HD 10.

As described above, the format data created by the processing shown in the flowchart of FIG. 20 or FIG. 21 collectively collects data extracted from one document image and stores them in the HD 10 in association with the image. This may be stored in the PAF obtained from the image, or an address indicating the position in the HD 10 where the format data is stored may be stored in the PAF.

A text frame format is created for each text block determined to be extracted as a text frame in step 16-6 of FIG.

The flowchart shown in FIG. 23 is the “text frame format creation processing”.

In step 23-1, 5-3 in FIG.
"Text block management data" shown in the side of the block /
From the vertical size, determine the size of the text frame, HD10
To memorize. Next, in step 23-2, the left, right, top and bottom margins of the text frame format are obtained as follows, and stored in the HD 10. First, as for the left margin, the indentation of the line information shown in 5-8 in FIG. 5 is checked, and the smallest indent of all the lines in the text block is set as the left margin. The right margin is determined by examining the distance and character width from the left of the block of character information shown in 5-9 in FIG. Is the right margin. The upper margin is 5 in FIG.
Investigate the distance from the block of character information shown in -9,
The largest value among all the characters in the first line in the text is set as the upper margin. The lower margin is obtained by examining the distance and the character height from the block of character information shown in 5-9 in FIG. 5, and comparing the lower ends of all the characters on the last line in the text block.
The difference between the largest value and the distance from the lower end of the block is defined as the lower margin.

Next, in step 23-3, 5 in FIG.
-3 is stored in the HD 10 as the set direction of the text frame format. Further, in step 23-4, the font size of all the text blocks (the font size of the character information shown in 5-9) is checked, the font size with the largest number is obtained, and stored in the HD 10 as the character size of the text frame format. I do. Then, in step 23-5, the average character spacing of all the text blocks (the average character spacing of the line information shown in 5-8) is investigated, and the average character spacing with the largest number is obtained.
The quotient obtained by dividing the width of the text area (the value obtained by subtracting the left and right margins from the horizontal size of the frame) by the value obtained by adding the obtained average character spacing and the character size of the text frame format (stored in the HD 10 in step 18-4) is the number of characters / Line, and store it in the HD 10. Further, in step 23-6, the font is set to Mincho font and stored in the HD 10. In step 23-7, the average line spacing of all the text blocks (the average line spacing of the OCR management data shown in 5-7) is checked. , The most common average line spacing, and the calculated average line spacing and the character size of the text frame format (stored in HD 10 in step 18-4)
Set the quotient obtained by dividing the height of the text area (the value obtained by subtracting the upper and lower margins from the vertical size of the frame) by the value obtained by adding
10 is stored.

The text frame format creation processing from a text block extracted as a text frame described in accordance with the flowchart of FIG. 23 creates a format only for characters in a text block determined to be a text frame. For example, the line type and thickness of the frame (ruled line) in the original image in the text frame may be extracted and set as a frame format. In this case, the PAF (Page Ana
Another “frame block management data” is added to the “block management data” 5-3 of the “lysis format”.

The "frame block management data" is a structure shown in 19-2 in FIG. 24, and is similar to other block management data (text, image, line, etc.) in the distance from the left side of the sheet and the top of the sheet. , And the horizontal / vertical size of each block in the number of pixels, and a table indicating what kind of data the block holds. The data specific to the “frame block management data” has a rectangular line type and a thickness. For example, when the image data shown in FIG. 6 is divided into regions by the OCR engine, the region is divided as shown in FIG. Among them, the frame indicated by 7-9 is a frame surrounding a character group in the block determined to be a text frame in step 16-6, and the frame block is indicated by 7-10. A block extracted when the rectangular image portion in the original image is divided into regions is referred to as a frame block.

Next, in step 12-4 of FIG.
The detailed processing in the case where the "frame area block extraction processing" includes a step of also identifying frame data will be described with reference to the flowchart of "frame area block extraction processing" in FIG. However, steps 15-1 to 15-9
Since each processing step of steps 15-10 to 15-13 is the same as the processing described according to the flowchart of FIG. 15, the same step numbers are given and the description is omitted here.

If it is determined in step 15-8 that the block is not a line block, in step 25-1, the type of block (5-3, 5-4, 5-5,
The type of the “block management data” shown in 19-2) is checked, and if it is a frame block, it is extracted as frame data in step 25-2, and HD1 is extracted.
Store to 0. If it is determined in step 25-1 that the frame block is not a frame block, the process proceeds to step 15-11. The value returned in the "text block check process" in step 15-11 is determined, and if it is determined that the frame is a text frame, step 15-1 is performed.
In step 3, the text block is extracted as a text frame, stored in the HD 10, and in step 25-3, to create the format of the text block,
Call "text frame format creation process". FIG. 26 is a flowchart showing details of the "text frame format creation process" executed in step 25-3.

In step 26-1, it is checked whether there is a frame block including the text block (the block stored in the HD 10 as the frame data in step 25-2 in FIG. 25) (for example, 7 in FIG. 7).
The text block indicated by -9 is included in the frame block indicated by 7-10). If there is a frame block including the text block, step 26-
If the determination is made in step 1, in step 26-3, the size of the text frame is determined from the block horizontal / vertical size of “frame block management data” shown in 19-2 of FIG. Is determined from the line type / thickness of "." And stored in the HD 10.

Next, in step 26-4, the left, right, top and bottom margins of the text frame format are obtained from the management data of the text block and the frame block containing the text block as follows, and stored in the HD 10. First, the left margin is the difference between the distance from the left of the “frame block management data” shown in 19-2 in FIG. 24 and the distance from the left of the “text block management data” shown in 5-3 in FIG. Is the left margin. The right margin is the value obtained by adding the block horizontal size to the distance from the left of the “frame block management data” of 19-2 in FIG. 24 and the left of the “text block management data” in 5-3 in FIG. The difference from the value obtained by adding the block horizontal size to the distance from is set as the right margin. The upper margin is 1 in FIG.
The difference between the distance from the sheet of “frame block management data” shown in 9-2 and the distance from the sheet of “text block management data” shown in 5-3 in FIG. 5 is defined as an upper margin.
The lower margin is a value obtained by adding the block vertical size to the distance from the sheet of “frame block management data” shown in 19-2 in FIG. 24 and the “text block management data” shown in 5-3 in FIG. The difference from the value obtained by adding the vertical size of the block to the distance from is set as the lower margin.

Next, at step 26-5, 5 in FIG.
-3 is stored in the HD 10 as the set direction of the text frame format. Further, in step 26-6, the font size of all the text blocks (the font size of the character information shown in 5-9) is investigated, the font size with the largest number is obtained, and stored in the HD 10 as the character size of the text frame format. I do. Then, in step 26-7, the average character spacing of all the text blocks (the average character spacing of the line information shown in 5-8) is checked, and the average character spacing with the largest number is obtained.
The calculated average character spacing and character size of the text frame format (character size stored in HD 10 in step 22-6)
The horizontal size of the text block (5
The quotient obtained by dividing the block horizontal size of the text block management data shown in -3) is set to the number of characters / line, and stored in the HD 10. Further, in step 26-8, the font is set to Mincho font and stored in the HD 10, and in step 26-9, the average line spacing (5-7
The average line spacing of the OCR management data shown in (1) is checked, the average line spacing with the largest number is obtained, and the obtained average line spacing is added to the character size of the text frame format (the character size stored in the HD 10 in step 26-6). Then, the quotient obtained by dividing the block vertical size of the text block (the block vertical size of the text block management data shown in 5-3) is set to the number of lines / page, and stored in the HD 10.

If there is no frame block containing the text block in step 26-1 (the block stored in the memory as the frame data in step 25-2 in FIG. 26), the process proceeds to step 23-1. . Since each processing step shown in steps 23-1 to 23-7 is the same as each processing in the flowchart of FIG. 23, the same step numbers are given and the description thereof is omitted here. However, in step 23-1, in this case, since there is no frame block including the text block,
The frame is stored in the HD 10 as “none”.

As described above, the extraction processing of each block (each processing step shown in the flowchart of FIG. 12) is performed on the image sample shown in FIG. 3, and the size and position of each area as a result of the completion are shown. FIG. 27-1
Is the header area, and the area 27-2 is the footer area. The area indicated by 27-3 is a text frame area, and the area indicated by 27-4 is a line area. 2
Each area shown in 7-5, 27-6, and 27-7 is a text area.

Here, as shown in FIG. 27, the frame is an area which can be set so as to overlap the body area (in FIG. 27, the text frame area 27-3 is the body area 27-6 and the body area). 27-7), the text (text) input to the text area is arranged avoiding the frame. That is, characters are sequentially arranged in the body region other than the region defined as the frame region.

As described above, each created format is stored in association with the PAF, and is displayed on the display 16 or printed from the printer 18 in response to a call instruction of the keyboard 12 or the mouse 13. A font corresponding to the character code of the data 5-6 in the text block is arranged according to the format, or the line block management data 5-5
Draws a line according to the data of and outputs it. As a result, it is possible to output a document with the same layout as the image data input from the scanner 1B, but in a state in which characters and graphics have been copied, and to edit the document on the display 16 by the instruction of the keyboard 12 and the mouse 13. . Here, even when a new character is inserted into the text block, the characters are rearranged in a manner defined as the format, so that the document desired by the operator is obtained in the same layout and format as the input image. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a character recognition document processing device according to an embodiment of the present invention.

FIG. 2 is a diagram in which a character recognition document processing device and an image reading device according to an embodiment of the present invention are connected.

FIG. 3 is a diagram showing an image image.

FIG. 4 is a diagram in which an image is divided into regions.

FIG. 5 is a diagram showing a file format of Page Analysis Format (PAF).

FIG. 6 is a diagram showing an image image.

FIG. 7 is a diagram obtained by dividing an image into regions.

FIG. 8 is a diagram illustrating a text block near a header and a footer area.

FIG. 9 is a diagram showing a text block in a text region.

FIG. 10 is a diagram showing a state where the text region is divided into other regions.

FIG. 11 is a flowchart illustrating “separation processing between a text region and another region”.

FIG. 12 is a flowchart illustrating “separation processing between a text block and another block”;

FIG. 13 is a flowchart illustrating “extraction processing of a header area text block”.

FIG. 14 is a flowchart showing “footer area text block extraction processing”.

FIG. 15 is a flowchart illustrating “frame area block extraction processing”;

FIG. 16 is a flowchart showing a “text block check process”.

FIG. 17 is a flowchart illustrating “extraction processing of a header area text block”.

FIG. 18 is a flowchart showing “footer area text block extraction processing”.

FIG. 19 is a flowchart showing “text block check processing”.

FIG. 20 is a flowchart illustrating a first example of “text format creation processing”;

FIG. 21 is a flowchart illustrating a second example of “text format creation processing”.

FIG. 22 is a diagram for describing block division when creating a text format.

FIG. 23 is a flowchart showing “text frame format creation processing”.

FIG. 24 is a diagram showing a second example of the data structure of the PAF.

FIG. 25 is a flowchart illustrating a second example of “frame area block extraction processing”.

FIG. 26 is a flowchart illustrating a second example of “text frame format creation processing”.

FIG. 27 is a diagram exemplifying each divided area.

FIG. 28 is a network system diagram.

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference)

Claims (61)

[Claims] 1. A character string region is extracted from input image information, and it is determined whether or not the character string region is a character string region of a text according to the layout information of the extracted character string region. Image processing method. 2. The image processing apparatus according to claim 1, wherein an area determined to be a character string area of the body and an area determined to be not a character string area of the body are stored separately. Method. 3. The image processing method according to claim 1, wherein the storage stores feature information extracted from each area. 4. The image processing method according to claim 1, wherein the storage is performed by adding identification information capable of identifying each of the storages. 5. The layout information of a character string region is a positional relationship with a different region.
The image processing method according to 1. 6. The image processing method according to claim 1, wherein the layout information of the character string area is the number of lines of a character string included in the character string area. 7. A plurality of image areas are extracted from input image information, and classified into a text area and other areas from the extracted image areas, and a first storage area and a second storage area are classified according to the classification. An image processing method characterized by storing information extracted from each area divided into areas. 8. The image processing method according to claim 7, wherein the image information is an image read by a scanner. 9. The image processing method according to claim 7, wherein the image information is an image input from another terminal connected via a line. 10. The image processing method according to claim 7, wherein the image information is an image read by a scanner at another terminal connected via a line. 11. A format for analyzing input image information to determine a character arrangement, and arranging characters obtained by recognizing the input image information according to the generated format. Document processing method. 12. The image processing method according to claim 11, wherein the image information to be analyzed is a text string region extracted from an input image. 13. The image processing method according to claim 11, wherein the image information to be analyzed is one of a plurality of regions obtained by dividing input image information. 14. The image processing method according to claim 11, wherein the format defines an arrangement of character strings in an area extracted from an input image. 15. The image processing method according to claim 11, wherein a document in which the characters are arranged is displayed on a display screen, and editing is performed on the display screen. 16. The image processing method according to claim 11, wherein a document in which the characters are arranged is displayed on a display screen. 17. The image processing method according to claim 11, wherein the document in which the characters are arranged is printed by a printer. 18. The image processing method according to claim 11, wherein the characters to be arranged are fonts. 19. The image processing method according to claim 11, wherein the document in which the characters are arranged is edited, and the edited document is rearranged according to a format in which the characters are arranged. 20. The image processing method according to claim 11, wherein the format includes column information. 21. The image processing method according to claim 11, wherein the format includes a paper size. 22. The image processing method according to claim 11, wherein the format includes margin information. 23. The image processing method according to claim 11, wherein the format includes a combination direction. 24. The image processing method according to claim 11, wherein the format includes a character size. 25. The image processing method according to claim 11, wherein the format includes information on the number of characters per line. 26. The image processing method according to claim 11, wherein the format includes information on the number of lines per area. 27. The image processing method according to claim 11, wherein the format includes typeface information. 28. A method of extracting a plurality of image areas from input image information, selecting a frame area from the plurality of extracted image areas, and creating a frame format from the image information of the selected frame area. Image processing method. 29. The image processing method according to claim 28, wherein the frame area is arranged so as to overlap another area. 30. The image processing method according to claim 28, wherein the frame area is a text area in a set direction different from a set direction of characters in a body area. 31. A character string extracting unit for extracting a character string region from input image information, and the character string region is a character string region of a body according to layout information of the character string region extracted by the character string extracting unit. An image processing apparatus, comprising: a determination unit configured to determine whether the image processing is performed. 32. The apparatus according to claim 31, further comprising storage means for distinguishably storing an area determined to be a character string area of the text and an area determined not to be a text string area of the text. The image processing apparatus according to any one of the preceding claims. 33. The image processing apparatus according to claim 31, wherein said storage means stores feature information extracted from each area. 34. The storage device according to claim 31, wherein said storage means stores identification information capable of identifying each of them.
An image processing apparatus according to claim 1. 35. The layout information of the character string area includes:
The image processing apparatus according to claim 31, wherein the image processing apparatus has a positional relationship with different areas. 36. The layout information of the character string area includes:
32. The image processing apparatus according to claim 31, wherein the number of lines of the character string included in the character string area is set. 37. An extracting unit for extracting a plurality of image regions from input image information, a classifying unit for classifying the image regions extracted by the extracting unit into a text region and other regions, and An image processing apparatus comprising: a storage unit that stores information extracted from each of the first storage area and the second storage area according to the classification. 38. The image processing apparatus according to claim 37, further comprising a scanner that reads the image information. 39. The image processing apparatus according to claim 37, further comprising control means for controlling input of the image information from another terminal connected via a line. 40. The image processing apparatus according to claim 37, further comprising control means for controlling the other terminal connected via a line to read the image information by a scanner. 41. A format creating means for analyzing input image information to create a format for determining an arrangement of characters, and a character obtained by recognizing the input image information in accordance with the format created by the format creating means. A document processing apparatus having control means for controlling the placement of a document. 42. A body character string region extracting unit for extracting a body character string region from the input image information, wherein the format creating unit is configured to format the body character string region extracted by the body character string region extracting unit. 43. The image processing apparatus according to claim 42, wherein: 43. An area dividing unit that divides the input image information to obtain a plurality of areas, and wherein the format creating unit creates each format of the plurality of areas obtained by the area dividing unit. 43. The image processing device according to claim 42, wherein: 44. The image processing apparatus according to claim 42, wherein the format defines an arrangement of a character string in an area extracted from an input image. 45. The image processing apparatus according to claim 42, further comprising: display means for displaying a document on which the characters are arranged on a display screen; and editing means for performing editing specified on the display screen. apparatus. 46. The image processing apparatus according to claim 42, further comprising display means for displaying a document in which the characters are arranged. 47. The image processing apparatus according to claim 42, further comprising a printer for printing a document in which the characters are arranged. 48. The image processing apparatus according to claim 42, wherein the arranged characters are fonts. 49. An editing unit for editing a document in which the characters are arranged, wherein the control unit controls to rearrange the document edited by the editing unit in accordance with the format in which the characters are arranged. The image processing apparatus according to claim 42, wherein 50. An image area extracting means for extracting a plurality of image areas from input image information; a frame area selecting means for selecting a frame area from the plurality of image areas extracted by the image area extracting means; An image processing apparatus comprising: a frame format creation unit that creates a frame format from image information of a frame area selected by the selection unit. 51. The image processing apparatus according to claim 50, wherein the frame area is arranged so as to overlap another area. 52. The image processing apparatus according to claim 50, wherein the frame area selecting means selects a text area in a set direction different from the set direction of characters in the body area as a frame area. 53. A storage medium readable by a computer, comprising: a control program for extracting a character string area from input image information; and a character string area according to layout information of the extracted character string area. And a control program for determining whether or not the character string area is a character string area. 54. A control program for storing an area determined to be a character string area of the text and an area determined not to be a text string area of the text separately. Item 54. The storage medium according to Item 53. 55. A storage medium readable by a computer, comprising: a control program for extracting a plurality of image areas from input image information; and a text area and other areas from the extracted image areas. A storage medium storing a control program for classifying, and a control program for storing information extracted from each of the first storage area and the second storage area according to the classification. . 56. A storage medium readable by a computer, comprising: a control program for analyzing input image information to create a format for determining a character arrangement; and the input image according to the created format. A storage medium storing a control program for arranging characters obtained by recognizing information. 57. A control program according to claim 56, wherein a control program for displaying a document in which the characters are arranged on a display screen and a control program for editing on the display screen are stored. Storage medium. 58. The storage medium according to claim 56, wherein a control program for displaying a document in which the characters are arranged on a display screen is stored. 59. The storage medium according to claim 56, wherein a control program for printing a document in which the characters are arranged by a printer is stored. 60. A control program for editing a document in which the characters are arranged, and a control program for rearranging the edited document in accordance with a format in which the characters are arranged. 61. The storage medium according to item 56. 61. A computer-readable storage medium, comprising: a control program for extracting a plurality of image areas from input image information; and a control for selecting a frame area from the plurality of extracted image areas. A storage medium storing a program and a control program for creating a frame format from the image information of the selected frame area.