JPH07107711B2 - Document image processing device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing system, and more particularly to an image understanding system suitable for an apparatus that electronically changes the layout of an input document image and outputs the image.

[Background of the Invention]

The conventional electronic document image file device merely stores each page of a document as an image, and secondary information for retrieval needs to be provided externally by a code input means such as a keyboard. In addition, in order to make it easier to see, you can edit or modify the layout of the document, and in order to extract in advance an index image that reminds you of the content of the document, you can use a coordinate input means such as a crosshair cursor or mouse to display the image. It was necessary to give the area from the outside. However, in order to save the labor of inputting / editing files, it is desirable to automatically read the titles and authors described in the document to generate the secondary information.
In order to further improve the search, it is necessary to use captions for figures and tables, automatic input of chapter / section titles, or automatic keyword extraction by recognizing the text itself. Also, divide the image of the target document into parts such as titles, authors, abstracts, text, figures, and photos, recognize and encode the character parts, and extract the necessary areas from multiple documents. The creation of abstract journals was also requested to reduce the recording space and diversify search units.

In the prior art, in order to deal with such a problem, a grammar has been proposed in which a target area in a document is designated from the outside, an image is extracted, or a character recognition device automatically converts it into character data. There is.

IPSJ 28th National Convention Lecture Paper "A Search Method for Document Image Files Using Image Processing" (Proceedings 5C-
1, 1984) Patent publication gazette No. 60-17565 "image memory retrieval device"
(Published January 29, 1985) However, this method has a problem that it takes a lot of labor to specify the search target area.

In order to deal with the problem of the manual designation method, there is a method of automatically extracting a target area by performing image processing, or understanding the content of the document and processing the document based on the understanding result.

IPSJ 29th Annual Conference Lecture Paper "Algorithm for newspaper headline extraction for information systems for document images"
(Proceedings 6M-9, 1984) Patent Publication No. 39065, "A device for detecting character strings in mail" (published on September 20, 1984) IPSJ 23rd National Convention Lecture Paper "Complex "Automatic analysis of structured document images" (Papers 6C-2, 1981) However, these document processing technologies are methods that simply process image data (bottom-up method), so the extracted areas have meaning. Although it is difficult to give it, or the method of processing image data after giving meaning to the area in advance (top-down method) is adopted, since the processing procedure is the method described in the control program, Changing the type of documents processed has never been easier.

In other words, since these document understanding technologies are targeted at newspapers and mail destinations, they are standardized to some extent like journals and patent gazettes, but their formats vary widely. On the other hand, it is not necessarily suitable for the purpose of efficiently extracting secondary information such as titles and authors necessary for search. Further, there is no suitable means for improving the extraction method when the secondary information extraction fails.

Regarding the method of outputting an image, the conventional system mainly based on characters (TeX) and the system mainly based on graphics (GKS)
and so on. The former aims at automating the selection of print type. Since the latter is intended to synthesize figures and images, it is not considered to understand a document as an image and then output an image corresponding to the content. Regarding the method of arranging the images, in the above system, the coordinate system can be arbitrarily set in order to match the resolution of the display device. Also, the position on the screen can be arbitrarily set by a rectangular area which is a viewport. However, coordinate systems or viewports cannot be represented hierarchically.

Furthermore, there is no suitable method for rearranging the image area to which the semantic information is added.

[Object of the Invention]

An object of the present invention is to provide a separation image processing apparatus capable of automatically analyzing a document structure from an image layout of a document that is almost standardized and obtaining a document image rearranged in a desired layout. To provide.

[Outline of Invention]

In order to achieve such an object, the processing apparatus of the present invention comprises means for inputting an unknown document as an image and converting it into a digital image, and arrangement of each rectangular area by a notation describing the image as a set of a plurality of rectangular areas. And the size are expressed by including the variables, and thus the first storage area for holding the first format data defining the format common to the document to be processed and the layout of the image to be output are expressed by the above notation. Storage means having at least a second storage area for holding the defined second format data, a third storage area for holding the digital image, and a fourth storage area for storing the image to be output. , The first format data in the first storage area is referred to, each rectangular area described in the first format data is searched on the digital image in the third storage area, and thus the unknown document is obtained. of Structure analysis processing for specifying the existing regions of the partial images corresponding to the respective format elements of the expression, and using the existing areas specified by the structural analysis processing, the second format data is added to the respective format elements described in. A control unit that executes image processing of cutting out corresponding partial images from the third storage area and transferring the partial images to a position defined by the second format data in the fourth storage area. , And an image output unit for displaying a set of partial images accumulated in the fourth storage area. In the above notation, the document image is expressed as a set of rectangular areas. In particular, the first format data corresponds to each format element (for example, when a technical paper consists of a title, an author's name, a text, and a drawing, each is referred to as a format element) in order to define a format common to documents to be processed. The variable includes the absolute or relative size of the rectangular area on the document image and the quantity indicating the absolute or relative relationship between the rectangular areas. Also, a search method for a rectangular area can be specified. Further, the rectangular area is also expressed as a set of rectangular areas, and by such a hierarchical expression, the document format can be expressed in detail.

When an unknown document is input in the syntactic analysis process, a rectangular area is searched according to the search method specified by the first format data, information on whether or not the search is successful, and a parameter (rectangular area) determined at the time of search. The absolute or relative size of and the absolute or relative relationship between the rectangular regions) is extracted. The syntactic analysis unit substitutes the numerical values of the above parameters into the variables in the above first format data, and performs the following analysis, thereby sequentially advancing the structural analysis of the document. In the image processing after the analysis is completed and the content of the image is understood, the image is again arranged and output according to the second format data. Therefore, the second storage area stored in the second storage area is
An image can be output in a different format by changing the content of the format data of.

The principle of the present invention will be described below. FIGS. 1 and 2 show examples of pages of technical papers having a substantially constant format. In the following explanation, a technical paper will be taken as an example of the target, but even for other documents, a common format can be specified for the target document by changing the contents of the format data or part of the expression method. The present invention can be applied, and the present invention is not limited to an example of the above technical paper. Figure 3 shows
2 It is an output image in which the contents of the figure are understood and the title, author name, and representative figure are changed to a layout with a tabular structure.

Next, an example of an expression method for describing the structure of a document (hereinafter, abbreviated as document structure expression) is shown.

(Defform F (form F1 (10 90 10 40)) (form F2 ………) (form F3 ………)) (defform F1 (form F11 (10 90 10 50)) (form F12 (10 90 60 90)) )) (Defmac LINE-1 (% 1) (point? Y1 (mode IN Y LESS)) (point? Y2 (mode OUT Y LESS)) (form% 1 (0? W? Y1? Y2))) The structural representation will be described with reference to the example of the document image in FIG.

The first defform F ... indicates that the form F is formed by additionally forming the form elements F2 and F3 side by side under the form element F1 as shown in FIG. In FIG. 4, the portions of F1, F2 and F3 corresponding to FIG. 5 are shown surrounded by broken lines. The four numbers 10 90 10 40 sandwiched by () next to the format element name F1 indicate the position of the area of the format element F1 when the entire area corresponding to the format F is 100 × 100. Here, the coordinate system has the upper left as the origin. The numerical value indicating the area is the minimum value of X coordinate, X
It is the maximum value of the coordinates, the minimum value of the Y coordinates, and the maximum value of the Y coordinates. When the parameter value is known as in this example, the value may be directly described. Similarly, the format elements F2 and F3 are also described in rectangular areas.

In the next defform F1 ..., the format element F1 is
It shows that F11 and F12 are vertically arranged. That is, the Y-direction area of the format element F11 is 10 to 50, and the Y-direction area of the format element F12 is 60 to 90. Format element F
The position of the area of 11 and the format element F12 is described in the coordinate system with the upper left of the format element F1 as the origin. Therefore, the format F has a relative coordinate system.

In this way, the image can be generalized by expressing the format in the rectangular area and hierarchically expressing the area as a set of the areas. Of course not hierarchical,
As shown in FIG. 6, it may be described in the absolute coordinate system with the format F as a reference. In this case, in order to specify the same rectangular area as in FIG. 5, the following may be done.

(Defform F (form F11 (18 82 13 25)) (form F12 (18 82 28 38)) (form F2 ………) (form F3 ………)) Next defmac LINE-1 (% 1) , Is a macro definition. The following three lines, which are the main body of this macro definition,
This expresses that the first line from the top of the rectangular area is the format element% 1.

(Point? Y1 (mode IN Y LESS) (point? Y2 (mode OUT Y LESS)) (form% 1 (0? W? Y1? Y2)) where? W is the horizontal size of the format element, ? H represents the vertical size of the format element, and? Y1,? Y2 are variables specified by the search as described below.

point means to search for a point that satisfies a certain condition and substitute it into a variable. The search condition is specified by mode. IN−
OUT indicates whether the search point is a change point from white to black or a change point from black to white, Y indicates a search axis (X or Y), and LESS indicates a search direction. Note that although it does not exist in this example, point
There is an area corresponding to. This indicates the area of the search range as described later.

The search method will be described with reference to FIG. 7 by taking the description of the macro definition as an example. (A) indicates that lines such as Title ..., Author ... are present in the format. Coordinate values of these rows in the Y direction,
That is, the first line exists from? Y1 to? Y2 and the second line exists from? Y.
(B) describes that it exists from 3 to? Y4
And (C). As described above, (B) is a macro that defines that the format element in the first line is% 1,
(C) is a macro that defines that the format element on the second line is% 1. The variable with% is a temporary variable and is executed after being replaced by the argument when the macro was called. Therefore, you can call these macros as follows.

(LINE-1 F1) (LINE-2 F2) That is, the format element name on the first line is F1, and the format element name on the second line is F2. The search condition for the coordinate value? Y1 designated by point in the second line in (B) is IN Y LESS. Therefore, the condition is that the change point from white to black, the search axis is Y, and the direction is LESS, that is, the search is performed from the one having a smaller Y coordinate value. When searching from the one with the largest Y coordinate value, GR
It should be EATER. 1 that satisfies these conditions
The uppermost coordinate value of the line is? Y1. Point on the 3rd line of (B)
It is sufficient to describe that the lower limit coordinate value? Y2 in the first line designated by is the changing point from black to white in the above search condition. That is, the search condition of? Y2 is OUT Y LESS.

Next, (C) which defines the second line in the format will be described. Since the second line is the line following the first line, the lower limit of the first line
? Y2 is searched, and? Y3 indicates the area of the search range by area. That is, by setting the rectangular area to be searched as 0? W? Y2? H, the same search as in (B) can be performed from the lower limit of the first line.

In the document structure analysis processing, the first format data written in the above representation method is referred to, and it is sequentially checked whether or not the rectangular area described therein exists in the document. When a rectangular area including a variable is searched for, a numerical value of the variable is obtained, and thereafter, the numerical value is substituted for the variable and used.

Next, the calculation between the rectangular areas will be described. In an actual document, an area having a shape other than a rectangle also appears. FIGS. 8A and 8B are examples of regions having shapes other than rectangles. Further, (C) shows an example in which one rectangular area is separated into two rectangular areas. 8 (A) and 8 (B) can be considered as the sum or difference of two rectangular areas, as indicated by broken lines. Further, in (C), if it is considered that two rectangular areas are connected to each other to virtually form one rectangular area, the expression becomes simple. In order to enable calculation between such rectangular areas, virtual transfer of areas is defined as follows.

(Map & form F (space? W? H) (position ((? X0? Y0) (? Xmin? Xmax? Ymin? Ymax)) (...))) Figure 9 shows the meaning of this definition. . space
Indicates that a rectangular area of width? W and height? H is newly set as the format F, and that transfer is performed in this area. position
Represents the upper left coordinates of the rectangular area of the transfer destination. The rectangular area of the transfer source indicated by four values (? Xmin? Xmax? Ymin? Ymax) is copied to the above transfer destination.

This virtual transfer will be specifically described with reference to FIG. It is assumed that the actual format to be analyzed is arranged as shown in (A). This is called a multi-column or double column. Format element F1 and format element F2 are
Although they are spatially arranged side by side, they are arranged vertically vertically as shown in (B). The operation between such rectangular areas can be expressed by (map & form F (space 50 60) (position ((10 10) (10 40 10 40)) ((10 40) (10 70 10 30)))) . In the virtual format shown in (B), a rectangular area having a width of 50 and a height of 60 is set by the space shown in (A). Then, the relationship between (A) and (B) is expressed as (position ((10 10) (10 40 10 40)) ((10 40) (40 70 10 30))). Rectangular area in (A) (10 40 10
40) is transferred to the area whose origin is (10 10) in (B).

By combining the virtual transfer described above, a region having a complicated shape as shown in FIG. 8 can be expressed by an operation between two or more rectangular regions. For example, the 8th
The figure (A) can be expressed as a case where two rectangular areas having different sizes are adjacently transferred.

Next, the contents of the image processing for rearranging the images and displaying them in a desired layout, especially the image transfer method when the aspect ratio is changed will be described. FIG. 11 shows a case where the scale of an image is changed and transferred according to the aspect ratio of the rectangular area. In (1), it is assumed that the width of the rectangular area of the input image is W, the height is H, and the image of the area is the pattern of the character A. (2) shows the case where the width of the rectangular area of the transfer destination is W ', (3) the height is H', and (4) shows the case where the width is H'and the height is W '. By the structure analysis process, the variable of the first format data is replaced with a numerical value, and the existence area of the partial image corresponding to a certain format element is specified as shown in (1) of FIG. 11,
In the image processing, the partial image is cut out and transferred to the rectangular area in the second storage area. If the rectangular area defined in the second format data has a different shape from the original existing area as shown in Fig. 11 (2), (3) or (4), the image, that is, the character pattern is corrected accordingly. Will be transferred. 12th
The figure shows a case where the output image is transferred without changing the aspect ratio. The input image of (1) is divided into xyz inside the rectangular area of the transfer destination as in (2). (3) is x = 0, (4)
Is y = 0, (5) is x = y, and (6) is an arbitrary division ratio.

As can be seen from the above description, in the document structure representation proposed by the present invention, the structure of a document is grasped as a combination of rectangular areas and the relationship between the rectangular areas is described, so that the expressive power of the document increases and It is possible to describe a target that has been difficult to handle in the past, such as when the number of lines of is uncertain or when the appearance of a rectangular area is uncertain. Therefore, it is possible to analyze and rearrange various documents.

Example of Invention

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 13 is a block diagram showing the configuration of an apparatus that employs the image understanding method according to the embodiment of the present invention. In the present embodiment, an image to be understood will be described for a document, but the present invention can be applied even when a general image, that is, a grayscale image such as a figure or a photograph is included. Each unit of the device is connected to the bus 1, and the entire operation is controlled by the control unit 2. Information on the document 3 (document image information) is photoelectrically converted by the scanning unit 4.
It is digitalized into a digital image and stored in the memory 61 via the bus 1. The memory 61 will be described later 62, 63,
It forms a part of the memory 6 together with 64, 65, 66 and 67. Instead of obtaining the digital image 61 from the scanner 4, it may be read from a digital image file device such as an optical disc. Alternatively, the character code information may be obtained from the input unit 5 and the image pattern corresponding to the code may be read from the file 8. In the following description, it is assumed that one pixel is binarized into one bit, but one pixel may be represented by multiple values, or color information may be provided by photoelectric conversion by a color scanner. The memory 62 stores a normalized image obtained by performing known position correction processing, tilt correction processing, and the like on the document image by the control unit 2.

It is assumed that the format data common to the target document written according to the document structure expression including the variables described above is stored in the memory 64 in advance. The control unit 2 uses the format data to perform the structure analysis process of the normalized image. Here, the structural analysis processing is to decompose the normalized image into a plurality of rectangular areas, search each rectangular area by referring to the format data stored in the memory 64, and search the variables included in this format data according to the search result. It means replacing with a numerical value. Of the areas obtained as a result of the structural analysis processing, the area of the area that is predetermined as the recognition target area is sent to the character / graphic recognition unit 7 to recognize the internal character / graphic pattern. Generally, the original document image has a complicated shape, but since the area obtained as a result of the document structure analysis processing has a rectangular shape, it is possible to easily cut out and recognize characters and figures by a known method. A character code string obtained as a character recognition result, a character code string obtained by editing the character code string, or vector data or a symbol string obtained as a figure recognition result can be used as search information for a designated area. An image pattern corresponding to the search information can be read from the file 8 and used as a pattern to be rearranged. The search information of the input document obtained as described above is output to the file 9, and the digital image of the rearranged document is output to the file 8. When outputting the digital image of the document to the file 8, it may be separately output in units of a plurality of decomposed rectangular areas. Further, the file 8 and the file 9 may be the same.

Further, the format data of the output document to be displayed written according to the above-mentioned notation is stored in the memory 65 in advance. However, the format data stored in this memory 65 is
It does not include variables, that is, the size and arrangement of the rectangular area corresponding to each format element is specified and specified numerically. The control unit 2 uses the format data to perform the image processing of the normalized image. Here, the image processing refers to the second format data of the memory 63 for holding the output image by cutting out the partial image corresponding to each format element described in the second format data from the regular image. It refers to the process of storing in each specified area. That is, a plurality of rectangular areas are combined again and stored in the memory 63. Then, the image output unit 10 displays the output image.

The details of the document parsing process are described below. 14 and 15 are diagrams for explaining the flow of document understanding processing.
The process flow is written in PAD (Program Analysis Diagram) format. The contour of the document image is extracted at 100 and stored in the memory 66. A known method may be used for the contour extraction. A so-called connected area extraction method may be used instead of the contour extraction. The maximum and minimum values of the X and Y coordinates of each contour i extracted in 200, Xmin (i) Xmax (i)
Ymin (i) Ymax (i) is extracted. The circumscribed rectangle of the contour i is obtained from these four numerical values. 300, 400, and 500 are initialization, main body, and end determination of the syntax analysis process, respectively.

In 300, the format data stored in the memory 64 is copied to the work memory 67, and various tables and program internal variables are initialized.

The main body 400 of the parsing process is composed of 410 to 460. Four
The control unit 10 controls the processing from 420 to 450 to be repeated until the end determination is made in 460. At 420, the statement in the format data is retrieved. An unprocessed statement is a line in which some of the variables contained in it have undetermined values, or the corresponding document area has not yet been determined. Reference numeral 430 is a judgment to skip the processing of 440 when there are no unprocessed statements left. In this case, the end determination is made. 420
If the statement fetched in step is an unprocessed statement, 440 processing is performed. Reference numeral 440 is a portion that determines the type of statement and branches, and the processing content changes depending on the type of state met. 14 and 15 and the following description, only the case of the form statement, that is, (form F0 (? Xmin? Xmax? Ymin? Ymax) (shrink? X? Y)) is described. Statement-specific processing is performed.

FIGS. 441 to 448 are parts for processing the predicate form. 441
Then, check whether the format name F0 has been registered, and if it is not registered,
Register F0 in the format table with 442. In 442, variable name? X
Check whether the character string written at the position of min,? Xmax,? Ymin,? Ymax,? X,? Y is a variable or a numerical value, and if it is a variable, it has been registered.
If not registered, register these in the variable table. If the variable is already registered, the value is checked to see if it is fixed. If not, the form process ends (in this case, this statement is unfinished). If confirmed, rewrite the variable name in the statement with the above numerical value.

As a specific example, when? Xmin = 0,? Xmax = 90,? Ymin,? Ymax: unregistered? X = 5,? Y = 5, the above statement is (form F0 (0 90? Ymin? Ymax) (Shrink 5 5)) is rewritten, and the variables? Xmin and? Ymax are registered in the variable table, and the values are undetermined.

At 443, it branches depending on whether all the variable names in the statement have been rewritten to numerical values, and when all have been rewritten to numerical values, the form execution processing of 444 is performed. Details of form execution processing are represented by 445 to 448. Reference numeral 445 indicates that the following processing is repeated for the contour i extracted in 200.
In 446, the minimum value and the maximum value Xmin (i) Xmax (i) Ymin (i) Ymax (i) of the X coordinate and the Y coordinate of the contour i are set in the variable? Xmin? Xmax? Ymin? Ymax? X? Y in the statement. ? Xmin <Xmin (i) <Xmax (i) <? Xmax? Ymin <Ymin (i) <Ymax (i) <? Ymax? X <Xmax (i) -Xmin (i)? It is determined whether or not the contour satisfies Y <Ymax (i) -Ymin (i). In 447, when the above condition is satisfied, the contour i is registered in the F0 component table. In 448, when there is no contour satisfying the above conditions, an analysis failure flag is set.

As described above, by the processing of 441 to 448, it can be detected that the structure corresponding to the statement form in the format data exists in the input image. The same applies to statements other than form. In the case of form, there is no output data, but depending on the statement, there are also those that substitute the parameter obtained at the time of analysis into the variable in the statement,
The result is used in other statements.

At 450, the parsing failure flag is checked and if the parsing fails, it goes back and tries again. In this case, the variable in which the parameter is substituted in the parsed statement is rewritten to the previous state, and control is performed to search another possibility.

At 460, it is detected whether or not the analysis failure flag is set or whether there is the analysis failure flag after the backward return retry, and the end determination is performed.

Reference numeral 500 is a part for passing the data obtained as a result of the analysis to the outside. The data to be transferred to the outside includes the coordinates on the document of the rectangular area detected corresponding to the format name.

If parsing fails for a statement with the parsing-failed flag set, this document is incomprehensible and will be rejected. For example, the final result or intermediate result of document understanding is displayed on the console 11 and is corrected man-machinely.

Next, the contents of the form execution process will be specifically described with reference to FIG. Fig. 16 (A) shows noise components and characters 1, A,
2 shows the case where B component is present.

(B) shows the execution parameters of the form statement, (form F (20 80 10 50) (shrink 0 0)) (C) shows the execution parameters of the form statement, (form F (20 80 10) 50) (shrink 5 5)). As shown in the figure, in the component table of format F, in the case of (B), the noise component and the character 1 and A components are registered, and in the case of (C), the character 1 and A components are registered, but the noise component The component is not registered by the shrink specification and is removed. In addition, after executing the form for the rectangular area of the format F, the area can be normalized by the character components included in the area as shown in the figure, and the area size can be flexibly changed according to the content of the image. Can be specified.

FIG. 17 concretely describes the method of selecting the contour component when executing the above form. FIG. 17 (A) shows a circumscribed rectangle as a result of processing the contour image shown in FIG. 16 (A) according to FIG. 14 200. That is, 5 is a noise component, 1-8 is a character component, and 6-8 is a so-called inner contour. Of these ingredients
Xmin, Xmax, Ymin and Ymax are shown in (B). Whether or not it is included in the format F is 20 <Xmin (i) <Xmax (i) <80 10 <Ymin (i) <Ymax (i) <50 5 <Xmax (i) -Xmin (i) 5 <Ymax (I) -Ymin (i) is determined based on whether or not it holds. In this example, the contours i = 1 and 3 hold. Since the character component of 3 includes the component of 6, it may be omitted from the format F.

Next, the details of the image processing for rearranging and displaying the images based on the result of the parsing processing will be described. FIG. 18 shows an outline of a case where partial images are arranged. The image pattern corresponding to the character code information ABCD is arranged as shown in (2). In this example, the width of the character pattern is W,
The height is H and the character spacing is S. Then, as shown in (2), the partial image of (1) is transferred to the format 3 defined by the format 2. This scale is determined by the ratio of the rectangular area of the format as described above. Further, as a function of transferring an image, reduction, enlargement, rotation, affine conversion, etc. may be included. The character pattern is stored as contour data or pixel data as shown in FIGS. 19 (1) and 19 (2). 20th
The figure describes the features of the contour data shown in FIG. 19 (1). In (1), each bending point is described by a flag and a point sequence. The flag indicates whether it is an outer contour or an inner contour. In this case, 1 is the outer contour and 2 is the inner contour. Further, the point sequence is described by numbers, X coordinates, and Y coordinates as shown in (2). In the present embodiment, contour data composed of straight line elements is taken up for simplification of description, but of course it may be expressed by a mathematical function such as a quadratic function or a spline function. By expressing the image with the contour data, there is an advantage that processing such as reduction, enlargement, rotation, and affine conversion becomes easier as compared with the case of expressing with the pixel data. Therefore, it can also be used to create fonts and deformed fonts that have many sizes such as character patterns. The format described in the rectangular area will be described with reference to FIG. (1), (2)
Is a partial image as shown in (3) according to (form G-ABC (20 80 10 50) (free Y)) (form G-123 (20 80 50 60) (free X)). You can get an image with. free is a parameter that controls the arrangement of images when they are transferred as described in FIG. In this example, when (free Y) is designated, the ratio in the Y direction is x = y, and (free X) is the one in which the Y axis is replaced with the X axis in the above. FIG. 22 shows how the arranged partial images are regarded as one partial image and are arranged again. (A) and (B) are described in the following formats (1) and (2).

(Defform G-ABC-123 (width 100) (height 60) (form G-ABC (20 80 10 50) (free Y)) (form G-123 (20 80 50 60) (free X)) ……… (1) (defform G (width 100) (height 80) (form G-ABC-123 (0 80 0 60)) (form G-ABC-123 (50 100 50 80))) ……… (2) ( In 2), the format G-ABC-123 defined in (1) is placed twice with a different scale ratio.The format of G-ABC-123 is 100x60 (width is width and width is width). , The height is he
It is expressed in ight. ) And in this format (20 80 10 5
It is shown that G-ABC exists in the region (0) and G-123 exists in the region (20 80 50 60). The format G is defined using the format created here. In (2), format G is G-ABC
Indicates that −123 exists in the (0 80 0 60) and (50 100 50 80) regions.

With the above processing method, images can be arranged in an arbitrary format. FIG. 23 shows how the contents of the document are understood page by page when a plurality of documents having the same format exist. By re-arranging according to the format shown in FIG. 24, it becomes possible to create an abstract as shown in FIG. This method will be described in detail. Figure 23 shows that the document format is F, F1, F11, F12, F2, F3, and pag corresponding to format F11.
The character string for each e is extracted. FIG. 24 shows a format G of an output image. The title column consists of G1, G2, and G3. The contents of this are G1 (page i), G2 (pag
ei) and G3 (page i) (i indicates the number of pages). 25th
The figure shows partial images for each format to be rearranged. These partial images may be arranged according to the format data in the memory 65 by the method as described in FIG.

〔The invention's effect〕

As described above, according to the present invention, it is possible to automatically analyze an input target document and output it in an arbitrary layout. Further, since the format data describing the target document and the format data describing the format of the document to be output can have the same expression format, it is possible to easily cope with various formats. Further, even if the structure of the corresponding document is changed, if the format data is changed, it can be dealt with immediately.

[Brief description of drawings]

1 and 2 are diagrams showing an example of an input document, FIG. 3 is a diagram showing an example of an output document, and FIGS. 4, 5, 6, 7, 8, 9, 9, 10, 11, and 12 are the principles of the present invention. 13 is a block diagram showing the structure of an apparatus for carrying out the document processing system of the present invention, and FIGS.
FIG. 5 is a flow chart for explaining the processing in the control unit 2 in FIG. 13, and FIGS. 16, 17, 18, 19, 20, 21, and 22 are explanatory diagrams for explaining the processing contents of FIG. 23 and 25 are diagrams when applied to the creation of an abstract of a document, and FIG. 24 is a diagram showing a format of an image for output. 1 ... bus, 2 ... control unit, 3 ... document, 4 ... scanner, 5 ... input unit, 6 ... memory, 7 ... character / figure recognition unit, 8,9 ... file, 10 ... … Image output part, 11… Console.

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Hiroi Ueda 1-280 Higashi Koigakubo, Kokubunji, Tokyo Metropolitan Research Laboratory, Hitachi, Ltd. (72) Seiji Kashiwaoka 1-280 Higashi Koigakubo, Kokubunji, Tokyo Hitachi Ltd. Central Research Laboratory

Claims (1)

[Claims] Claim: What is claimed is: 1. A processing device for processing a substantially standardized document, outputting the image in which the document is captured as an image and the layout is rearranged, and the unknown document is input as an image and converted into a digital image. First format data that expresses the arrangement and size of each rectangular area by a notation that describes an image as a set of a plurality of rectangular areas, and thus defines a common format for the document to be processed. In advance, a second storage area in which the second format data in which the layout of the image to be output is defined by the above notation is stored in advance, and a third storage in which the digital image is stored. Storage means having at least an area and a fourth storage area in which the image to be output is stored, the first format data in the first storage area being referred to, The region corresponding to each rectangular area marked in formula data searched on the digital image of the third memory area,
Accordingly, the structure analysis processing for specifying the existence area of the partial image corresponding to each format element of the format of the unknown document, and the existence area specified by the structure analysis processing are used to describe in the second format data. Image processing for cutting out the partial image corresponding to each format element from the third storage area and transferring the partial image to the position defined by the second format data in the fourth storage area. Control unit to execute,
And a document image processing apparatus having an image output unit for displaying a set of partial images accumulated in the fourth storage area.