JPS6245581B2 - - Google Patents

Description

[Detailed Description of the Invention] The present invention provides an arrangement device for drawings, photographs, characters, etc.
In particular, it relates to an input device for layout formats.

Conventionally, in printing systems for newspapers, magazines, documents, etc., the formatting (allocation) of given drawings, photographs, characters, etc. on a paper of a fixed size, that is, the positioning on the paper, is as follows:
This was done manually by skilled people, or by displaying the entire page on a special display and interactively specifying the positions of photographs, drawings, and text. Formatting means where to position photographs, drawings, characters, etc. within a predetermined space of paper. For example, for a rectangular photograph, the coordinates of the upper left corner of the paper to be formatted and Formatting can be performed if two pieces of information, the vertical and horizontal sizes of the photograph, are obtained (these two pairs of information are called format information, and the set is called a format). With the large increase in the number of printed sheets, computers have been introduced to save labor in formatting work, but as mentioned above, the formatting process is largely performed using a special display. It is a part. The disadvantage of using such a display is that the equipment is expensive, and in most cases the size and aspect ratio of the screen displayed on the display are different from the actual paper to be printed, making it easy to make layout mistakes or errors. It was something that would happen. Other format input methods include a method in which a paper (form) on which a formatting area (for example, a rectangular frame) is drawn is read using an image scanner such as a facsimile machine, and a computer calculates and detects the format.
However, when formatting multiple drawings onto a single sheet of paper, for example, multiple pieces of format information are generated, but computers cannot identify each one individually, so they are simply assigned sequential numbers and classified in the order in which they are input. It's just being done. In this case, the correspondence with the plurality of drawings to be formatted must be done by a person while looking at the correspondence table using numbers that have no relation to the drawings, making it easy for incorrect formatting to occur.

SUMMARY OF THE INVENTION An object of the present invention is to eliminate the above-mentioned drawbacks, provide a format input device that has a form close to the size of an actual form, and is capable of automatic identification and classification.

According to the present invention, there is provided an image input device that binarizes and outputs grayscale image data obtained by scanning a secondary image, and one or more rectangular regions are extracted from the two-dimensional image data obtained by the image input device. _C1 , _C2 ,
..., a character recognition device that cuts out Cn, performs character recognition on one or more characters included in each region, and outputs the corresponding character code and character position and size on the two-dimensional image data; Among the characters recognized by the character recognition device, character rectangular areas S ₁ , S ₂ , ..., containing a specific type of character code
A character rectangular area calculation device that extracts Sk and calculates the position and size of each area from the reference point on the two-dimensional image data, and one or more character rectangular area calculation devices from the two-dimensional image data obtained from the image input device. a format rectangular region calculation device that cuts out format rectangular regions R ₁ , R ₂ , ..., Rk and calculates the position and size of each region from a reference point on the two-dimensional image data; Rectangular area R ₁ obtained from the calculation device,
From the position and size of R ₂ ,..., Rk, a rectangular area
_The positional relationship between R _{1 ,} R _{2 ,} .
For R ₂ , ..., Rk, the corresponding character rectangular area S ₁ ,
It has an area identification device that allocates the character code string included in S ₂ , ..., Sk as an identification code, calculates the position and size of one or more format rectangular areas described on a two-dimensional image, and It is possible to automatically assign identification character code strings to areas and classify them.

Next, the present invention will be explained in detail with reference to the drawings. Hereinafter, the numbers attached to signal lines in the drawings will be explained as referring to the signals on the signal lines as well.

FIG. 1 is a block diagram showing an embodiment of the format input device of the present invention.
It is composed of a character recognition device 2, a character rectangular area calculation device 3, a format rectangular area calculation device 4, and an area identification device 5. Figures 2a, b, and c are schematic diagrams for explaining the principle of the format input device, and Figure 2a shows the format input form,
A rectangular area C ₁ to be processed by the character recognition device 2,
C ₂ ,...Cn, and FIG. 2 b shows a character rectangular area in which a character to be processed by the character rectangular area calculation device 3 is written in the same format input form.
S ₁ , S ₂ , . . _. , Sk are shown, _and FIG. .

The image input device 1 can use an image scanner such as FSS, ITV, or CCD scanner, and scans a two-dimensional image to obtain two-dimensional image data. For example, a circuit for binarizing multivalued data is provided after the image scanner. The character recognition device 2 can use the recognition part of an OCR (optical character recognition device), and from the two-dimensional image data from the image input device 1, one or more rectangular areas C ₁ , C ₂ , ..., Cn ( This is a device that cuts out the image shown in FIG. 2a) and performs character recognition on one or more characters included in each area. The rectangular areas C ₁ , C ₂ , ..., Cn are cut out based on the position and size from the reference point on the two-dimensional image data (for example, the origin is the upper left corner coordinates), and this information is output. At the same time, in each area, for recognized characters, a character code corresponding to the recognized character is output, and for unrecognized characters, a character code indicating that the character was not recognized is outputted. The character rectangular area calculation device 3 extracts character code examples of predetermined character types (for example, alphanumeric characters) from the output characters of the character recognition device 2 as format identification characters, and extracts characters including the extracted character code string. rectangular area
S ₁ , S ₂ , ..., Sk (Fig. 2 b) in the rectangular area C ₁ ,
Select from C ₂ , ..., Cn and create a rectangular area for each character.
For S ₁ , S ₂ , ..., Sk, reference points (C ₁ , C ₂ ,
..., which is the same as the reference point for Cn), and outputs it along with the corresponding character code example. The format rectangular area calculation device 4 calculates one image from the secondary image data obtained from the image input device 1.
Or a plurality of format rectangular areas R ₁ , R ₂ ,
..., information for specifying the position and size of Rk (Fig. 2c) (for example, one corner coordinate and diagonal corner coordinate of a rectangular area, one corner coordinate, vertically long and horizontally long, or four corners) This device extracts a format instruction line segment indicating the coordinates) and calculates the position and size of each region R ₁ , R ₂ , . . . , Rk from a reference point on the two-dimensional image data. The area identification means 5 is
Based on the positions and sizes of the format rectangular regions R ₁ , R ₂ , ..., Rk obtained by the format rectangular region calculating device 4, the format regions R ₁ , R ₂ , Rk are calculated.
..., Rk and the character rectangular areas S ₁ , S ₂ , . ₁ ,
This is a device that examines character rectangular areas Si existing at a constant distance or within a certain range from R ₂ , . . . , Rk, and outputs corresponding areas as a pair. Specifically, character code strings (referred to as format identification characters) included in the corresponding character rectangular areas S ₁ , S ₂ , ..., Sk are assigned as identification codes to the position and size of the format rectangular area Ri, and the format A format identification character is added to the position and size of the rectangular area Ri and output as a pair.

Before entering into a detailed block description of the present invention, the processing procedure until a format is input according to the present invention will be explained based on an embodiment.

FIG. 3 is a diagram showing an example of a two-dimensional image showing a part of a format input form, in which one or more formats and associated format identification character strings are described. Figure 2a
What corresponds to Ci in is the rectangular area represented by the reference grid 90 in FIG.
What corresponds to Si in Figure b is a rectangular area (character rectangular area) in which the format indication line segment ELi and format identification character EDi are written, and what corresponds to Ri in Figure 2c is the format indication line Minute ELi. The format is described by a format instruction line segment ELi with point P _LUi as the starting point and point P _RDi as the end point, and written in a format rectangular area (hatched area) with point P _LUi as the upper left corner point and point P _RDi as the lower right corner point. It is stipulated accordingly. The format indicating line segment ELi connecting the point P _LUi and the point P _RDi is not a right-angled line segment as shown in FIG. 3, but may be any line segment as long as the point P _LUi and the point P _RDi are connected.
Furthermore, although an example is given in which the format identification character string is located inside the format rectangular area (shaded area), it may also be defined as existing at a certain distance or a certain position from the format rectangular area. In FIG. 3, the dashed line lattice serves as a reference when format instruction line segments ELi and format identification characters are written. The reference grid 90 is described in pop-out colors, as described above in FIG. 2a.
Corresponds to Ci.

FIG. 4 shows binarized two-dimensional image data after the two-dimensional image on the format input form shown in FIG. 3 is input into the image input device 1. The data includes a format instruction line segment pattern FLi and a format identification character string pattern FDi. The reference grid 91 described in FIG.
Although this data does not actually exist on the next image data, it is provided to facilitate the subsequent explanation.

In Fig. 4, the coordinates of the upper left corner point Q _LUi of the formatted rectangular area Ri (shaded area) are expressed as (x _LUi , y _LUi ), the coordinates of the lower right corner point Q _RDi are (x _RDi , y _RDi ), the horizontal length of the format rectangular area Ri is m _i , the vertical length is n _i , and the grid width of the reference grid 91 is Δm in width and Δn in height. shall be. All of these are variables or constants that take values not in units of reference grids but in units of dots, and m _i ,
The following relationship holds between n _i , x _LUi , y _LUi , x _RDi , and y _RDi .

The i used here is a suffix that takes a value from 1 to k, and is used to identify a plurality of format rectangular areas.

FIG. 5 shows character codes obtained by inputting the two-dimensional image data in FIG. 4 into the character recognition device 2 and performing character recognition on what is written in the reference grid. A reference grid is used for display in order to correspond to the format input form shown in FIG. In FIG. 5 as well, a reference grid 92 is described in order to correspond with FIG. 3, but this data does not actually exist. Row and column designations are used here to refer to individual reference grids. Character recognition is performed through two processes: cutting out a recognition target area and character recognition processing. In this embodiment, the recognition target area is defined as a unit of the reference grid 92, and character recognition is performed for characters or symbols written within the reference grid 92. Specifically, in order to find the cutout position and area size of the recognition target area, the relative position in dot units from the coordinate origin (x _p , y _p ) on the two-dimensional image data shown in FIG. The size calculation is performed for each reference grid 9
The position and size of 2 are determined. Reference grid 9
As mentioned above, the sizes of 2 are Δm and Δn, respectively, so the position of the character code written in the I row, J column of the reference grid (the upper left corner coordinates of the I row, J column) is:
It is given by (Δm×(I-1), Δn×(J-1)). The area size, that is, the reference grid size is Δm, Δn
It is. In FIG. 5, the recognized character codes are assigned to the reference grid 92 in order to make it easier to see the correspondence between the reference grid 92 and the recognized character codes. According to the character recognition device 2, in addition to recognizing format identification characters, recognition processing is also performed on patterns or blank spaces in format instruction line segments, but since these are not characters, they are different from format identification characters. A code (for example, a prearranged code indicating that it is not recognized) is assigned and output. In this way, the character recognition device 2 outputs the recognized character code string and the coordinates and size of each character.
Although it is possible to output all of the recognized character code strings, it is also possible to output a predetermined type of character code string. Ru.

The character code string recognized as above is
The coordinates Li (x _Li , y _Li ) on the two-dimensional image data where the character code string corresponding to the format identification character and the first character of each character code string are located are input to the character rectangular area calculation device 3 and output. do.
The format identification character code is extracted based on the difference in code or code type. The coordinates on the two-dimensional image data where the first character of the extracted format identification character code string is located are the characters described in the reference grid row I and column J, similar to the extraction of the recognition target area by the character recognition device 2. If x _Li =
It can be determined by Δm×(I-1) and y _Li =Δn×(J-1). Alternatively, it can be determined using the character coordinates and size obtained from the character recognition device 2. In the case of Fig. 5, alphabets are specified as the format identification characters, so the characters corresponding to the characters A, B, and C written in the reference grids (I, J), (I, J+1), and (I, J+2) Codes 41 ₁₆ , 42 ₁₆ , 43 ₁₆ are extracted as format identification characters, and the first character A of the character code string is
, that is, the position of the character rectangular area occupied by the format identification character string x _Li =Δm×(I-1), y _Li =Δ
n×(J-1) and area size Δm×3, Δn
It is output with. Here, in the area size, Δm×3 indicating the width means that the characters A, B, and C are 3 in the row direction.
Since the pieces are continuous, the width is calculated by multiplying Δm by 3. Generally, recognized characters are
The number of consecutive pieces in the column direction is used.

When the two-dimensional image data (FIG. 4) obtained from the image input device 1 is input to the format rectangular area calculating device 4, the position of the character rectangular area outputted from the character rectangular area calculating device 3 (Fig. Li (x _Li , y _Li )) in Fig. 5, and the area corresponding to the area size (Δm×3, Δn in Fig. 4) are masked, and the area corresponding to the format identification string 2 is masked. Patterns on the dimensional image data are removed. This is a process to detect only the format instruction line segment FLi for format specification, and from the end points Q _LUi and Q _RDi of the format instruction line segment FLi obtained in this way, the upper left corner of the format rectangular area Ri is detected. Coordinates and area size are output.

The coordinates of the end points Q _LUi and Q _RDi of the format indicating line segment FLi shown in FIG. 4 are determined by the following procedure. This will be explained based on FIG. First, sequential scanning is performed in the y direction, and the first detected dot is Q _LUi
shall be. Next, starting from Q _LUi , trace the format indicating line segment FL _i where the dot exists, and find the most y
Define Q _RDi for a dot with large coordinates.

FIGS. 6d, b, and c are schematic diagrams for explaining an example of a method for tracing a format instruction line segment. In Fig. 6a, the dot D _ij that we are currently focusing on and the dots D _i-1 , _j , D _i+1 , _j , D _i , _j-1 , D
_i and _j+1 are shown. Figure 6b shows which direction, up, down, left, or right, to trace from the currently focused dot D _ij .
Follow the state transition diagram shown in . As shown in the figure, there are four states: an upward state ST ₁ , a rightward state ST ₂ , a downward state ST ₃ , and a leftward state ST ₄ , which indicate the direction currently being tracked. Transition from one state to another is possible in the direction indicated by the arrow, but each state has a priority, and if there are two states that can transition from one state, the one with the highest priority and only if the transition is impossible, transition to a lower priority state. The highest priority is
In the upward state ST _1, the value decreases in the following order: rightward state ST ₂ , downward state ST ₃ , and leftward state ST ₄ . In the case shown in FIG. 6c, the following procedure is involved in tracking from the starting point A ₁ to the ending point A ₁₉ . Assuming that the initial state is in the upward state ST ₁ , the starting point A ₁ cannot be traced upward, so it is in the rightward state
Transition to ST ₂ , and tracking moves to point A ₂ . Since it cannot be traced upward to points A ₂ and A ₃ , it is traced to point A ₄ without incurring the benefit of priority. Since upward tracking is possible at point _A4 , the state transitions to upward state _ST1 and tracking moves to _A5 . When moving to point A ₆ , it transitions to rightward state ST ₂ . At point _A6 , since tracking is not possible in both the upward and rightward directions, the state transitions to downward state _ST3 and the tracking moves to _A7 . At point A ₇ , tracking is possible up and to the right, but since a transition from downward state ST ₃ is only possible to right state ST ₂ , the tracking moves to A ₈ . At A ₁₀ , the downward tracking direction changes with the same operation as A ₆ and comes to A ₁₄ while remaining in the downward state _ST3 . From _A14 , tracking is possible only in the right or left direction based on the state transition priority, so tracking moves to _A15 , and the state shifts to leftward state _ST4 . At _A15 , the tracking moves downward from the state transition priority, and at _A17 , the same operation as _A7 is performed, and at _A18 , the same operation as _A6 is performed to reach the end point _A19 .

In this way, the upper left corner and lower right corner coordinates (x _LUi , y _LUi ), (x _RDi ,
y _RDi ) is calculated. Format rectangular area Ri
The area size of is expressed as in the above equation (1) using two coordinate values (upper left corner, lower right corner).

The area identification device 5 generates format information (the upper left corner of the format rectangular area Ri) that is the output of the format rectangular area calculation device 4 based on the format identification character code string and its coordinates that are the output of the character rectangular area calculation device 3. Coordinates and their area sizes) that have a corresponding relationship are output as pairs. Now, the output of the format rectangular area calculation device 4 is assumed to be the following k pairs of data.

Here, m _i and n _i are the horizontal and vertical sizes of the format rectangular area given by equation (1). Assuming that each format rectangular area R _i shown by equation (2) does not overlap with each other, the following coordinate relationship holds true.

The coordinates of the k format identification character strings corresponding to the character rectangular area S _i are expressed as follows.

The correspondence between R _i in equation (2) and C _i in equation (5) is performed as follows.

As described above, the format rectangular area R _i
The position (x _LUi , y _LUi ), the area size m _i , n _i and the corresponding format identification character string are outputted from the area identification device 5 as a pair.

Next, one of the embodiments will be described in detail. Since the image input device 1 and the character recognition device 2 are well known, the remaining character rectangular area calculation device 3, format rectangular area calculation device 4, and area identification device 5 will be explained.

FIG. 7 is a block diagram showing an embodiment of the character rectangular area calculation device 3. In the figure, 30 is a character code comparison circuit, 31 is a format identification character code table in which character codes determined for format identification are registered, and 32 is a character rectangular area calculation circuit. The character code string 201 from the character recognition device 2 is sent to the character code comparison circuit 3
0 is compared with the character code registered in advance in the format identification character code table 31, and if they match, a match signal 301 is output, and if they do not match, a mismatch signal 302 is output. The character rectangular area calculation circuit 32 receives the standard grid sizes Δm202 and Δn203 of the format identification form and the matrix sizes M204 and N205 of the standard grid of the format identification form from the character recognition device 2, and receives the reference grid sizes M204 and N205 of the format identification form from the first row From the character code of the column, the column number J changes at first, and each time J becomes equal to M, the row number I increases by 1, and then the column number J
The row and column numbers are determined in synchronization with the match signal 301 for the character code strings that are sent in series according to the row and row numbers that occur in the order in which the count starts again from 1. Furthermore, for a continuous format identification character code string, the upper left corner coordinates of the first character x _Li 321, y _Li 322 and the area size of the continuous character string (Δm x number of characters in the row direction)
323, Δn x number of characters in column direction 324). For non-consecutive characters, the upper left corner coordinates x _Li 321, y _Li
322 and character area size (Δm323, Δn324)
Output.

FIG. 8 is a block diagram showing one embodiment of the format rectangular area calculation device 4.

In the figure, 40 is a mask circuit for removing a character rectangular area from the two-dimensional image data 101, and the character rectangular area information 321, 322, 323, 324 transferred from the character rectangular area calculation device 3 is transferred to a register group 4.
3 and specify the area to be masked.

As described above, the two-dimensional image data 401 masked in this manner is used to determine the coordinates of the upper left corner point of the formatted rectangular area by the start point detection circuit 41. This circuit 41 is a circuit that scans two-dimensional image data in the y direction to find a starting point.

As shown in FIG. 9, two adjacent points A (x _c -1, y _c ) and B (x _c ,
When the value of y _c ) appears for the first time after starting scanning (in the direction of the arrow), let x _c and y _c at that time be the starting point coordinates Q _LUi .

Next, the end point detection circuit 42 determines the coordinates of the end point of the format instruction line segment, that is, the lower right corner point of the format rectangular area, and uses the format rectangle information as the format rectangular area positions 421, 422 and size.
You can get 423, 424.

Since the end point detection circuit 42 includes a tracking circuit as described above, it will be explained using the block diagrams of FIGS. 10a, b, and c. In Figure 10a, 61,
Reference numerals 62, 63, and 64 indicate flip-flops for latching dots of two-dimensional image data, corresponding to FIG. 6a. Although the flip-flop corresponding to D _ij shown in FIG. 6a is not drawn, since D _ij always takes the value 1, the flip-flop is unreliable.

CK is each flip-flop 61, 62, 63,
64 is used to latch data u, r, d, l. Each flip-flop 61, 62, 6
The outputs of 3 and 64 are U, R, D, and L, respectively. FIG. 10b shows the shift register 70,
A downward shift (71→74 direction) is performed by the signal SL, and a right shift (74→71 direction) is performed by the signal SR. The shift register 70 has four bits 71,
72, 73, and 74, and maintain the above-mentioned upward, rightward, downward, and leftward states, respectively. Figure 10c
is a circuit that operates state transition. 81 is
For the input data U, R, D, L, the next tracking direction allowed in the current tracking state ST _i = (i = 1, 2, 3, 4) (for example, for ST _2, upward, right, downward direction) This is a tracking direction limiting circuit 81 that gates and outputs the data in the . For example, in ST ₂ state 8
1 outputs U', R', and D', and L' is not output. 82 is a priority direction selection circuit 82 which selects the data having the value 1 and having the highest priority among the data U', R', D', and L' gated in 81 according to the priority order described above. be. These outputs U″, R″, D″, L″ are the next stage state transition determination circuit 8
3, from the current tracking state and the next tracking direction (determined by one of U″, R″, D″, L″), the shift register shown in Figure 10b is selected. Outputs SR and SL that determine the shift direction.

The area identification device 5 is simply a circuit for finding pairs that satisfy the magnitude relationship of equation (6). A block diagram is shown in FIG. Positions x _Lj 321 and x _j 322 of the character rectangular area S _i obtained from the character rectangular area calculating device 3 and positions x LUi 421, y _{LUi 422} of the format rectangular area R _i obtained from the format rectangular area calculating device 4.
Comparison circuit 1 from area size m _i 423, n _i 424
When formula (6) is satisfied using 100, 1200, 1300, 1400, the coincidence signal 11001 is x _LU
It is output together with _i , y _LUi , m _i and n _i .

As described above, the format input device according to the present invention is capable of inputting a format in a form close to the actual document size, and is also capable of automatically identifying and classifying format information.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the configuration of the present invention, FIG. 2a is a schematic diagram showing a recognized character area on a two-dimensional image, and FIG. FIG. 2c is a schematic diagram showing a character rectangular area in which an identification character is written. FIG. A schematic diagram showing an example of an instruction line segment and a format identification character; FIG. 4 is a schematic diagram showing two-dimensional image data as a result of the two-dimensional image in FIG. 3 being read by the image input device 1;
FIG. 5 is a schematic diagram showing the result of recognition of the two-dimensional image data in FIG. 4 by the character recognition device 2;
FIG. 6a is a schematic diagram showing a reference area when tracing a format instruction line segment, FIG. 6b is a schematic diagram showing the transition of the tracking direction state, and FIG. 6c is a format diagram for explaining the tracking procedure. A schematic diagram showing a part of an example of a designated line segment, FIG. 7 is a character rectangular area calculation device 3
8 is a block diagram showing an embodiment of the format rectangular area calculating device 4. FIG. 9 is a block diagram showing an embodiment of the starting point detection circuit for the format instruction line segment. FIG. 10 is a block diagram showing one embodiment of the format instruction line segment tracing in the format rectangular area calculation device 4, and FIG. 11 is a block diagram showing one embodiment of the region identification device. In the figure, 1... image input device, 2...
Character recognition device, 3...Character rectangular area calculation device, 4
... Format rectangular area calculation device, 5 ... Rectangle identification device, 90, 91, 92 ... Reference grid, 30
...Character code comparison circuit, 31...Format identification character code table, 32...Character rectangle area calculation circuit, 40...Mask circuit, 41...Start point detection circuit, 42...End point detection circuit, 43...Character rectangle Area information storage circuit, 61, 62, 63, 64...
...Dot register, 70...Shift register, 8
1... Tracking direction limiting circuit, 82... Priority direction selection circuit, 83... State transition determining circuit.

Claims (1)

[Claims] 1. An image input means for binarizing and outputting gray scale image data obtained by scanning a two-dimensional image; and
Alternatively, cut out multiple rectangular areas C ₁ , C ₂ , ..., Cn, perform character recognition on one or more characters included in each area, and identify the corresponding character code and characters on the two-dimensional image data. character recognition means for outputting the position and size; and character rectangular areas S ₁ , S ₂ , ..., Sk containing a specific type of character code from among the characters recognized by the character recognition means,
character rectangle area calculation means for calculating the position and size of each area from a reference point on the two-dimensional image data; and one or more format rectangles from the two-dimensional image data obtained from the image input means. A formatted rectangular area calculation means for cutting out the regions R ₁ , R ₂ , ..., Rk and calculating the position and size of each of the regions from a reference point on the two-dimensional image data; Based on the positions and sizes of the rectangular regions R ₁ , R ₂ , ..., Rk, the rectangular regions R ₁ , R ₂ , ..., Rk and the character rectangular regions S ₁ , S ₂ , ..., Sk is calculated, and character code strings included in the corresponding character rectangular regions S ₁ , S ₂ , ..., Sk are used as identification codes for the format rectangular regions R ₁ , R ₂ , ..., Rk. A format input device comprising allocation area identification means, which calculates the position and size of one or more format rectangular areas described on a two-dimensional image, and allocates an identification character code string to each area. Device.