JP2010178105A - Image analysis device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image analysis device and a program analyzing image data to acquire positional information of each region by an easy technique as compared with a conventional technique. <P>SOLUTION: A region is identified by OCR software, an image displayed by drawing a frame border of its region data is screen-captured to extract the frame border, the coordinate values of the frame border are calculated, and thereby the coordinate values of the region data with the region identified by the OCR software can be obtained, thereby this image analysis device and this program capable of acquiring positional information of each region by an easy technique as compared with a conventional technique can be provided. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image analysis apparatus and a program for acquiring coordinate data of an area included in a document.

As a conventional technology, there is an image processing method for automatically acquiring the coordinate data of the area frame line as a result of area identification using OCR (Optical Character Reader) software for the input image data. is there.
An example of this type of image processing technique is disclosed in Patent Documents 1 and 2.

  By the way, there is layout extraction processing as preprocessing when using this OCR software. The generally known OCR software uses the input document image data to identify areas (layout extraction) with each element attribute value such as characters, non-characters such as figures and photos, tables, ruled lines, etc. The result is overwritten on the image data input as a frame line.

In the internal processing in an electronic computer such as a PC (Personal Computer), if vector data storing coordinate data of a rectangular area is stored, the coordinate data can be directly acquired.
However, when the coordinate data is expressed as drawing data (bitmap or the like), the coordinate data of the rectangular area cannot be acquired. In this case, the user must specify coordinates using a pointer such as a mouse. However, when there are a large number of areas, the process is very complicated.

Here, FIG. 9 is a diagram showing image data of only a specific color, and FIG. 10 is a diagram showing a result of frame line extraction processing by a conventional method.
In FIG. 9, a rectangular B smaller than the rectangle A (for example, a blue line having a width of 0.5 mm) is formed on the lower long side of the rectangle A (for example, a blue line having a width of 1 mm) from which a part (the short side on the right side) is missing. Overlapping conditions are shown.
The missing portion of the rectangle A is a portion that overlaps the frame line that represents the area of the table.
When an area surrounded by a frame line (also referred to as an outline) is extracted from the image data shown in FIG. 9 only by the technique described in Patent Document 1, the result shown in FIG. As a result, an extraction result of an area surrounded by a frame line in which the lower portions of the rectangle A and the rectangle B are integrated as shown in FIG. Even if the technique described in Patent Document 2 is used, the same result as in FIG. 10 is obtained.

  This is because the technique described in Patent Document 1 determines the rectangular area by looking at the connected components of the pixels. The graphic shown in FIG. 9 is regarded as an integration of the rectangle A and the rectangle B because the pixel of the rectangle A and the pixel of the rectangle B are connected (contacted). As a result, the intended rectangular area cannot be obtained as shown in FIG.

  Here, when image data subjected to region identification is printed out, the type of the region can be recognized by the line type, but naturally the position information of each region cannot be stored. Accordingly, even if a document on which image data is printed is read and image data is input, in order to acquire position information of each area, the input image data must be subjected to area identification again. It was.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an image analysis apparatus and a program that can analyze image data and acquire position information of each region by a simpler method than in the prior art.

  In order to solve the above-described problem, the invention described in claim 1 is an image analysis apparatus for acquiring position information of an area included in a color original, wherein the area included in the color original corresponds to an attribute of the area. An input means that is distinguished from other regions by a frame line and that inputs image data generated by reading the color document, and an extraction means that extracts a predetermined frame line from the input image data And acquiring means for acquiring positional information of the region by tracing the extracted frame line, and when the acquisition means has a plurality of directions in which the frame line should be traced, A determining means for determining a direction to be traced according to the traced trace is provided.

  According to a second aspect of the present invention, in the first aspect of the present invention, the area includes a character area, a table area, and an image area, and only the character area or the table area input by the input means is the frame line. And a discriminating means for discriminating according to the color and the thickness of each.

  The invention described in claim 3 is characterized in that, in the invention described in claim 2, when the frame line overlaps with another frame line and is interrupted, the frame line is provided with a complementing means for complementing the frame line. To do.

  The invention according to claim 4 is a program for acquiring position information of an area included in image data, the computer extracting a frame line of a predetermined area from the image data, and the extracted frame Executes a procedure for acquiring position information of the area by tracing a line, and a procedure for determining a direction to trace according to a trace traced so far when there are a plurality of directions in which the frame line should be traced. It is characterized by making it.

  According to a fifth aspect of the invention, in the fourth aspect of the invention, the area includes a character area, a table area, and an image area, and the computer displays only the character area or the table area as a color of the frame line. A procedure for discriminating according to the thickness is executed.

  According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the computer is caused to execute a procedure for complementing when the frame line overlaps with another frame line and is interrupted.

  According to the present invention, an OCR software identifies an area, draws a frame line of data in the area, screen captures the displayed data, and extracts a frame line or a hand-written frame line By calculating the coordinate value, the coordinate value of the area data identified by the OCR software can be obtained, so that the position information of each area can be acquired with a simpler method than the conventional technology It is possible to provide an image analysis apparatus and a program that can perform the above.

It is a hardware block diagram which shows one Embodiment of the image-analysis apparatus based on this invention. It is an example of a functional block diagram of the image analysis apparatus shown in FIG. It is an example of the whole processing flow used for the image analyzer which concerns on this invention. It is an example of the specific color extraction process flow used for the image analysis apparatus which concerns on this invention. It is an example of input original image data. 6 is an example of image data in which a frame line of a region identification result of input image data illustrated in FIG. 5 is drawn. FIG. 6 is an example of image data in which the input document image data shown in FIG. 6 is an example of image data in which the input original image data shown in FIG. It is a figure which shows the image data of only a specific color. It is a figure which shows the frame line extraction process result by a conventional system. It is a figure which shows the extraction result by the image analysis apparatus which concerns on this invention. It is explanatory drawing of the line in which an X direction or a Y direction corresponds. It is a figure which shows the state which complemented the interrupted location and became the closed rectangle. (A)-(d) is a figure which shows the vertex in a rectangular area. It is a figure which shows an example of a pixel trace. It is a figure which shows an example of a pixel trace. It is a figure which shows an example of coordinate data.

<Features>
In the present invention, a region is identified by OCR software, a frame line of the region data is drawn and displayed, screen capture is performed, a frame line is extracted, and a coordinate value of the frame line is calculated. As shown in FIG. 11, the coordinate value of the area data identified by the OCR software can be obtained.
Further, in the present invention, the interval between the short sides of the short side of the rectangle B shown in FIG. Generate a rectangle.

However, since it is not possible to extract a region using a connected component simply by complementing the broken part of the frame line, the region 1 and the region 2 in FIG. 10 are extracted by an extraction method instead of extracting a connected component.
Here, the reason why the region cannot be extracted simply by complementing the discontinuity of the frame line is that the connected component is an overlapping portion of the frame line forming a plurality of regions. That is, when the regions partially overlap, the border lines that distinguish the regions overlap, and in such a case, a circumscribed rectangle that includes all the regions is extracted as one region. Therefore, it is not possible to accurately extract the region simply by complementing the broken part of the frame line.
Specifically, the rectangular area is characterized by having one vertex shown in each of FIGS. 14A to 14D and no intersection. These features are rectangular patterns. The connected components of the pixels are traced so as to match this rectangular pattern (see FIG. 15). In FIG. 15, □ indicates an untraced pixel, and ■ indicates a traced pixel. The order of tracing in FIG. 15 may proceed from the upper left to the lower direction, from the upper left to the right direction, or simultaneously.

FIGS. 15 and 16 are schematic diagrams in which the two frame lines shown in FIG. 13 are represented by pixels.
FIG. 16 shows a state in which tracing is performed simultaneously from the upper left pixel in the right direction and the lower direction.
When the order of tracing the pixels reaches the intersection (the position of ◆), the tracing direction is determined from the position of the pixel immediately before ◆. In the case of FIG. 16, since the position of the traced pixel (■) immediately before ◆ is on the left side of ◆, the next pixel to be traced is the pixel on the right.

<Definition>
Prior to the description of the configuration, variables in the present embodiment are defined as follows.
Si shown in FIG. 5 is an example of input document image data (for example, monochrome data).
Sr shown in FIG. 6 is an example of image data in which a frame line of the region identification result of the input image data shown in FIG. 5 is drawn. L1 is a line (for example, blue line) surrounding the character “report”, L2 is a character area (blue line in this case), L3 is a figure area (red line), L4 is a character area (blue line), and L5 is a table An area (green line) and L6 indicate a character area (blue line), respectively.
Sv shown in FIG. 7 is image data in which the input document image data shown in FIG. 5 is indicated only by a frame line of a specific color (in this embodiment, blue, red, and green).

Here, let Vn (R, G, B) be the color map of the specific color corresponding to the color of the frame line.
The coordinate value for a specific pixel is S * (X, Y) (* is either Si, Sr, or Sv).
Let Sr (R, G, B) be the color map of Sr (X, Y).

<Configuration>
FIG. 1 is a hardware configuration diagram showing an embodiment of an image analysis apparatus to which the present invention is applied.
The image analysis apparatus shown in FIG. 1 mainly includes a CPU (Central Processing Unit) 101, an input image memory unit 104, an area information storage unit 106, an image processing unit 107, a scanner unit 102, an external storage unit 103, a display Unit 105 and communication control unit 108.
The input image memory unit 104, the region information storage unit 106, and the image processing unit 107 are composed of ROM (read-only memory) and RAM (random-access memory).
The communication control unit 108 is connected to the network 109. The external storage unit 103 is connected to a CD (Compact Disc) / DVD (Digital Versatile Disc) / memory card 110 and a hard disk 111.

  The CPU 101 controls the entire image analysis apparatus. The scanner unit 102 reads a document or a document file from the external storage unit 103. The input image memory unit 104 stores the input image read by the scanner unit 102. The display unit 105 displays a preview of the input image. The image processing unit 107 performs character extraction processing / determination processing. The area information storage unit 106 stores area information of extraction / determination results. Information on the processing result is stored in the aforementioned external storage unit. In addition, an input image is obtained from the network 109 or processing result information is distributed via the communication control unit 108.

FIG. 2 is an example of a functional block diagram of the image analysis apparatus shown in FIG.
The control unit 220 illustrated in FIG. 2 includes an extraction unit 200, an acquisition unit 201, a determination unit 202, a determination unit 203, and a complement unit 204.
The extraction unit 200 extracts a frame line of a predetermined area from the image data.
The acquisition unit 201 acquires the position information of the region by tracing the extracted frame line.
When there are a plurality of directions in which the frame line is to be traced, the determining unit 202 determines the direction to be traced according to the trace traced so far.

Here, the area includes a character area, a table area, and an image area.
The discriminating means 203 discriminates only the character area or the table area by the color and thickness of the frame line.
The complement means 204 complements the frame line when the frame line overlaps with another frame line and is interrupted.
The input unit 205 (reads a document and inputs image data) corresponds to the scanner unit 102 in FIG. 1, and the storage unit 206 corresponds to the external storage unit 103 in FIG.

The control means 220 corresponds to the CPU 101 in FIG.
The display unit 207 corresponds to the display unit 105 in FIG. 1, and the communication control unit 208 corresponds to the communication control unit 108 in FIG.
The image processing unit 209 corresponds to the image processing unit 107 in FIG. 1, and the information storage unit 210 corresponds to the area information storage unit 106 in FIG.

The document image illustrated in FIG. 5 includes character elements, non-characters such as graphics, and a table.
When region identification (layout recognition) is performed on this original image using OCR software, the result shown in FIG. 6 is obtained. Frame lines L1, L2, L4, and L6 (for example, blue lines) are character regions, frame line L3 (for example, red line) is a non-character region, and frame line L5 (for example, green line) is a table region. However, the color of the border may change depending on the OCR software. FIG. 7 shows an image of only all the frame lines in FIG.

FIG. 3 is an example of an overall processing flow used in the image analysis apparatus according to the present invention.
When the process starts (step F201), the input image data Si in step F202 inputs color image image data obtained by input means such as file reading, scanner scan data, or via a network.
In step F203, the image data in which the area frame line as a result of area identification by the OCR software is drawn is read.
In step F204, the specific color Vn (R, G, B) designated by the user is extracted from Sr, and image data Sv of only the specific color is generated.
The specific color extraction process is a process of extracting a specific color by the method described in Patent Document 2. FIG. 8 shows an example of image data of only specific colors (for example, blue frame lines L1, L2, L4, and L6).

FIG. 4 is an example of a specific color extraction processing flow used in the image analysis apparatus according to the present invention.
The specific color extraction process will be described along the specific color extraction process flow of FIG.
First, the user determines a color map value Vn (R, G, B) for a specific color, and performs processing in a loop for checking the presence or absence of the specific color for all pixels within the range (step F302).
Determine the value Vo (R, G, B) by subtracting the color map Si (R, G, B) of the specific coordinate value Si (X, Y) of Si from the specific color Vn (R, G, B) (Step) F303).

Here, it supplements about step F303.
Determine the values of V1, Si as follows:
V1 (255, 255, 255)
Si (128, 128, 128)
If V1 − Si = Vo,
Vo is obtained by the following mathematical formulas (1) to (3).
R: 255−128 = 127 (1)
G: 255 − 128 = 127 (2)
B: 255 − 128 = 127 (3)

Therefore, Vo (127, 127, 127).
However, n of the specific color Vn (R, G, B) is an ordinal number.

  It is determined whether Vo (R, G, B) and Sr (R, G, B) match within a certain range (step F304).

Here, it supplements about step F304.
Step F304 is a process for absorbing a color error.
An example of determination is shown below.
Vo (127, 127, 127) and
Sr (128, 128, 128) is determined.
When the range threshold is 5,
Vo .Vo
-5 Sr +5
R: 122 ≤ 128 ≤ 132
G: 122 ≤ 128 ≤ 132
B: 122 ≤ 128 ≤ 132
Final judgment: Matched However, “Vo” indicates a negative threshold, and “.Vo” indicates a positive threshold.

Apart from this
Vo (125, 100, 127) and
Sr (128, 128, 128) is determined.
When the range threshold is 5,
Vo .Vo
-5 Sr +5
R: 120 ≤ 128 ≤ 130
G: .95 ≤ 128 ≤ 105, so they do not match (out of range)
B: 122 ≤ 128 ≤ 132
Final judgment: No match

If they match within the range, the pixel is drawn at the coordinates (X, Y) of the image data Sv of only a specific color (step F305).
With the above processing, image data of only a specific color is output (step F306).

Returning to FIG. 2, in step F205, lines are extracted from the image data Sv of only a specific color by line extraction processing. In the line extraction process, lines are extracted using a reference technique [see Japanese Patent Application Laid-Open No. 2001-291097] or the method shown in FIG.
In step F206, the thickness B of the extracted line is calculated.
A line in which each line matches the X direction or the Y direction is extracted (step F207).

Here, FIG. 12 shows a line that coincides in the Y direction surrounded by an ellipse (line 1, line 2). The ellipse is added for convenience of explanation. Also, for convenience of explanation, it is assumed that the thickness of line 1 and line 2 is B and matches on the reference line in the Y direction but does not match on the line parallel to the reference line in the Y direction (the reference line is These lines are described for convenience of explanation and are not included in the actual image data.)
It is determined whether or not the distance between the line 1 and the line 2 matches the thickness B (step F208).
If the distance between line 1 and line 2 matches the thickness B (step F208 / Yes), the pixels in the distance between line 1 and line 2 are complemented and joined to form a closed rectangle. (Step F209).
A pattern including these features is a rectangular pattern. The connected components of the pixels are traced so as to match this rectangular pattern. Here, tracing means sequentially tracking pixels constituting a line (frame line) (FIG. 15). The order of tracing may be downward, rightward, or simultaneous progress.

In FIG. 16, when the trace target pixel (■) reaches the intersection (♦), the tracing direction is determined from the direction of the immediately preceding pixel (■). In the case of FIG. 16, since the trace of the immediately preceding pixel (■) is in the right direction, the trace is performed in the right direction after the pixel (□) to be traced.
After the series of trace processes (step F210), it is determined whether or not the area is a rectangular area (step F211).
In the rectangular area determination processing, if the shape of the vertex matches all of FIGS. 14A to 14D and the above-described trace closed rectangle is satisfied, the rectangle is satisfied. Judge as a region. That is, it is determined that a portion lacking a (frame) line should not be complemented with a portion that is not divided by another (frame) line. Therefore, as shown in FIG. 12, line 1 and line 2 match on the reference line in the Y direction, but line 1 and line 2 do not match on the line parallel to the reference line in the Y direction (right side line). , It doesn't do the completion which becomes a closed rectangle.
Coordinate data is calculated for the frame line determined as a rectangular area (step F212). For the calculation of the coordinate data, the coordinate data is extracted by the rectangle extraction process disclosed in the reference technique [Patent Document 1].

As for the coordinate data, the upper left corner of the document image data is the origin (0, 0), and the coordinate data of area 1 is composed of (Xs, Ys, Xe, Ye) (FIG. 17).
By performing the above processing, the coordinate value of the frame line can be extracted.

<Program and storage medium>
The image analysis apparatus according to the present invention described above is realized by a program that causes a computer to execute image analysis processing. Examples of the computer include general-purpose computers such as personal computers and workstations, but the present invention is not limited to this. Therefore, as an example, a case where the present invention is realized by a program will be described below.

A program for acquiring position information of an area included in image data,
On the computer,
(1) A procedure for extracting a border of a predetermined area from image data,
(2) Procedure for obtaining position information of an area by tracing the extracted frame line,
(3) When there are multiple directions in which the frame line should be traced, a procedure for determining the direction to be traced according to the trajectory traced so far,
The program which performs is mentioned.

In addition to the above configuration,
The area includes a character area, a table area, and an image area.
(4) A program that executes a procedure for discriminating only a character area or a table area based on the color, thickness, and position of a frame line may be used.

In addition to the above configuration,
(5) A program that executes a procedure for complementing when a frame line overlaps with another frame line and is interrupted.

  Thus, the image analysis apparatus according to the present invention can be realized anywhere as long as there is a computer environment capable of executing the program.

Such a program may be stored in a computer-readable storage medium.
Here, examples of the storage medium include a computer-readable storage medium such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (CD Recordable), and a DVD (Digital Versatile Disk). , Flash memory, RAM (Random Access Memory), ROM (Read Only Memory), semiconductor memory such as FeRAM (ferroelectric memory), and HDD (Hard Disc Drive).

  The above-described embodiment shows an example of a preferred embodiment of the present invention, and the present invention is not limited thereto, and various modifications may be made without departing from the scope of the invention.

101 CPU
102 Scanner Unit 103 External Storage Unit 104 Input Image Memory Unit 105 Display Unit 106 Area Information Storage Unit 107 Image Processing Unit 108 Communication Control Unit 109 Network 110 CD / DVD
111 hard disk

Japanese Patent No. 2948840 JP 2003-295847 A

Claims (6)

An image analysis apparatus for acquiring position information of an area included in a color document,
The area included in the color document is expressed separately from other areas by a frame line according to the attribute of the area,
Input means for inputting image data generated by reading the color original;
Extracting means for extracting a predetermined frame line from the input image data;
Obtaining the position information of the region by tracing the extracted frame line, and
The image analysis apparatus according to claim 1, wherein the acquisition unit includes a determination unit that determines a direction to be traced according to a trajectory traced so far when there are a plurality of directions in which the frame line is to be traced. The region includes a character region, a table region, and an image region,
And a discriminating unit that discriminates only the character region or the table region input by the input unit based on the color and thickness of the frame line extracted by the extracting unit and the position information acquired by the acquiring unit. The image analysis apparatus according to claim 1.   The image analysis apparatus according to claim 2, further comprising a complementing unit that complements the frame line when the frame line overlaps with another frame line and is interrupted. A program for acquiring position information of an area included in image data,
On the computer,
A procedure for extracting a border of a predetermined area from the image data;
A procedure for obtaining position information of the region by tracing the extracted frame line,
A procedure for determining a direction to be traced according to a trajectory traced so far when there are a plurality of directions in which the frame line should be traced;
A program characterized by having executed.   The area includes a character area, a table area, and an image area, and causes the computer to execute a procedure for determining only the character area or the table area based on a color, a thickness, and a position of the frame line. The program according to claim 4.   6. The program according to claim 5, wherein the program causes the computer to execute a complementing procedure when the frame line overlaps with another frame line and is interrupted.

Images