JP2013080349A - Image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve problems of increasing the size of a file when characters include many variations of colors, for the necessity of describing new modification information every time the modification information of the characters in a text is changed when adding the modification information of the character such as a color to the character.SOLUTION: An image processor comprises: extracting means for extracting a plurality of pixel blocks whose pixel values are approximate in pixels constituting an input image including a character string; specifying means for specifying an area constituted by the plurality of pixel blocks as at least one of a character area and an area other than the character area; acquiring means for analyzing a character from the pixel blocks specified as the character area, and acquiring character information including at least a character code and position information of the character; designating means for designating a character string including a space character from a disposition of the character indicated by the character information; and adding means for acquiring color information from the pixel blocks of the character area at a position indicated by the character information, and adding the color information to the character information. The image processor adds the color information of characters before/after the space character in the character string with respect to the character information of the space character included in the character string.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for generating editable electronic document data from a paper document or document image data.

  In recent years, when creating a document, advanced functions have been used, such as adding decorations to fonts, creating drawings freely, and capturing photos, etc., as well as simply typing characters. .

  However, the higher the content of the creation, the more labor is required to create a completely new document. Therefore, it is desired that a part of a document created in the past can be reused as it is or after being processed and edited.

  On the other hand, the spread of networks such as the Internet has increased the opportunities for electronic distribution of documents, but electronic documents are often distributed in a printed state on paper. In this way, even when only a paper document exists at hand, a technique for obtaining the contents as reusable data from paper is disclosed. For example, Patent Document 1 describes that when a paper document is electronically read by a device, a document that matches the content is retrieved from a database and can be used in place of the read paper data. ing. If the same document cannot be identified from the database, the image of the read document is converted into electronic data that can be easily reused. In this case as well, the contents of the document can be reused.

  Conventionally, there has been an OCR technique as a technique for converting character information in a document image into electronic data that can be easily reused. Further, as a technique for converting graphic information composed of lines and planes into electronic data that can be easily reused, there has been a vectorization technique. For example, Patent Document 1 discloses a technique for converting characters in a document image into character codes using the above technique, or converting the outline of a figure into reusable data by using vector data.

  Patent Document 1 further discloses a technique for identifying areas such as characters, line drawings, natural images, and tables in a document image and constructing data that represents the relationship between the areas in a tree structure. Then, by arranging the character code, vector data, image data, etc. according to the same structure, conversion into an electronic document page editable by an application is performed. This electronic data has the same layout as the original document, and it is easy to change the position and size of characters and figures, as well as geometric deformation and coloring, just like an electronic document page newly created by a document creation application. It can be carried out.

  In addition, with the spread of color printers and the like, opportunities to receive colorfully printed paper documents are increasing. When reusing the contents of such a color document, it is required to reproduce the color information and convert it into reusable data. In response to such a request, in Patent Document 2, the color image is reduced to an image that can have two or more pixel values so that color information such as characters is not lost, and then the pixels that have the same color. A block is extracted to identify the region. By obtaining a pixel block having color information using this technique and performing the above-described processing such as vectorization, it is possible to obtain reusable data after reproducing the color information.

  In Patent Document 3, color information obtained based on the pixels of the input image is added to each character code of the character recognition result. Thereby, reusable character data reproducing color information can be obtained.

Japanese Patent No. 4251629 US Patent Application Publication No. 2008/0123945 JP 2009-205232 A Specification

  When converting an input image into an electronic document, if modification information such as color is added for each character, there is a problem that the file size increases accordingly.

  If there is a space character (blank character) in the character recognition result, there is no pixel information corresponding to the space character, so the space character is determined as having no color information. If a character string with color information and a space character without color information are mixed in the character string of the character recognition result, if conversion to an electronic document is performed as it is, the color for each character It becomes necessary to describe the presence or absence of information. As a result, the file size of the electronic document increases, and there is a problem that convenience is lost during storage and reuse.

  In order to solve the above problems, the present invention has the following configuration. That is, an image processing apparatus that generates electronic data that can be edited from an input image, and the pixel values approximate in input means that inputs an image including a character string as the input image and pixels that constitute the input image An extraction unit that extracts a plurality of pixel blocks, an identification unit that identifies a region formed by the plurality of pixel blocks as at least one of a character region and other regions, and is identified as the character region Analysis means for analyzing a character from the pixel block and acquiring character information including at least a character code and position information of the character, and specifying a character string including a blank character from an arrangement of characters indicated by the character information Color information adding means for acquiring color information from a pixel block of a character area at a position indicated by the character information and adding the color information to the character information; Generating means for generating a description defining the electronic data from a character string and character information of a character included in the character string, and the color information adding means includes character information of blank characters included in the character string. On the other hand, color information of characters before and after the blank character in the character string is added.

  By adding front and rear modification information to invisible characters, the modification information in the character string is collected and the file size is reduced without changing the appearance information of the generated electronic document.

The figure which shows an example of a system configuration. The block diagram which shows the change of the data by the process of each site | part. The flowchart which shows the process in a pixel block analysis part. The flowchart which shows a labeling process. The figure which shows the example of a labeling process. The figure which shows the example of the process result by a pixel block analysis part. The flowchart which shows the process in a layout analysis part. The flowchart which shows the process in a graphics data generation part. The figure which shows the example of the process result in a layout analysis process part. The flowchart which shows the process in a character recognition part. The figure which shows the example of the processing result of a character area. 5 is a flowchart showing color information addition processing. The figure which shows the example of the process result of a color information addition process. The figure for demonstrating pixel block color information. The flowchart which shows the process in an electronic document production | generation part. The figure which shows the example of the table which defines the output object for every area | region classification. The figure which shows the example of an output of electronic document data. The figure which shows the example of the processing result of the character area which concerns on 2nd embodiment. The figure which shows the output example of the electronic document which concerns on 2nd embodiment.

<First embodiment>
[System configuration]
The best mode for carrying out the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of a system configuration using an image processing apparatus according to the present invention. The image processing apparatus 100 includes a scanner 101, a CPU 102, a memory 103, a hard disk 104, and a network I / F 105. The scanner 101 converts the page information of the read document into image data. The CPU 102 executes a program for performing electronic document generation processing on the image data. The memory 103 is used for temporary storage of work memory and data when executing the program. The hard disk 104 stores the program and data. The network I / F 105 performs data input / output with an external device. The image processing apparatus 100 is connected to a wired or wireless network 110 such as a LAN or the Internet via a network I / F 105. The network 110 is further connected to a general-purpose personal computer (PC) 120. The PC 120 can receive data transmitted from the image processing apparatus 100 and use it for display / editing on the computer. Is possible.

[Configuration of electronic document generation processing]
FIG. 2 is a block diagram showing the configuration of the electronic document generation process executed by the CPU 102 of the image processing apparatus according to the present invention. Further, various data generated in the electronic document generation process are shown. The input image 200 and the output electronic document 210 in FIG. 2 are input data and output data for the electronic document generation process, respectively. A flow of processing until the input image 200 is output as the output electronic document 210 and an outline of each processing unit that performs processing will be described. Detailed processing contents of each processing unit will be described later.

  The input image 200 is image data that is a target of the electronic document generation process of FIG. For example, in the image processing apparatus 100 shown in FIG. 1, the contents of a paper document read by the scanner 101 are document image data converted into electronic pixel information by photoelectric conversion. Alternatively, image data supplied from the outside through the network I / F 105 or image data generated in the image processing apparatus 100 may be used. Specifically, the input image 200 is input to a subsequent processing block in a state where it is stored in the memory 103 or the hard disk 104.

  The output electronic document 210 is electronic data output as a result of the electronic document generation process. The output electronic document 210 is a representation of the contents of the input image 200 in a format that allows the user to display and edit on the application of the personal computer. In the output electronic document 210, according to the contents of characters, figures, photographs, and the like included in the input image 200, the contents are expressed in an appropriate data format for each type. The purpose is to make it possible to output an optimal electronic document for each of different uses such as display, storage, retrieval, editing, and reuse. Specific examples of each data format and electronic document format will be described later.

  The pixel block analysis unit 201 analyzes the pixel content (pixel information) of the input image 200, groups connected pixels regarded as the same color, and forms a connected pixel block. Then, the pixel block analysis unit 201 generates pixel block data 206 including the pixel shape of the formed connected pixel block and the relative positional relationship.

  The layout analysis unit 202 receives the pixel block data 206 generated by the pixel block analysis unit 201 as input, classifies each pixel block into characters and non-characters, and performs grouping. As a result, the layout analysis unit 202 identifies an area existing in the input image 200. The types of areas specified here include character areas, line drawing areas, natural image areas, and table areas. Then, the layout analysis unit 202 generates region data 207 including information on the types of the identified regions, coordinates and relative relationships, and pixel clusters included in the regions.

  The graphics data generation unit 203 receives the area data 207, the pixel block data 206, and the input image 200 as input, and generates graphics data 208 corresponding to the contents of each area in the output electronic document 210. The graphics data 208 is used to generate a graphics object description corresponding to each area in an electronic document description generation unit 205 described later. Specifically, for example, the graphics data generation unit 203 identifies a photo area from the area data 207 and uses the pixel information of the same area in the input image 200 to extract the cut-out image data of the photo portion. Generate. Alternatively, the graphics data generation unit 203 identifies a line drawing area, extracts the outline from the pixel shape information of the corresponding pixel block data, and generates vector data of the line drawing portion by straight line / curve path approximation. Furthermore, the graphics data generation unit 203 generates background image data that is image data in which pixels in the foreground portion such as characters, photographs, and line drawings in the input image 200 are filled with surrounding colors.

  The character recognition unit 204 identifies a character region from the region data 207, and further reconstructs the pixel shape of the character from the pixel block data 206 corresponding to the region as a binary image. Then, the character recognition unit 204 performs character recognition processing on the constructed binary image to obtain a recognized character code string in the character area. Furthermore, the character recognition unit 204 generates character data 209 including these character code strings and other information that can be used on the electronic document. The character recognition unit 204 determines the vertical direction of the input image 200 in order to correctly perform the recognition processing. If the direction is other than the top, the character recognition processing is performed by rotating the binary image and the region information. Also good. The character data 209 may include not only the character code string of the character recognition result but also the format information estimated accompanying the character recognition process, such as the coordinate information of each character, the estimated character size, pitch, and line pitch. . Further, the character data 209 may include color information of each character estimated using color information included in the character pixel block data.

  The electronic document description generation unit 205 receives the region data 207, the graphics data 208, and the character data 209 as input, selects, transforms, and combines them into a format suitable for the application, and outputs the output electronic document 210 as an output. Generate a description. The electronic document description generation unit 205 may generate one page of output electronic document for one input image 200, or may generate one multi-page electronic document for a plurality of input images. Good.

[Operation of each processing unit]
Next, detailed operation examples of the processing units constituting the electronic document generation process of FIG. 2 will be described in order. The processing by each processing unit is realized by the CPU 102 included in the image processing apparatus 100 reading and executing a program stored in the memory 103 or the like that is a storage unit.

(Processing by the pixel block analysis unit)
FIG. 3 shows a flowchart for explaining an operation example of the pixel block analysis unit 201.

  In S <b> 301, the input image 200 is input to the pixel block analysis unit 201. Here, if the input image 200 is a color image, it is assumed that the input image 200 is input in a state of being developed in the memory 103 as a set of pixels having a page size in which each pixel is represented by three 8-bit values for each of RGB. This is merely an example, and may be expressed in a color space other than the RGB format including the gray format. Alternatively, the input image 200 may be input as a stream of compressed images, and the pixel block analysis unit 201 may decompress the RGB pixels or the like in the memory 103.

  In step S <b> 302, the pixel block analysis unit 201 generates a color-reduced image obtained by performing a color-reduction process on each pixel of the input image 200. The range of values that each pixel takes in this subtractive color image is a value of 0 to N (N ≧ 2) that is less than or equal to the pixel value range of the input image 200. Since the color reduction processing method itself is out of the essence of the present invention, a detailed description thereof is omitted. However, the effect of the present invention is exhibited by performing processing so that pixel values after color reduction are not monochrome binary values but pixel values that retain color features such as characters and lines included in the original input image 200. It should be noted that. That is, applying the processing according to the present invention to an image that can have three or more types of pixel values rather than an image having two types of pixel values, such as black and white binary, has the effect of the present invention. It is a premise to get.

  As an example of such color reduction processing, when the input image 200 is in RGB format, each pixel is less than 128 or more than 128 for each of R, G, and B elements. There is a way to reduce the color. There is also a method of subtracting colors by calculating the luminance value Y of each pixel and quantizing this Y into four stages. There is also a method in which N representative colors are estimated from a pixel value histogram of an image, an ID value is assigned to the representative color, and an ID value of the closest representative color is assigned to each pixel.

  In step S303, the pixel block analysis unit 201 performs a known labeling process on a set of connected pixels having the same pixel value in the reduced color image, and extracts a pixel set having the same label as a pixel block. This means that a connected pixel block whose color is approximate in the input image 200 is extracted. For the determination of the connected pixels, 8-connected determination in consideration of the adjacent pixels in all the diagonal directions and the upper, lower, left, and right sides is used. The labeling process using 8-connection determination will be described later with reference to FIGS.

  In S304, the pixel block analysis unit 201 updates the pixel block information so as to hold information indicating the presence or absence of contact between the pixel blocks for all the pixel blocks in the pixel block information generated in S303. Specifically, with respect to the pixel block of interest, the pixel block ID that satisfies both the following conditions: 1) a pixel block that touches or overlaps the circumscribed rectangle, and 2) there is a run in contact with both pixel block runs. The list is registered as a list of contact pixel blocks. This is executed for all pixel block combinations. Note that the above condition is an example, and other conditions may be used. Further, in the process of acquiring the pixel block to be contacted, it is possible to specify the contact relationship at a higher speed by associating the contact labels at the time of labeling in S304. However, since the efficiency of this process is not related to the essence of the invention, its description is omitted.

  In S305, the pixel block analysis unit 201 generates inclusion information indicating that a certain pixel block includes another pixel block or is included in another pixel block for all the pixel blocks in the pixel block information. Add to chunk information. In this example, simple determination processing is performed for defining that the two-pixel block is in an inclusive relationship under the following conditions. 1) The two pixel blocks are in contact, and 2) the circumscribed rectangle of one pixel block completely encompasses the circumscribed rectangle of the other pixel block. This is to reduce the amount of processing during inclusion determination and save processing time. Note that the determination may be performed by using other conditions, and an accurate inclusion determination between pixels may be used.

  In S306, the pixel block analysis unit 201 regards the inclusion relation added to the pixel block information as a parent-child relationship between the pixel blocks, and sets the entire image as an ancestor (root) and each pixel block as a node. Generate a tree structure of chunks. Note that the processing conditions used in S305 may include a pixel block that does not have a parent pixel block. In this case, the tree structure is generated so that the pixel block has the same parent as the parent of the pixel block in contact relation. In some cases, a pixel cluster having a plurality of parents may exist. In this case, the structure may be generated so that only one parent, for example, the deepest parent in the hierarchy has a parent-child relationship. The pixel block tree 206 generated by the pixel block analysis unit 201 is a combination of the pixel block tree structure generated in S306 and the information of each pixel block.

(Labeling process based on 8-connection judgment)
An example of the labeling process based on the 8-connection determination performed in S303 will be described with reference to the flowchart of FIG.

  In S401, the pixel block analysis unit 201 initializes the label value k to 1. In step S <b> 402, the pixel block analysis unit 201 extracts a run in which the same pixel value is continuous in the target line of the reduced color image as the current run. At first, attention is paid to the uppermost line of the subtractive color image, and a range in which pixels having the same pixel value from the leftmost pixel continue in the right direction is extracted as a run. The extracted run is stored as run information including a set of the x coordinate of the start and end points and the y coordinate of the target line. As will be described later, when S402 is processed again on the same line of interest, a run starting from the next pixel at the right end of the processed run is extracted.

  In step S <b> 403, the pixel block analysis unit 201 checks whether there is a run already extracted on the line above the target line, and whether there is a connection between the current run and the pixel value. Here, “8 connected” means that y coordinate = k, x coordinate and start / end point = (s, e), current run of y coordinate k−1, x coordinate (s−1, e + 1). The condition is that the run exists even in one pixel and the pixel values are the same. In addition, when the attention line is the top line, there is always no connected run. If there is no connected run that meets the conditions (NO in S403), the process proceeds to S404, and the pixel block analysis unit 201 assigns a new label Lk to the current run. In step S405, the pixel block analysis unit 201 sets +1 to the label value k. On the other hand, if there is a linked run that meets the conditions in S403 (YES in S403), the process proceeds to S406.

  In step S <b> 406, the pixel block analysis unit 201 checks whether there are a plurality of linked runs that meet the condition and whether the linked run has a plurality of label types. If there are a plurality of label types (YES in S406), the process proceeds to S407, and the pixel block analysis unit 201 assigns the label of the first detected connected run to the current run. Further, the pixel block analysis unit 201 associates the label values so that the labels of all the connected runs are regarded as the same group. On the other hand, if there is a single connected run, or if multiple connected runs have the same type of label (NO in S406), the process proceeds to S408, and pixel block analysis unit 201 uses the label that the connected run has. Grant to the current run.

  After S405, S408, and S409, the process proceeds to S409, and the pixel block analysis unit 201 checks whether or not the next run is on the line of interest, that is, whether or not the end point of the current run is the right end of the image. If there is a next run (YES in S409), the pixel block analysis unit 201 proceeds to S402 so that the run is extracted, and thereafter repeats the processing. If there is no next run on the target line (NO in S409), the process proceeds to S410.

  In step S410, the pixel block analysis unit 201 checks whether the target line is the last line. If it is not the last line (NO in S410), the process proceeds to S411, and the pixel block analysis unit 201 moves to the next line. Then, returning to S402, the pixel block analysis unit 201 extracts a new run from the pixel at the left end of the line and repeats the subsequent processing. If the target line is the last line (YES in S410), the process proceeds to S412.

  In step S412, the pixel block analysis unit 201 creates pixel block information including a set of runs to which the label is assigned for each label value. When configuring this set of runs, processing is performed so that runs having a plurality of types of label values associated in S407 are collected into one pixel block run information. As pixel block information to be finally generated, one pixel block includes a set of ID for identification, circumscribed rectangle information, pixel value, and a set of run information collected in the pixel block.

(Example of labeling process)
An example of processing when the labeling processing of FIG. 4 is applied is shown in FIG. FIG. 5A shows an example of a subtractive color image to be processed. An image having a width of 6 × 3 pixels in which one square represents one pixel, and a numerical value in each square represents a pixel value. FIG. 5B is an example of the result of the labeling process with respect to FIG.

  In the labeling process, first, paying attention to the uppermost line (line of y = 0), a run 501 having a pixel value of 3 starting from the left end is extracted. Since there is no line above this, there is no connected run and the run 501 is given the first label L1. A new label L2 is similarly given to the subsequent run 502 having a pixel value of 1.

  Since there are no more pixels in the uppermost line, the process moves to the next line (y = 1 line), and a run 511 having a pixel value of 1 is extracted. Since there is no run of pixel value 1 connected to the run 511 in the upper line (line of y = 0), the new label L3 is given to the run 511. The next run 512 with a pixel value of 3 has a run 501 with the same value on the upper line (line with y = 0). Since this run 501 is the only connected run, the label L1 of the run 501 is also given to the run 512. Subsequently, a run 513 with a pixel value of 1 is extracted, and a label L2 of a run 502 that is a connected run of the upper line (y = 0 line) is also given. Subsequently, a run 514 with a pixel value of 3 is extracted, and since there is no connected run, a new label L4 is given. Moving to the third line (y = 2 line), a run 521 with a pixel value of 1 is extracted, and a label L3 of a run 511, which is the upper connected run, is given. A subsequent run 522 with a pixel value of 3 includes a run 512 and a run 514 as a connected run of the upper line (line of y = 1). In addition, since the label values of the run 512 and the run 514 are different from those of L1 and L4, the run 522 is given the label L1 of the run 512 detected first among the connected runs. In addition, association information for regarding the labels L1 and L4 as the same label is generated. Finally, a run 523 with a pixel value of 2 is extracted and given a new label L5.

  FIG. 5C shows an example of pixel block information generated from the labeled run set. The pixel block of ID1 is configured as a collection of runs 501, 512, and 522 of label L1 and runs 514 of label L4 based on the association described above. Further, (0,0)-(5,2) and pixel value 3 are stored as the rectangular range. The pixel block of ID2 is a pixel block having a rectangular range (2, 0)-(5, 1) and a pixel value of 1 constituted by the run 502 and the run 513 of the label L2. Hereinafter, ID3 and ID4 are also shown in the same manner. Note that once the pixel block information is configured, the label information Lk used in configuring the run set may be discarded. Further, the coordinates of each run information may be reset so that the upper left corner of the circumscribed rectangle of each pixel block is the origin as shown in FIG.

(Example of processing results by the pixel block analysis unit)
FIG. 6 shows an example of the processing result by the pixel block analysis unit 201. FIG. 6A shows an example of a subtractive color image input to the pixel block analysis unit 201. FIG. 6B is an example of pixel block information extracted from the color-reduced image shown in FIG. 6A, and is a diagram showing the contact relationship of the pixel blocks generated in S304 with arrows. FIG. 6C is a diagram showing the inclusion relationship generated in S305 with arrows in the example of the pixel block information shown in FIG. In this arrow, the tip of the arrow indicates a child, and the original indicates a parent. FIG. 6D is an example of pixel block tree information configured from FIGS. 6B and 6C. Since the pixel block 601 in FIG. 6D has no parent pixel block due to the inclusion relationship, the tree structure is constructed so that the pixel block 602 that is in contact with the pixel block 602 is the parent.

  In the flowchart of FIG. 3, it has been described that the entire input image data is processed at once by the pixel block analysis processing. On the other hand, the input image data may be divided into a plurality of parts, and input of each partial image and extraction of pixel block information may be repeated. For example, Patent Document 3 describes an example in which 32 pixel square tiles are used as one processing unit, and image input, quantization, and creation of a blob that is a pixel block within the tile are repeated in order from the upper left of the image. Yes. Further, in the description according to Patent Document 3, by combining the blobs in the upper tile and the left tile as processed tiles with the blob of the current tile, a pixel having an arbitrary size including the same size as the input image 200 as a result. A lump has been generated. By applying this processing method, it is possible to greatly save the memory and processing time consumed for the pixel block data generation processing according to the present embodiment.

(Processing by the layout analysis unit)
Next, the processing of the layout analysis unit 202 will be described using the flowchart of FIG. In this process, the pixel block data 206 on the memory 103 is input, and area data 207 based on the structure between document areas such as characters, line drawings, natural images, and tables is configured on the memory 103.

  In S701, the layout analysis unit 202 classifies each pixel block in the input pixel block data 206 into a character candidate pixel block and other pixel blocks. The classification of whether or not the pixel block is a character candidate may be performed using a character pixel block determination method used in a known document image analysis technique. For example, there is a method of using a circumscribed rectangular size of a pixel block and using a character candidate that falls within a predetermined height and width range.

  In this example, the size considered as a character is from 6 points to 50 points, and a character candidate that has a pixel block width or height that falls within the number of pixels Tmin to Tmax converted from the resolution of the input image 200 is set as a character candidate. . Here, setting a lower limit on the size has an effect of preventing a character candidate from including a small pixel block corresponding to the background extracted from the inside of the character. Pixel density, ratio, pixel color, and the like may be further added to the character candidate determination conditions. The size threshold may be determined dynamically by obtaining frequency information of width and height from a set of pixel blocks actually extracted from the input image 200.

  In step S <b> 702, the layout analysis unit 202 groups the pixel candidates for character candidates classified in step S <b> 701 with respect to those adjacent to each other. The determination as to whether or not the pixel is in the vicinity can be made by calculating the Euclidean distance between the circumscribed rectangular coordinates of the pixel block and not more than a predetermined threshold value. This is merely an example, and another calculation method such as a city block distance may be used for the distance calculation. In addition, since a plurality of characters are described in a line, and the character spacing within the line is generally narrower than the line spacing, character candidate pixel blocks forming a character line are grouped with a small distance threshold, and a plurality of characters are Lines may be grouped. In this grouping process, only character candidate pixel blocks having the same parent in the pixel block tree structure are to be grouped. This is to reduce the number of pixel block combinations to be subjected to the neighborhood calculation and speed up the processing.

  In step S703, the layout analysis unit 202 determines, for each group, whether the set of character candidate pixel blocks grouped in step S702 is actually a character set. Then, the layout analysis unit 202 specifies the existence ranges of the pixel blocks of the group that is regarded as a character set as character regions. Then, for each identified area, area information including the area coordinates and association information to the corresponding pixel block is stored as a component of the area data 207. That is, for the character region, association information to the grouped character candidate pixel set and circumscribed rectangular coordinate information surrounding the same pixel block are stored as character region information.

  For example, there is a method for determining whether or not the character set is a character set by determining whether a character candidate pixel block is projected vertically and horizontally in a rectangular range including a group, and indicating whether or not the character string is aligned. Specifically, in horizontal projection for horizontal writing and vertical projection for vertical writing, if a frequency distribution with peaks in the line part and valleys in the line is observed, it is determined that there is a high possibility of a character area. it can. In addition, with the exception of italics and other characters, the circumscribed rectangles do not overlap significantly. Therefore, the fact that there is no large overlap with other pixel blocks is also an effective means for determining whether or not it is a character area. However, in order to exclude the case where one character is divided into a plurality of overlapping pixel blocks such as kanji, it is effective to limit the overlap determination to only between pixel blocks having a certain size or more.

  After the character area is specified, another pixel block in the same area may be added to the set of character candidate pixel blocks. For example, there is a high possibility that a pixel block of an independent point portion within a punctuation mark or character is not selected as a character candidate pixel block due to size restrictions. In order to include in these character candidate pixel blocks, a process of adding a small-sized pixel block having the same color as that of the pixel block that is already a character candidate in the vicinity may be performed.

  In step S <b> 704, the layout analysis unit 202 selects a pixel block as a line drawing / table frame candidate from the pixel blocks classified in addition to the character candidates in step S <b> 701. The determination as to whether or not the candidate is a line drawing / table frame can be made based on a size larger than that of the character candidate and a low pixel density with respect to the entire pixel cluster existence range.

  In step S705, the layout analysis unit 202 determines whether or not the pixel block determined as the line drawing / table frame candidate in step S704 is a table frame, and displays the existence area of the pixel block determined as the table frame. Specify as an area. The layout analysis unit 202 stores the association information of the corresponding table frame with the pixel block and the region information including the circumscribed rectangular coordinates of the pixel block in the region data 207 as the table region information.

  Whether or not the table area information table frame is present may be determined, for example, by calculating a vertical and horizontal pixel histogram from the run information of the pixel block in the presence range of the pixel block and determining the shape. That is, if a pixel block corresponds to a front frame, the fact that a plurality of sharp peaks are generated on the histogram in a portion where vertical and horizontal outer frames and ruled lines exist is utilized. By detecting these, it is possible to determine whether the frame is a table frame. Alternatively, the determination can be made based on a set of pixel blocks corresponding to the children of the table frame pixel block. Since the pixel block that is a child of the table frame corresponds to a frame region in the table, all the child regions are rectangular and aligned without overlapping, which is an effective determination means for being a table frame. Become.

  In step S <b> 706, the layout analysis unit 202 identifies a pixel block existence area that has been determined to be a line drawing / table frame candidate in step S <b> 704 and has not been determined to be a table frame in step S <b> 705 as a line drawing area. Then, the layout analysis unit 202 stores the association information of the corresponding line drawing with the pixel block and the region information including the circumscribed rectangular coordinates of the pixel block in the region data 207 as the line drawing region information. At this time, a range obtained by grouping pixel blocks in the vicinity of the pixel block determined to be a line drawing may be used as the line drawing region.

  In step S707, the layout analysis unit 202 selects a pixel block or a set of pixel blocks determined as a natural image region such as a photo from pixel blocks that do not correspond to any of the regions stored so far. The existing area is stored as natural image area information. Then, the layout analysis unit 202 stores the association information to the corresponding pixel block set and the region information including the coordinates of the existence range of the pixel block as the natural image region information in the region data 207.

  Whether or not it is a natural image area is determined so that pixel clusters of multiple colors exist so that they overlap or are included, and the set of pixel clusters forms a rectangle within a certain size. The natural image area corresponding to the rectangular photograph is determined. This determination is merely an example, and an arbitrarily shaped pixel block set may be targeted.

  In step S <b> 708, the layout analysis unit 202 stores a pixel block having a density and area of a certain level or more as a flat region from pixel blocks that do not correspond to any region stored so far. An area that occupies the entire plain page, a colored background area that is colored in order to provide a meaningful grouping of characters and the background of a figure, and the background of a table cell correspond to this flat area.

  In step S709, the layout analysis unit 202 configures an area tree that expresses the relative relationship with each area stored in the area data 207 by S708 as a node. A special root node corresponding to the entire input image range is arranged at the starting point of the region tree. The parent-child relationship between the nodes of the region tree is made to match the parent-child relationship of the pixel block node corresponding to each region in the pixel block tree. As a specific process for constructing the tree structure, the tree structure is constructed by giving each region data a list of link information to the parent region and link information to the child region.

(Example of processing result by layout analysis unit)
An example of the result of processing in the layout analysis unit 202 described with reference to the flowchart of FIG. 7 will be described with reference to FIG. FIG. 9A is an example of a document image that is reduced in color by the pixel block analysis unit 201 and decomposed into pixel blocks. FIG. 9B is an example diagram in which pixel blocks extracted from the document image shown in FIG. 9A are expressed by a pixel block tree structure. 9A and 9B, pixel blocks 901, 902, and 903 are sets of pixel blocks corresponding to characters in the original image. Pixel blocks 904 to 907 are pixel blocks constituting a table. A pixel block 908 is a pixel block corresponding to a star-shaped line drawing. Pixel blocks 909 and 910 are pixel blocks corresponding to a photograph. In FIG. 9, the description of the small pixel block inside the character is omitted.

  In FIG. 9B, arrows indicate parent-child relationships based on inclusion relationships. For example, in the pixel blocks 904 to 907 constituting the table, the pixel block 905 is a monochrome background area in the left frame of the table. Since the pixel block 905 includes a pixel block 902 that is an area of two characters inside, the two have a parent-child relationship. On the other hand, the background area in the right frame of the table is divided into pixel clusters 906 and 907 of different colors in the vertical direction. As a result, since the character pixel block 903 is not included in either pixel block, it becomes a direct child of the table block pixel block 904 serving as the parent. Such a case may occur when the original input image is originally color-coded as described above, or the monochromatic region is unintentionally excessively divided due to noise or color reduction processing. In any case, it should be considered that this is a case that occurs universally in color pixel block structure extraction processing.

  FIG. 9C is an example of a region tree generated from the pixel block tree of FIG. 9B, and the generation process will be described below according to S701 to S709 of the flowchart of FIG. First, in S701 to S703, the layout analysis unit 202 selects three pixel block groups of pixel blocks 901, 902, and 903 as character candidate pixel blocks. The layout analysis unit 202 stores child area nodes 921, 922, and 923 in which each existence range that satisfies the character area determination condition is a character area.

  In step S704, the layout analysis unit 202 selects the pixel blocks 904 and 908 as line drawing / table frame candidates. In step S705, the layout analysis unit 202 determines that the pixel block 904 is a table frame, and stores a child region node 924 as a table region. In step S706, the layout analysis unit 202 determines that the pixel block 908 is a line drawing, and stores a child area node 928 as a line drawing area.

  In step S707, the layout analysis unit 202 determines that the pixel blocks 909 and 910 constitute a natural image area, and stores a child area node 929 as the natural image area. In S708, the layout analysis unit 202 stores the remaining pixel blocks 900, 905, 906, and 907 as flat regions. In step S709, the layout analysis unit 202 uses each region as a node and generates a region tree structure that reflects the parent-child structure of the corresponding pixel block tree. In FIG. 9C, areas connected by lines have a parent-child relationship.

(Processing by the graphics data generator)
Next, the processing of the graphics data generation unit 203 will be described. The graphics data generation unit 203 generates graphics data 208 for expressing each area included in the area data 207 as a graphics object. The data generated here is used when the contents of each area are described as an object in the electronic document description generation unit 205 described later. Hereinafter, the processing of the graphics data generation unit 203 will be described with reference to the flowchart of FIG.

  In step S <b> 801, the graphics data generation unit 203 generates vector data for expressing the object of the line figure part in the output electronic document 210 with graphics. The vector data generation target area in this example is a line drawing area and a table area existing in the input area data 207. The generated vector data is stored in the memory 103 or the hard disk 104 after being associated with the corresponding area node in the area data 207. For vector data generation, a known vectorization method, that is, a straight line or curved path approximation method is used from the outline information of the pixel block associated with the target region. It is assumed that color information associated with a pixel block is set as a paint color of a path generated from each pixel block.

  In step S <b> 802, the graphics data generation unit 203 generates cutout image data for expressing an area that is not vectorized in the output electronic document 210. The cutout image data generation area in this example is a natural image area (natural image area) existing in the area data 207. The extracted image data is stored in the memory 103 or the hard disk 104 after being associated with the corresponding area node in the area data 207. Here, the clipping process is a process of referring to the input image 200 and generating image data of the same size including only pixels in the target range. The clipped image data may be compressed by a known compression technique such as JPEG.

  In step S <b> 803, the graphics data generation unit 203 generates background image data used for the background in the output electronic document 210. The generated background image data is associated with the root node of the area data 207 and stored in the memory 103 or the hard disk 104.

  A background image is such that the output electronic document 210 has the same appearance as the input image 200 by rendering the vector data and cutout image data generated in S801 and S802 as foreground data and overlaying the background image. It will be prepared. For the background image data, a process of erasing the pixel information of the area where the foreground data exists from the input image 200 is performed.

  Erasing pixel information has the effect of preventing the data from appearing double in the synthesized output electronic document 210. Alternatively, there is an effect that compression efficiency is improved by eliminating useless pixel information existing in an area hidden by superposition, and the output electronic document is made compact. For example, the pixel information can be erased by uniformly painting a rectangular area of the target area with surrounding colors. When the target area is a line figure area, if only the pixels corresponding to the line portion are filled with the neighboring pixel color, the color information of the portion other than the line portion can be left in the background information.

  In S801 and S802, for which type of region the vectorization process or the image cutout process is performed is not limited to the above-described example. For example, image cutting processing may be performed on a line drawing and a table area. In addition, a character area may be added to the vectorization target area. In these target selections, the processing target area type may be set by an external instruction as a control item of the electronic document generation process. In addition, when there are a plurality of formats of electronic documents to be generated and each has a different use, a data format suitable for each use may be changed for each region type.

  In addition, it may be set as a control item of the electronic document generation process for which type of region the pixel information erasure process is performed at the time of generating the background image data in S803. You may make it change together. Even when a character area is not a vectorization target, when character data 209 is output in the processing of the character recognition unit 204 described later, background data is generated so that character pixels are removed. It may be.

  FIG. 9 shows an example of background data generated for the example of the input image shown in FIG. In FIG. 9A, the pixel block 901 to 903 which is a character area, the pixel block 908 which is a line drawing area, and the line figure partial pixel of the pixel block 904 which is a table frame area are filled with surrounding pixel colors. . In addition, a rectangular range of the pixel block 909 that is a natural image region is filled with surrounding pixel colors.

(Processing by the character recognition unit)
The processing of the character recognition unit 204 will be described with reference to the flowchart of FIG. In step S1001, the character recognition unit 204 generates a character image to be input to the character recognition process. In this description, in the character recognition process, it is assumed that a binary image including characters is input, and a binary image of each character region is generated. A binary image of a character area is a binary image having the same number of pixels as the input image, where 1 is the character pixel in the area and 0 is the other. In actual processing, for each character area in the area data 207 generated by the layout analysis unit 202, the pixel block information existing in the area is read from the pixel block data 206 generated by the pixel block analysis unit 201. Then, an image of the same size as the input image 200 is generated so that the run portion of each pixel block is 1 and the others are 0.

  In S <b> 1002, the character recognizing unit 204 may not input the document image including the character, that is, the upward direction of the character written in the input image 200 may be 90 °, 180 °, or 270 °. Is assumed. Then, the character recognition unit 204 performs a direction determination process for determining a rotation angle necessary to correct them in the correct direction.

  The direction discrimination processing here is performed by a known method using the binary image generated in S1001. Since the method of direction discrimination processing is different from the essence of the present invention, the details are omitted. As an example of the direction discrimination processing, some characters in the image are recognized in four directions in a state rotated to 90 °, 180 °, and 270 ° in addition to 0 °, and the recognition score at that time is the highest. One way is to determine the direction as the correct direction.

  In S1003, the character recognition unit 204 checks whether or not the necessary rotation angle obtained in S1002 is 0 °. If the required rotation angle is 0 °, that is, if there is no need for rotation (YES in S1003), the process proceeds to S1006. If any rotation of 90 °, 180 °, or 270 ° is necessary (NO in S1003), the process proceeds to S1004.

  In S1004, the character recognition unit 204 rotates the binary image generated in S1001 by the necessary rotation angle obtained in S1002. In S1005, the character recognition unit 204 adds the rotation angle obtained in S1002 to the area data 207 as rotation information.

  In step S1006, the character recognition unit 204 executes a known character recognition process in each character region using the rotated binary image and character region information, and obtains a character recognition result including a character code string. When the rotation angle is other than 0 °, it is assumed that the area information specified in the character recognition process is also rotated so as to match the area on the rotated binary image. The character recognition result includes character area information, line information, and recognized character information. The character area information includes information on the coordinates of a range where characters exist as position information and the number of recognized character lines. The line information includes information on the number of characters in each line. The recognized character information includes a character code recognized for each character and information of circumscribed rectangular coordinates of the character. Information of each character additionally obtained by the character recognition process may be added to the recognized character information. For example, character size estimated from the average character height and pitch within a line, character modification information such as bold, italic, and underline, font type, and the like may be added.

  In step S1007, the character recognition unit 204 adds color information to each character of the character recognition result output in step S1006. In this process, the circumscribed rectangular coordinates of each pixel block and the pixel value, that is, the color information of the pixel block, stored in the pixel block data 206 are used. However, the character recognition process is performed by designating a binary image in units of character areas, and the character unit as a result of the character recognition process is not associated with the pixel block of the pixel block data 206.

(Color information addition processing)
The process of S1007 by the character recognition unit 204 will be described with reference to the flowchart of FIG. In step S <b> 1201, the character recognition unit 204 selects one unprocessed pixel block as the processing target C from the pixel block data 206 belonging to the character area of the area data 207. At this time, the character recognition unit 204 clears the processing information of the character data 209. Since the processing order for the pixel block data varies depending on the processing method of the pixel block data, the input direction of the original, and the like, it is not assumed to be indefinite, that is, the reading order of characters. When a rotation angle other than 0 ° is given to the area data 207, coordinates rotated by the rotation angle are used in the processing of the following steps for the pixel block C to be processed. Thereby, when the direction of the input image is different from the normal orientation at the time of character recognition processing, it is possible to eliminate the mismatch of the data obtained from both.

  In step S <b> 1202, the character recognition unit 204 selects one of the unprocessed characters from the character data 209 as the processing target O. In step S <b> 1203, the character recognition unit 204 determines whether the circumscribed rectangle overlaps the pixel block C and the character data O area. If the circumscribed rectangles of the C and O regions overlap (YES in S1203), the process proceeds to S1204. If the circumscribed rectangles of the areas C and O do not overlap (NO in S1203), the process proceeds to S1202, and the character recognition unit 204 sets the next unprocessed character data as the processing target O.

  In step S1204, the character recognition unit 204 determines whether the pixel block C is already associated with the character O. If not associated (NO in S1204), the process proceeds to S1206. If it is associated (YES in S1204), the character recognition unit 204 sets the pixel block C as the related pixel block Oc of the character O, and proceeds to S1205. Here, it can be determined whether the character and the pixel block are associated with each other based on whether color information is added to the character.

  In step S <b> 1205, the character recognition unit 204 determines whether the overlapping area of the pixel block Oc and the character O already associated with the character O is approximate. Here, the character recognizing unit 204 determines the size of the overlap area of the pixel Oc and the character O already associated with the character O and the size of the overlap area of the pixel block C and the character O. Alternatively, it may be determined simply by the size of the overlap between the pixel block C and the character O and the proximity of the area of the character O, that is, the size of the area where the pixel block C covers the character O.

  This step has the effect of preventing the addition of the noise color existing in the character rectangle by ignoring the small area, and the effect of associating with the pixel block having more accurate color information. As described above, a character pixel block and a character do not necessarily correspond one-on-one. For example, in a Chinese character consisting of bias and 旁, one pixel block is formed for both bias and 旁. Either color information may be used as long as the bias and the color are the same, but the color of the character outline portion is generally different due to the problem of ink bleeding and the optical resolution of the scanner. Therefore, in a small pixel block, the area occupied by the contour portion becomes large, and there is a high possibility that inaccurate color information is extracted. Therefore, a pixel block estimated to have more accurate color information is selected by comparing the areas. If the area is approximate (YES in S1205), the process proceeds to S1206. If the area is not approximate (NO in S1205), the process proceeds to S1202, and the character recognition unit 204 sets the next unprocessed character data as the processing target O.

  In step S <b> 1206, the character recognition unit 204 adds color information of the associated pixel block C to the character O. In the present embodiment, the color information of the related pixel block Oc with respect to the character O can be referred to by associating the pixel block C as the related pixel block Oc of the character O. Then, it progresses to S1207.

  In step S <b> 1207, the character recognition unit 204 determines whether processing has been completed for all characters of the character data 209. If processing has ended (YES in S1207), the process advances to S1208. If there is unprocessed character data (NO in S1207), the process proceeds to S1202, and the character recognition unit 204 sets the next unprocessed character data as the processing target O.

  In step S <b> 1208, the character recognition unit 204 determines whether processing has been completed for all pixel block data of the pixel block data 206. If there is unprocessed pixel block data (NO in S1208), the process proceeds to S1201, and the character recognition unit 204 sets the next unprocessed pixel block data as a processing target C. If processing has been completed for all pixel blocks (YES in step S1208), the process advances to step S1209.

  In step S1209, the character recognition unit 204 performs a coloring process on character data without color information. Since the character data output from the character recognition unit 204 is based on a pixel block, there is a matching pixel block in the output character data, but it matches exceptionally when an invisible character such as a blank is recognized. There is no pixel block. If the previous character exists, the character recognition unit 204 adds the same character color as that of the previous character to the color information. If the previous character does not exist, the character recognition unit 204 refers to the color information of the subsequent character. If the character does not exist, the character recognizing unit 204 traces the character behind and adds the character color when the character color exists. By performing this process on all character data having no color information, the color information of the preceding or following character string is attached to all blank characters. In the present embodiment, the color of the preceding character is preferentially added, but the color of the subsequent character may be preferentially added.

(Example of processing result of color information addition processing)
Hereinafter, the processing of the flowchart of FIG. 12 will be described using FIG. 11 as an example of the character region to be input. FIG. 11A shows an example of a character area, which includes eight characters consisting of three lines “ab c”, “123”, and “if” and a space (blank character). Here, “ab c” is black, “123” is red, and “if” is blue. It is assumed that a space is included between “ab” and “c” in the first line.

  FIG. 11B is an example of the result of character recognition in FIG. In this example, character code information, rectangular coordinate information, estimated character size, and line information of nine characters from character O01 to character O09 are recognized. Since character recognition is performed based on binary information, color information is not added.

  FIG. 11C is an example of the pixel block data 206 obtained from the pixel block analysis unit 201 for the character area. Nine pixel blocks represented by C01 to C09, their circumscribed rectangular coordinates, and color information are stored. In this example, the color information is expressed in three-digit hexadecimal RGB notation of red component 4 bits, green component 4 bits, and blue component 4 bits. At this time, the value of the color information is “# 000” for black, “# F00” for red, and “# 00F” for blue. Since the pixel block data is composed of connected pixels as described above, the character recognition result and the area are different. For example, there is no pixel block for a portion recognized as a space in the character recognition result.

  Further, as shown in FIG. 13A, the character rectangle of the character O08 is one area surrounded by a solid line 1300, whereas the pixel block is two areas C07 and C08 surrounded by a broken line 1301 and a broken line 1302. Divided into regions. In addition, although the color information of the pixel block of C07 is actually “# 00F” representing blue, it is obtained as “# 33F” due to the blur of characters or the scan resolution. Character bleeding and color error due to scan resolution will be described with reference to FIG.

  FIG. 14A is an enlarged view of the scanned image of the letter “i” in FIG. A region 1401 composed of 3 × 3 small regions is obtained in a lighter color than the original color due to the influence of color blur and scan resolution. Since the area 1402 is larger than the area 1401, there are few areas that are affected by color blur or scan resolution.

  FIG. 14B illustrates the result of the color reduction process (S302) performed on FIG. 14A. The pixel block in the area 1401 is gathered lighter than the original color by the color reduction processing.

  Next, the color information addition processing for the information shown in FIG. 11 will be specifically described along the flowchart of FIG.

  Assume that the pixel block C01 shown in FIG. 11C is selected in S1201. In S1202, the unprocessed character O01 shown in FIG. 11B is selected, and in S1203, the determination of the circumscribed rectangle overlap is performed. At this time, the circumscribed rectangle of the pixel block C01 is included in the circumscribed rectangle of the character O01, and it is determined that the pixel rectangle C01 overlaps, and the process proceeds to S1204. Here, since color information is not added to the character O01, the process proceeds to S1206, and is associated with the character C01, and the color information “# 000” of the pixel block C01 can be referred to as the color information of the character O01. By similar processing, pixel block C02 and character O02, pixel block C03 and character O04, pixel block C04 and character O05, pixel block C05 and character O06, pixel block C06 and character O07, and pixel block C07 and character O08 are associated with each other. .

  Next, it is assumed that the pixel block C08 is selected in S1201, the character O08 is selected in S1202, and the process proceeds to S1203. At this time, since the circumscribed rectangle of the pixel block C08 and the circumscribed rectangle of the character O08 overlap, the process proceeds to S1204. In S1204, since the pixel block C07 is already associated with the character O08, the process proceeds to S1205. In S1205, the difference in the area where the pixel block C07 already associated and the character O08 overlap is compared.

  The entire area of the pixel block C07 is included in the character O08, and the overlapping area is the area of the pixel block C07 indicated by the region 1311 in FIG. 13A, that is, 3 × 3 = 9. On the other hand, in the pixel block C08 and the character O08, the entire region of the pixel block C08 is included in the character O08, and the rectangular area 7 × 13 = 91 indicated by the region 1312 in FIG. In this case, it is determined that the pixel block C08 is closer to the character O08, and the pixel block C08 and the character O08 are associated in S1206. As a result, the color information of the character O08 is changed from “# 33F” of the pixel block C07 to “# 00F” of C08.

  Subsequently, the process proceeds to S1201, and the pixel block C09 and the character O09 are associated with each other. Processing is completed for all pixel blocks, and the process advances to step S1209. FIG. 13B shows color information values for character data at this point. Since the character O03 is a blank character (space), there is no corresponding pixel block and no color is added. In step S1209, the character recognition unit 204 adds the preceding and following color information to the character O03 to which color information is not added. Since the color information # 000 is added to the previous character O02, the character recognition unit 204 adds “# 000” to O03 and ends the process. FIG. 13C shows an example of character data after the entire processing of the character recognition unit 204 is completed.

  When the processing of S1205 is not performed, it is uncertain which color of C07 or C08 is added to O08, and therefore C07 “# 33F” that is not the original color may be added.

(Processing by electronic document description generator)
The processing of the electronic document description generation unit 205 will be described with reference to the flowchart of FIG. In step S <b> 1501, the electronic document description generation unit 205 outputs data describing the start portion of the output electronic document 210. In this description, the output destination is an output buffer secured in the memory 103 or the hard disk 104. Thereafter, each time data is output in the process, the contents are stored in the output buffer so as to be added to the end of the output data.

  In step S <b> 1502, the electronic document description generation unit 205 outputs data describing the start part of the page in the output electronic document 210. In the electronic document generation process according to the present embodiment, the content of one input image is assumed to correspond to one page of the output electronic document 210. In step S <b> 1503, the electronic document description generation unit 205 sets the root node in the area data 207 as the attention node that is the first processing target.

  In step S1504, the electronic document description generation unit 205 checks whether the target node is an output target area. If it is an output target area (YES in S1504), the process proceeds to S1505. If it is not an output target area (NO in S1504), the process proceeds to S1506. Here, whether or not the node of interest is an output target area is determined based on a definition table set in the electronic document description generation unit 205. In this definition table, the presence / absence of output and the method (data format) are defined for each of the foreground / background, character, line drawing, natural image, and table area types.

  FIG. 16 shows an example of an output target area definition table. In FIG. 16, two types of definition tables, a definition 1601 and a definition 1602, are defined. Which table is used may be instructed in advance in the electronic document generation process, or may be automatically selected in the electronic document generation process depending on the input content.

  In step S <b> 1505, the electronic document description generation unit 205 outputs graphics data 208 or character data 209 associated with the area for the node of interest. Note that rotation angle information other than 0 ° may be given to the region data 207 in S1005 of FIG. This corresponds to a case where the orientation of the input image 200 is different from the normal orientation in which characters can be read. As described with reference to FIG. 10, the character data associated with the region at this time is obtained from the binary image rotated in the normal direction, and has coordinates in the normal direction. On the other hand, the graphics data 208 has coordinates in a direction different from the normal position obtained from the input image 200. In order to eliminate these inconsistencies, data obtained by rotating the coordinates in the normal direction is output to the graphics data 208.

  In step S <b> 1506, the electronic document description generation unit 205 acquires a next node that is a node on which output processing should be performed next to the node of interest. In step S1507, the electronic document description generation unit 205 determines whether the next node has been acquired in step S1506. If the next node can be acquired (YES in S1507), the process proceeds to S1508, and if it cannot be acquired (NO in S1507), the process proceeds to S1509. In step S1508, the electronic document description generation unit 205 returns to step S1504 with the next node acquired in step S1506 as a new node of interest, and repeats the subsequent processing. In step S <b> 1509, the electronic document description generation unit 205 outputs page end data on the assumption that there is no region node to be output in the region data, that is, the data description for one output page has been completed.

  In step S1510, the electronic document description generation unit 205 checks whether there is an additional page. An additional page is generated when an additional image is input when the electronic document generation process operates to output a multi-page electronic document. If there is an additional page (YES in S1510), the electronic document description generation unit 205 returns to S1502 and repeats the subsequent processing. If there is no additional page, that is, if no more images are input (NO in S1510), the process proceeds to S1511.

  In step S1511, the electronic document description generation unit 205 outputs the end data in the electronic document data. Completed electronic document data is constructed on the output buffer by outputting the end data. In step S1512, the electronic document description generation unit 205 transmits the electronic document data on the output buffer to the PC or the like designated in advance by the user as the output electronic document 210, and ends the electronic document generation process.

  In this description, the entire electronic document data is processed to be written to the output buffer. However, the electronic document data may be processed with a smaller output buffer size. For example, when the end data of each page is output, the content of the same page may be transmitted to the designated transmission destination, and the content of the next page may be written again from the top of the output buffer. Alternatively, writing to the output buffer, transmission, and clearing may be repeated in smaller units.

(Example of electronic file conversion)
FIG. 17 is a part of an example in which the character recognition result of FIG. 13 is converted into electronic document data by the electronic document description generation unit 205. In this example, the virtual format is expressed in an XML (extensible Markup Language) format. The description format is not limited to this, and other formats may be used. The textLine element is a single-line character string, and the run element is a group of character strings having the same modification information. A character string surrounded by text elements is a character string that is actually output, and includes a color attribute indicating color information and a size attribute indicating character size. FIG. 17A is a part of an example in which the state where the processing of S1205 and S1209 in FIG. 12 is not performed is converted into an electronic file.

  FIG. 17B is a part of an example in which the processing of S1209 is performed, that is, FIG. 13C is converted into an electronic file. In the example of FIG. 13B, since there is no color information in the space between “ab” and “c” in the first row, three run elements are created. Here, “&nbsp;” is a symbol representing a space. In addition, since the color information of “i” in the third row becomes “# 33F”, two run elements are created in the third row, and a total of six run elements in three rows are created.

  On the other hand, in FIG. 13C, the color information of the previous character is added to the space by the processing of S1209, so that “ab c” on the first line is represented by one run element. Since the color information is “# 00F” by the processing of S1205 of the third line “i”, it is expressed by one run element, and is described by a total of three run elements. Since the space is an invisible character code, it does not affect the display of the electronic document.

  In this manner, by applying the present invention, it is possible to omit the description of characters indicating white space such as spaces after adding color information to the text, and to reduce the amount of description.

<Second embodiment>
In the first embodiment, color information is added in step S1209 of FIG. 12, but not only color information but also character modification information may be added to undefined characters. Hereinafter, processing using FIG. 18 as an example of the input character area will be described. Examples of the modification information other than the color information here include character size, font shape, and information for decorating characters.

  FIG. 18A shows an example of a character area, which is composed of one line of character string “ab c”. At this time, the character strings of the characters a, b, and c are green, and the font shape is a character drawn with decoration information such as Gothic, italic, and bold.

  FIG. 18B is an example in which color information is added to the character recognition result by the processing of S1206. Four characters of character codes 0x61, 0x62, 0x20, and 0x63 are recognized. Further, it is assumed that the character codes 0x61, 0x62, and 0x63 have size information of the character size “14”, Gothic font shape information, italics, bold decoration information, and “# 0F0” color information representing green. For “0x20” representing a space, font shape information, decoration information, and color information are undefined.

  FIG. 18C is an example in which font shape information, decoration information, and color information before and after are added in the processing of S1209 to FIG. 18B. Here, it can be seen that information is added to the space.

  FIG. 19 is an example in which the example of the recognition result of FIG. 18 is converted into electronic document data. FIG. 19A shows the conversion result of FIG. 18B, and FIG. 19B shows the conversion result of FIG. 18C. In addition to the XML format text element described in FIG. 17, a font attribute representing a font shape, a b attribute representing bold, and an i attribute representing italic are added. FIG. 19A is composed of three run elements “ab”, a space, and “c”, whereas in FIG. 19B, “ab c” is one run element. Since the space is an invisible character code, it does not affect the display of the electronic document.

  As described above, by applying this embodiment, it is possible to omit the description of characters indicating white space such as a space even when the font shape and decoration information are recognized, and to reduce the amount of description.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  In the present invention, a part or all of each processing unit in FIG. 2 may be realized using hardware such as an electronic circuit.

Claims (6)

An image processing apparatus that generates electronic data that can be edited from an input image,
Input means for inputting an image including a character string as the input image;
Extraction means for extracting a plurality of pixel clusters whose pixel values approximate in the pixels constituting the input image;
Identification means for identifying an area formed by the plurality of pixel blocks as at least one of a character area and other areas;
Analyzing a character from the pixel block identified as the character region, and obtaining character information including at least a character code and position information of the character;
A specifying means for specifying a character string including a blank character from an arrangement of characters indicated by the character information;
Color information adding means for acquiring color information from a pixel block in a character area at a position indicated by the character information and adding the color information to the character information;
Generating means for generating a description defining the electronic data from the specified character string and character information of characters included in the character string;
The image processing apparatus, wherein the color information adding unit adds color information of characters before and after the blank character in the character string to character information of the blank character included in the character string. The analysis means further obtains at least one information of character size, font shape, bold, italic as modification information for the character of the pixel block identified as the character region,
The color information adding unit further adds modification information of characters before and after the blank character in the character string to character information of the blank character included in the character string. Image processing apparatus.   When there are two or more pixel blocks at a position indicated by one character information, the color information adding means has a maximum area overlapping the character range indicated by the character information in the pixel block. The image processing apparatus according to claim 1, wherein the color information is acquired from a pixel block.   The image processing apparatus according to claim 1, wherein the position information included in the character information is indicated by coordinates of a rectangular area including the character. An image processing method for generating electronic data that can be edited from an input image,
An input step in which an input means inputs an image including a character string as the input image;
An extracting step in which the extracting means extracts a plurality of pixel blocks whose pixel values approximate in the pixels constituting the input image;
An identification step for identifying an area formed by the plurality of pixel blocks as at least one of a character area and other areas;
An analyzing step of analyzing a character from the pixel block identified as the character region and obtaining character information including at least a character code and position information of the character;
A specifying step of specifying a character string including a blank character from an arrangement of characters indicated by the character information;
A color information adding step for acquiring color information from a pixel block in a character area at a position indicated by the character information and adding the color information to the character information;
A generation step of generating a description defining the electronic data from the identified character string and character information of characters included in the character string;
In the color information adding step, color information on characters before and after the blank character in the character string is added to the character information of the blank character included in the character string. Computer
Input means for inputting an image including a character string as an input image;
Extraction means for extracting a plurality of pixel clusters whose pixel values approximate in the pixels constituting the input image;
An identification means for identifying an area formed by the plurality of pixel blocks as at least one of a character area and other areas;
Analyzing means for analyzing a character from the pixel block identified as the character region and obtaining character information including at least a character code and position information of the character;
A specifying means for specifying a character string including a blank character from an arrangement of characters indicated by the character information;
Color information adding means for acquiring color information from a pixel block in a character area at a position indicated by the character information and adding the color information to the character information;
Function as generation means for generating a description defining electronic data from the specified character string and character information of characters included in the character string;
The color information adding means adds color information of characters before and after the blank character in the character string to character information of the blank character included in the character string.

Images