JP2005260452A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for enhancing the reproducibility of colors in original image data and preventing deterioration in a compression efficiency and image quality. <P>SOLUTION: The image processing apparatus demarcates a basic region including pixel lumps discriminated to be an individual character from at least part of image data being a processing object, limits colors of pixel values of each pixel configuring the pixel lump included in the demarcated basic region to colors of any of representative pixel value objects included in a set of representative pixel value objects, applies number of color discrimination processing to discriminate the number of colors of the pixel lump on the basis of the frequency of incidence of the representative pixel value objects in the pixel lump after limitation of the colors, and executes prescribed image processing on the basis of a result of the discrimination of the number of colors. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that separates a character part and a picture part from image data and performs predetermined processing.

  The raster image data may include many image elements having different properties such as a character (text) portion and a natural image portion (pattern portion). Due to the difference in properties of these image elements, it is often preferable to perform different image processing for each image element, for example, compression by a different method is suitable for compression processing.

  Therefore, conventionally, image processing called so-called T / I separation has been researched and developed. Conventionally, as a method of T / I separation, for example, a processing target image is binarized, a continuous region of black pixels is defined, and when the size of the defined region is lower than a predetermined threshold value, the region is included in the region. There is a method of determining that an included black pixel represents a character (Patent Document 1).

In addition, for the area thus determined as the character part, the representative colors of the pixels are determined based on the pixel values included in the area, and the values of the pixels constituting each character are set to the representative colors. Some have further increased the compression rate by setting the value (Patent Document 2).
JP 2003-8909 A (see paragraph 0026) JP 2002-165105 A JP 2002-175532 A

  However, when the original raster image data is JPEG-compressed or read by a scanner or the like, the pixel values of the characters that should have been originally monochromatic vary and are identified as multicolor pixels. There can be. Further, if all characters are handled as a single color, colored characters cannot be reproduced. For example, when a plurality of colors are used in one character such as a gradation character, the gradation is lost. There is also a point.

  On the other hand, if the values of the pixels constituting the character are left as they are, the compression efficiency is lowered and the image quality is deteriorated.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus that can improve color reproducibility in original image data and prevent deterioration of compression efficiency and image quality. .

  The present invention for solving the problems of the conventional example described above is an image processing apparatus, and defines a basic region including a pixel block that is determined as an individual character from at least a part of image data to be processed. And, for each pixel constituting the pixel block included in the defined basic region, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set, and the limited color Color number determination means for performing color number determination processing for determining the number of colors of the pixel block based on the appearance frequency of the representative pixel value candidates in the pixel block after conversion, and based on the determination result of the color number Thus, predetermined image processing is executed.

  Here, when the number-of-colors determination unit determines that the number of colors of the pixel block is plural, the plurality of representative pixel value candidates are identified in order of appearance frequency, and a new representative is generated based on the identified representative pixel value candidates. Pixel value candidates may be generated and included in the representative pixel value candidate set, and the color number determination process may be performed again. In this case, the operation of generating a new representative pixel value candidate and performing the color number determination process is repeated, and the result of the color number determination process after a predetermined number of repetitions (or when a predetermined condition is satisfied) is output. The

  Alternatively, when it is determined that the number of colors of the pixel block is plural, a plurality of representative pixel value candidates are identified in order of appearance frequency, and new representative pixel value candidates are generated based on the identified representative pixel value candidates. The color number determination process may be performed again with the plurality of specified representative pixel value candidates and the new representative pixel value candidate as new representative pixel value candidates.

  Further, information regarding the continuity of the pixel values is generated for the pixel block for which the number of colors is determined by the color number determination unit, and the pixel block is determined to be a gradation based on the information regarding the continuity of the pixel value. It is good also as a means to determine whether it has become, and may change the content of the image processing with respect to the said pixel lump based on the result of the said determination.

  The number-of-colors determination unit generates information regarding the width of the pixel block, and when the information regarding the width of the pixel block is less than a predetermined width threshold, the pixel block is expressed by one color. It may be determined.

  In addition, the present invention for solving the problems of the above-described conventional example is an image processing method using a computer, and includes pixel blocks that are determined as individual characters from at least a part of image data to be processed. A basic area is defined, and for each pixel constituting the pixel block included in the defined basic area, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set. The color number determination process for determining the number of colors of the pixel block is performed based on the appearance frequency of the representative pixel value candidate in the pixel block after the limited color, and a predetermined image is determined based on the determination result of the color number It is characterized by executing processing.

  Furthermore, an image processing program according to another aspect of the present invention provides a computer with a procedure for defining a basic region including a pixel block determined as an individual character from at least a part of image data to be processed; For each pixel constituting the pixel block included in the basic region, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set, and the pixel block after the limited color is formed A procedure for performing a color number determination process for determining the number of colors of the pixel block based on the appearance frequency of the representative pixel value candidate in the method, and a procedure for performing a predetermined image process based on the determination result of the color number. It is characterized by being executed.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an image input unit 13, and an image output unit 14. The control unit 11 operates in accordance with a program stored in the storage unit 12 and performs each image processing described later. The contents of this image processing will be described in detail later.

  The storage unit 12 holds a program executed by the control unit 11. The storage unit 12 also operates as a work memory that stores various data generated during the process of the control unit 11. Specifically, the storage unit 12 can be implemented as a computer-readable recording medium and a device that writes data to or reads data from the recording medium (for example, a hard disk device or a memory device).

  The image input unit 13 is, for example, a scanner, and outputs image data obtained by optically reading a document to the control unit 11. Here, it is assumed that the value of each pixel is expressed in an RGB (red, green, blue) color space in the image data output from the image input unit 13. The image output unit 14 outputs image data in accordance with an instruction input from the control unit 11. For example, the image output unit 14 outputs the image data to an image forming unit (printer or the like) or transmits it to an external device via a network. Is what you do.

  Next, the content of the process of the control part 11 is demonstrated. As functionally shown in FIG. 2, the control unit 11 of the present embodiment sets the image data input from the image input unit 13 as a processing target, and performs predetermined preprocessing on the image data that is the processing target. A pre-processing unit 21 that performs the pattern processing, a pattern candidate portion specifying processing unit 22 that specifies a portion that is a candidate for a pattern portion (design candidate portion), a character / line image extraction processing unit 23 that extracts a character / line image, and a layout process. The layout processing unit 24, the same color region separation unit 25, a hole filling processing unit 26, and a compression processing unit 27 are included.

  Hereinafter, each of these parts will be described in detail.

[1. Pre-processing section]
In the preprocessing unit 21, the value of each pixel of the image data (processing target image data) input from the image input unit 13 is converted from RGB to YCbCr (a value composed of luminance and color difference). Specifically, the conversion can be performed using the following equation (1). Here, the value of each component of RGB is assumed to be a value from 0x00 (“0x” indicates a hexadecimal number) to 0xFF. Further, the pre-processing unit 21 may correct the gradation of each pixel value based on the luminance and saturation of the background area. However, the gradation correction process is not always necessary.

[2. Design candidate area identification processing unit]
The pattern candidate portion identification processing unit 22 performs a process of identifying an area estimated as a pattern area from the image data expressed in the YCbCr color space output from the preprocessing unit 21 as a pattern candidate area. Specifically, since there are various processing methods of the pattern candidate portion identification processing unit 22, examples of two typical methods will be described below.

  As shown in FIG. 3A, the pattern candidate portion specifying processing unit 22 includes a binarization processing unit 31, a connected pixel extraction unit 32, a feature amount calculation unit 33, an attribute determination unit 34, and a non-pattern region. The processing unit 35, the background region painting unit 36, and the pattern candidate region creation unit 37 are configured.

  The binarization processing unit 31 copies and stores image data expressed in the YCbCr color space output from the preprocessing unit 21 in the storage unit 12. Then, referring to the value of each pixel of the copied image data (design area specifying image data), the luminance component of the pixel value is compared with a predetermined binarization threshold value, and the luminance component is Pixels above the binarization threshold are black pixels (value “1”), pixels whose luminance components are less than the binarization threshold are white pixels (value “0”), and the pattern area specific image data is binarized. Convert to image data. Here, the larger the luminance component, the darker (black).

  The connected pixel extracting unit 32 extracts a region (connected region) where black pixels are connected from the binary image data. For the extraction of the connected region, a process widely known as a labeling process can be used. When the labeling process is used, each connected area is given a different label identifier and stored in the storage unit 12.

  The feature amount calculation unit 33 calculates a predetermined scale feature amount for each of the connected regions extracted by the connected pixel extraction unit 32. Here, the scale feature amount includes the rectangular area determined in relation to the connected region, the density of black pixels in the rectangle (the number of black pixels in the rectangle divided by the number of all pixels in the rectangle) ) Etc. For example, the feature amount calculation unit 33 performs processing as follows. That is, the feature amount calculation unit 33 defines a rectangle circumscribing each connected region (a circumscribed rectangle) as a related region for each connected region. Specifically, for the circumscribed rectangle, among the pixels included in the connected area, the X coordinate of the pixel with the smallest X coordinate (the leftmost pixel in the image area specific image data) is defined as Xmin, and the Y coordinate is also the smallest. X coordinate and Y coordinate of the one with the largest X coordinate and the one with the largest Y coordinate are set to Ymin. , Xmax and Ymax are defined as rectangles having diagonal lines from (Xmin, Ymin) to (Xmax, Ymax). Then, the area A of the circumscribed rectangle is calculated as (Xmax−Xmin) × (Ymax−Ymin). The area A is equal to the number of all pixels included in the circumscribed rectangle. On the other hand, the feature amount calculation unit 33 counts the number of black pixels in the circumscribed rectangle and divides the number by the number of all the pixels (that is, the value of the area A) to obtain the density D.

  In this way, the feature amount calculation unit 33 uses, for each connected region, the area A (corresponding to the area feature amount) of the circumscribed rectangle and the black pixel density D (corresponding to the density feature amount) as the scale feature of the connected region. Calculate as a quantity. Then, the identifier (for example, label identifier) of each connected area and the scale feature amount are associated with each other and stored in the storage unit 12 as a scale feature amount database.

  The attribute determination unit 34 determines whether each connected region is a pattern candidate region while referring to the scale feature amount database stored in the storage unit 12. As a specific example, this attribute determination unit 34 performs processing based on the premise that there is a certain size or more if it is a character. That is, an area (small area) less than an area (so-called minimum area) corresponding to the minimum font size (for example, 6 points) used in a general document can be determined as a picture area that does not include characters. By removing these small areas, the character portion can be specified.

  Furthermore, in the present embodiment, for example, when the area is equal to or larger than the maximum font size (for example, 24 points) used in a general document (ie, the maximum area), it is basically determined that no character is included. it can. However, such a region may be a diagram (graph etc.) as well as a picture. Therefore, the black pixel density is also used here, and it is determined that the image has a pattern only when the black pixel density is relatively large among rectangles having an area larger than the maximum area. This is because a graph such as a graph is composed of lines, so that the density of black pixels is estimated to be relatively low.

  Specifically, the attribute determination unit 34 of the present embodiment includes a first area threshold value α1 representing the minimum area, a second area threshold value α2 representing the maximum area (where α1 <α2), First and second density threshold values (ρ1, ρ2) are determined in advance, and the determination is performed as follows using these threshold values.

  That is, as shown in FIG. 4, the attribute determination unit 34 selects one of the connected areas not yet selected in this process as a target area from the connected areas included in the scale feature amount database (S1). Then, the scale feature quantity associated with the region of interest is read, and whether or not the area A included in the scale feature quantity is less than the first area threshold value α1 (A <α1 or not). Check (S2). Here, if A <α1, the region of interest is determined as a picture region, and the identifier of the region of interest (for example, a label identifier) is associated with the determination result (information indicating that the region is a pattern region). (S3), it is checked whether or not there is a connected area not yet selected in this process in the scale feature amount database (S4). If there is an unselected connected area (if Yes), the process is performed. The process returns to S1 and continues. Further, if there is no unselected connected region in the process S4 (if No), that is, if the determination is completed for all the connected regions, the process is ended.

  If it is determined in step S2 that A <α1 is not satisfied, whether or not the area A of the attention area exceeds the second area threshold value α2 (whether A> α2) is checked (S5). If A> α2, it is checked whether or not the black pixel density D, which is one of the scale feature quantities of the region of interest, is less than the first density threshold ρ1 (whether D <ρ1) ( S6) If D <ρ1, it is determined that the attention area is a non-picture area, and an identifier of the attention area (for example, a label identifier) is associated with the determination result (information indicating that it is a non-picture area). Are stored in the storage unit 12 (S7), and the process proceeds to S4 (Y). Furthermore, if D <ρ1 is not satisfied in process S6, the process proceeds to process S3 (X) (that is, it is determined as a picture area) and the process continues.

  If A> α2 is not satisfied in process S5, the process proceeds to process S7 (Z) (that is, determined as a non-picture area) and the process is continued. The density threshold ρ1 is set to a value suitable for extracting line drawings and the like by experiment.

  Further, at least one of the area feature amount or the area threshold value may be corrected based on the resolution of the image data to be processed. For example, when the area threshold values α1 and α2 are values set at the resolution r0, when the resolution of the image data to be processed is r, the first area threshold value α1 is set to (r / r0). ) × (r / r0) × α1, and the second area threshold value α2 is corrected to (r / r0) × (r / r0) × α2.

  The non-design area processing unit 35 refers to the determination result for each connected area that is generated by the attribute determination unit 34 and stored in the storage unit 12, and is determined to be a non-design area in the design area specifying image data. The black pixels included in the connected area are converted into white pixels, and the image area specifying image data in which the non-picture area and the background are white pixels is generated.

  The background area filling unit 36 sets white pixels connected to the boundary of the pattern portion to predetermined values (white, black, etc.) for the pattern area specifying image data in which the non-pattern area and the background are white pixels. Fill with any color other than Here, a process of painting a closed region (inside or outside surrounded by a closed curve) as is widely known is used. The pattern candidate area creation unit 37 sets the color of the pixels that have not been painted with the predetermined value by the background area painting unit 36 to black. A portion set as a black pixel by the pattern candidate area creation unit 37 is handled as a pattern candidate area.

  The operation of the pattern candidate portion specifying processing unit 22 by this small area removal method will be described. Hereinafter, a case where the image data illustrated in FIG. 5A is a processing target will be described as an example. For convenience of illustration, FIG. 5A shows the whole as a diagram, but FIG. 5A shows a photograph portion P, a text portion T, a diagram portion G, and a map. And a plate part M such as Further, in the map M, there are cases where characters or the like partially overlapping with the line segments constituting the map are described (X). For example, letters indicating road names are often shown overlapping other roads.

  The pattern candidate portion specifying processing unit 22 binarizes this (FIG. 5B), extracts connected pixels, and performs a small area removal process based on the feature amount for each connected pixel. Then, the pattern area specifying image data is in a state where the text portion (T) is removed as shown in FIG. In FIG. 5, since the black pixel density of the diagram portion G is relatively high, the diagram portion G remains without being removed. In addition, the characters that overlap the line segment in the map portion M are extracted as connected pixels together with the overlapping line segment. For this reason, the area of the rectangle surrounding the connected pixel is increased, and is not removed as a small region, but remains (X in FIG. 5C).

  The pattern candidate portion specifying processing unit 22 fills the white pixel portion B, which is the background, with a predetermined color (FIG. 5D), and further black pixels other than the portion painted with the predetermined color. And As a method of painting the background part here, use the general paint processing that paints the area that is not partitioned by line segments from the four corners of the image data (upper left, upper right, lower left, lower right corners). Can be taken. Then, the pattern candidate portion specifying processing unit 22 sets the predetermined color portion as a white pixel and sets the others as black pixels. Then, as shown in FIG. 5 (e), pattern area specifying image data is obtained in which the pattern candidate part is a black pixel and the other part is a white pixel.

  In addition, the pattern candidate portion specifying processing unit 22 converts the black pixel included in the connected region determined as the non-pattern region in the pattern region specifying image data by the non-pattern region processing unit 35 to the white pixel. Expansion and contraction processing may be performed on the specific image data. That is, as shown in FIG. 3B, the processing by the pattern candidate portion specifying processing unit 22 is functionally a binarization processing unit 31, a connected pixel extraction unit 32, a feature amount calculation unit 33, and attribute determination. A part 34, a non-picture area processing part 35, an expansion / contraction part 38, a background area painting part 36, and a pattern candidate area creation part 37 may be included.

  Here, the expansion / contraction unit 38 has the pattern area specifying image data obtained by converting the black pixels included in the connected area determined to be the non-pattern area among the pattern area specifying image data by the non-pattern area processing unit 35 into white pixels. Are sequentially selected as the target pixel.

  Then, the expansion / contraction unit 38 performs processing (expansion) when the pixel of interest is a black pixel if there is at least one black pixel in the vicinity of the pixel of interest (four neighborhoods of upper, lower, left, and right pixels or eight neighborhoods of the surrounding eight pixels). Process) to expand the black pixel portion and select the next pixel of interest. The selection order may be the order in which lines are scanned from top to bottom and the lines are scanned from left to right (so-called raster scan order).

  When the expansion / contraction unit 38 completes the expansion processing for all the pixels, the expansion / contraction unit 38 again starts the contraction processing while sequentially selecting each pixel as the target pixel. In the contraction process, if there is at least one white pixel in the vicinity of the pixel of interest (near 4 or 8), the pixel of interest is set as a white pixel. Then, when the contraction process is completed for all the pixels, the process of the expansion / contraction unit 38 ends.

  In this case, the background area filling unit 36 sets the white pixels in the background part connected to the boundary of the pattern part to a predetermined value (with respect to the pattern area specifying image data processed by the expansion / contraction part 38). Any color other than white or black) will be painted.

  By performing the expansion / contraction process in this manner, the halftone dot region is easily distinguished from the background region and recognized as a pattern region.

  In the present embodiment, even if the character part is erroneously determined as a pattern in the process of specifying the pattern area candidate, the character part is further extracted by the layout process performed later, so that the extraction accuracy of the pattern candidate area is relatively high. It can be low.

  In the description so far, the background region filling unit 36 fills the background portion with a predetermined color, but the following may be used instead. That is, when a labeling process is used to extract a connected region, a label (background identification) that is larger than the maximum label value obtained at the time of extraction (for example, a value obtained by adding 1 to the maximum label value). (Label) may be set by replacing the label of the pixel in the background portion. In this case, the pattern candidate area creation unit 37 may generate the pattern area specific image data shown in FIG. 5E by setting the pixels not attached with the background specific label to black pixels.

  The control unit 11 stores coordinate information (hereinafter referred to as “design candidate area definition information”) that defines each of the identified design candidate areas in the storage unit 12.

[3. Character line drawing extraction processing unit]
The character / line drawing extraction processing unit 23 performs processing for extracting a character / line drawing part from the image data expressed in the YCbCr color space output from the preprocessing unit 21. For this processing, for example, processing as disclosed in Patent Document 3 can be used.

  The character / line drawing extraction unit processing unit 23 stores the coordinate information of individual characters and rectangles (character / line drawing circumscribed rectangles) surrounding the line drawing parts in the storage unit 12.

[4. Layout processing section]
The layout processing unit 24 reads, from the storage unit 12, the pattern candidate area defining information generated by the pattern candidate portion specifying processing unit 22 and the coordinate information of the character / line drawing circumscribing rectangle generated by the character / line drawing extraction processing unit 23.

  The layout processing unit 24 performs layout using different determination conditions for each of the pattern candidate area defined by the pattern candidate area definition information and the character / line drawing area defined by the coordinate information of the circumscribed rectangle of the character / line drawing. Analyze.

  Specifically, in the present embodiment, layout analysis processing is performed in the pattern candidate area defined by the pattern candidate area definition information read from the storage unit 12, and further character portions are extracted from the pattern candidate area. Then, a portion excluding the extracted character portion is defined as a pattern region, and is used for subsequent processing.

  Here, processing contents of the layout analysis processing will be described. As functionally shown in FIG. 6, the layout processing unit 24 includes a binarization processing unit 41, a connected pixel extraction unit 42, a basic rectangle defining unit 43, a first separator detecting unit 44, and a row rectangle defining unit 45. A second separator detection unit 46, a character region demarcation unit 47, a noise determination unit 48, and a character part specifying unit 49.

  The binarization processing unit 41 includes partial image data in an area defined by the pattern candidate area definition information among the image data (original image data) expressed in the YCbCr color space output from the preprocessing unit 21. Using (picture candidate part data) as a processing target, the pattern candidate part data that is the processing target is binarized to generate binarized picture candidate part data.

  The connected pixel extraction unit 42 performs a labeling process on the binarized pattern candidate partial data, and includes a plurality of continuous pixels having pixel values that satisfy a predetermined condition (for example, a condition such as a black pixel). A pixel group (connected pixel group) is specified.

  The basic rectangle defining unit 43 defines a rectangle (for example, a rectangle circumscribing the connected pixel group) related to the connected pixel group specified by the connected pixel extracting unit 42 as a basic rectangle, and the basic rectangle coordinate information about each connected pixel group Coordinate information for defining a rectangle) is generated. Then, a unique identifier is issued for each basic rectangle, and the identifier and the coordinate information of the basic rectangle are associated and stored in the storage unit 12 as a basic rectangle database.

  The first separator detection unit 44 scans each pixel one line from the left to the right with the pixel at the upper left corner of the pattern candidate partial data to be processed as the initial position, and similarly for the next lower line ( That is, when the pixels having pixel values that do not satisfy the predetermined condition in the labeling process (for example, white pixels) are continuously larger than a predetermined horizontal threshold value, the continuous pixel portion is scanned. Is detected as the first separator (in the horizontal direction), and the information specifying the first separator (the coordinates of the left end pixel and the right end pixel of the continuous pixel portion, such as the coordinates are the coordinates on the original image data) Or the local coordinates on the pattern candidate portion data may be generated and stored in the storage unit 12.

  The first separator detection unit 44 scans each pixel one line from the top to the bottom with the pixel at the upper left corner of the candidate pattern data to be processed as the initial position. Similarly, when pixels having pixel values that do not satisfy the predetermined condition in the labeling process (for example, white pixels) continue more than a predetermined vertical threshold value, the continuous pixel portion is (vertical) Information that is detected as the first separator (in the direction) and identifies the first separator (such as the coordinates of the upper end pixel and the lower end pixel of the continuous pixel portion, where the coordinates are coordinates on the original image data) It may be local coordinates on the pattern candidate portion data) and is stored in the storage unit 12.

  In these processes, the horizontal direction threshold and the vertical direction threshold can be arbitrarily determined by the user. The horizontal threshold is a threshold for dividing each stage in a multi-column layout, and the vertical threshold is a threshold for dividing each line from a document including two or more lines of character strings. In addition to the user setting, the horizontal threshold is a predetermined function value based on the basic rectangle width statistical value (average), and the vertical threshold is based on the basic rectangular height statistical value (average). Each may be determined as a predetermined function value.

  Specifically, the first separator is detected in a state as shown in FIG. In FIG. 7A, as a result of detecting the first separators adjacent to each other, it is shown as one first separator region.

  The row rectangle defining unit 45 selects one of the basic rectangles stored in the storage unit 12 as the target basic rectangle. Then, the following processing is performed while sequentially selecting the basic rectangles that have been stored in the storage unit 12 and have not been selected as the target basic rectangle so far as the processing target basic rectangles.

  That is, a vector is calculated from the center coordinates of the target basic rectangle (in the case where the coordinate information represents the coordinates of each vertex at the diagonal position, the center point coordinates thereof) to the center coordinates of the processing target basic rectangle. Further, the row rectangle defining unit 45 calculates the distance between the target basic rectangle and the processing target basic rectangle from the size of the vector (the square root of the sum of squares of each component). When the calculated distance is equal to or less than a predetermined distance threshold, it is checked whether the calculated vector intersects any of the detected first separators. For this process, a widely known process for checking whether or not two line segments intersect can be used. If the calculated vector does not intersect any of the detected first separators, the identifier of the target basic rectangle is associated with the identifier of the target basic rectangle and stored in the storage unit 12 as a basic rectangle relation database. .

  The row rectangle defining unit 45 performs the above processing while sequentially selecting the basic rectangles stored in the storage unit 12 as the target basic rectangle. Then, a series of basic rectangle groups (which may be plural) are linked and identified with reference to the basic rectangle relation database obtained as a result of this processing, and the basic rectangles included in the specified basic rectangle group A rectangle circumscribing is defined as a row rectangle (for example, FIG. 7B).

  For example, in the basic rectangle relation database, a basic rectangle with an identifier “1” is associated with a basic rectangle with an identifier “2”, and a basic rectangle with an identifier “2” and a basic rectangle with an identifier “3” are associated with each other. If they are associated, the row rectangle defining unit 45 integrates these results and specifies the basic rectangles having the identifiers “1”, “2”, and “3” as a series of basic rectangle groups. Then, from the basic rectangles included in the basic rectangle group, the maximum value and minimum value of the x (horizontal direction) value of the coordinate information are extracted, and similarly the maximum value and minimum value of the y (vertical direction) value are extracted. And extract. Then, a first coordinate that is a set of the extracted minimum value of x and a minimum value of y, and a second coordinate that is a set of the maximum value of x and the maximum value of y are respectively set as upper left coordinates and lower right coordinates. A line rectangle is defined as coordinates. That is, the row rectangle is defined by coordinate information including these two coordinate values.

  The row rectangle defining unit 45 issues a unique identifier for each row rectangle thus defined, and information for identifying each identifier, the coordinate information of the row rectangle, and the basic rectangle group included in the row rectangle (the identifier of each basic rectangle) Are stored in the storage unit 12 as a row rectangle database.

  The second separator detection unit 46 scans each pixel one line from the left to the right, using the pixel at the upper left corner of the candidate pattern data to be processed as the initial position, and similarly for the line below ( That is, when scanning is performed in the raster scan order), and pixels having pixel values satisfying a predetermined condition in the labeling process in the connected pixel extraction unit 42 (for example, black pixels) are continuously larger than a predetermined horizontal threshold value. In addition, the continuous pixel portion is detected as a second separator (in the horizontal direction), and information specifying the second separator (the coordinates of the left end pixel and the right end pixel of the continuous pixel portion, etc., where the coordinates are the original The coordinates may be coordinates on the image data, or may be local coordinates on the pattern candidate portion data) and stored in the storage unit 12.

  The second separator detection unit 46 scans each pixel one line from the top to the bottom with the pixel at the upper left corner of the pattern candidate partial data to be processed as the initial position, and for the right one line. When scanning is performed in the same manner and pixels having pixel values satisfying a predetermined condition in the labeling process in the connected pixel extraction unit 42 (for example, black pixels) continue more than a predetermined vertical direction threshold, Information that detects a continuous pixel portion as a second separator (in the vertical direction) and identifies the second separator (such as the coordinates of the upper end pixel and the lower end pixel of the continuous pixel portion, where the coordinates are on the original image data) And may be local coordinates on the pattern candidate portion data) and stored in the storage unit 12.

  In these processes, the horizontal direction threshold and the vertical direction threshold can be arbitrarily determined by the user. The horizontal threshold is a threshold for dividing each stage in a multi-column layout, and the vertical threshold is a threshold for separating each line from a document including two or more character strings. In addition to the user setting, the horizontal threshold is a predetermined function value based on the basic rectangle width statistical value (average), and the vertical threshold is based on the basic rectangular height statistical value (average). Each may be determined as a predetermined function value.

  The character area defining unit 47 selects one of the line rectangles defined by the line rectangle defining unit 45 as the target line rectangle. The character area demarcating unit 47 then performs the next processing while sequentially selecting the row rectangles stored in the storage unit 12 and not selected as the target row rectangle so far as the processing target row rectangle. Do.

  In other words, a line segment connecting the coordinates of each vertex of the target line rectangle and the coordinates of the corresponding vertex of the processing target line rectangle, and a polygonal area defined by each side of the target line rectangle and the processing target line rectangle are generated. Then, it is examined whether or not this polygonal region and the second separator (region) intersect (regions overlap at least partially). For this process, a widely known process for checking whether or not two regions intersect can be used.

  Here, when the polygonal region and the second separator do not intersect, the identifier of the target row rectangle and the identifier of the processing target row rectangle are associated with each other and stored in the storage unit 12 as a row rectangle relation database.

  The character area defining unit 47 performs the above processing while sequentially selecting the row rectangles stored in the storage unit 12 as the target row rectangle. Then, a series of row rectangle groups (which may be plural) related to each other in a chained manner are identified with reference to the row rectangle relation database obtained as a result of this processing, and the row rectangles included in the identified row rectangle group A rectangle circumscribing is defined as a character area.

  For example, in the row rectangle relation database, the row rectangle having the identifier “1” is associated with the row rectangle having the identifier “2”, and the row rectangle having the identifier “2” is associated with the row rectangle having the identifier “3”. If they are associated, the row rectangle defining unit 45 integrates these results and identifies the row rectangles having the identifiers “1”, “2”, and “3” as a series of row rectangle groups. Then, from the row rectangles included in the row rectangle group, the maximum value and the minimum value of the x (horizontal direction) value of the coordinate information are extracted, and similarly the maximum value and the minimum value of the y (vertical direction) value are extracted. And extract. Then, a first coordinate that is a set of the extracted minimum value of x and a minimum value of y, and a second coordinate that is a set of the maximum value of x and the maximum value of y are respectively set as upper left coordinates and lower right coordinates. Define the rectangle of the character area as coordinates. That is, the character area is defined by coordinate information including these two coordinate values.

  The character area defining unit 47 issues unique identifiers for the character areas thus defined, and information for identifying each identifier, the coordinate information of the character area, and the group of line rectangles included in the character area (the identifier of each line rectangle identifier). Are stored in the storage unit 12 as a character area database.

  The noise determination unit 48 performs noise determination processing for confirming whether or not a character is included in each of the character regions defined by the character region definition unit 47. Here, the noise determination processing is related to the first noise determination processing for determining whether or not each row rectangle includes a character based on the number of row rectangles or information indicating the property of each row rectangle, and the row rectangle. Second noise processing for determining whether or not a character is included in the row rectangle based on information on the basic rectangle.

  Specifically, the processing of the noise determination unit 48 is performed as shown in FIG. First, the noise determination unit 48 selects a character region stored in the storage unit 12 that has not yet been selected as a target character region as a target character region (S11). Then, the number of line rectangles included in the target character area is checked to determine whether it is 2 or more (that is, whether the character area is composed of a plurality of lines) (S12). Here, if the number of row rectangles is 2 or more (if Yes), the statistics for testing variation such as the width and height of each row rectangle included in the target character area, and their average and standard deviation Is calculated (S13). Then, based on these statistics, it is compared whether or not the variation in the width and height of each row rectangle is larger than a predetermined threshold (S14). This comparison can be, for example, a comparison as to whether or not the standard deviation exceeds a predetermined threshold value. If it is determined by this processing S14 that the variation is large (in the case of Yes), it is determined that no character is included in the target character region, and the target character region is deleted from the character region database in the storage unit 12. (S15), the process proceeds to S18. These processes from S12 to S15 correspond to a first noise determination process. That is, here, the width and height of each row rectangle and their statistics are used as information representing the properties of the row rectangle.

  On the other hand, if it is determined in the process S14 that the variation is small (in the case of No), an in-line determination process (second noise determination process) is performed on each line rectangle included in the target character area (S16). The specific contents of this process S16 will be described later. Then, based on the ratio of the number of lines determined as noise (no character is included) in this process S16 and the number of line rectangles included in the target character area, the number of lines determined as noise is It is determined whether or not the predetermined ratio is greater than or equal to the number of line rectangles included in the character area (S17). When the ratio is equal to or greater than the predetermined ratio, it is determined that no character is included in the target character area. Then, the process proceeds to process S15.

  In step S17, if the ratio is less than the predetermined ratio, it is checked whether or not there is a character area that is not yet a focused character area in the character area database of the storage unit 12 (S18), and there is an unselected character area. If so, the process returns to step S11 to continue the process. Furthermore, if there is no unselected character area in process S18 (if all the character areas have been processed), the noise determination process is terminated.

  Further, if the number of row rectangles is one in process S12 (if No), the process proceeds to process S16 to continue the process. In this case, it is determined whether or not a character is included in the single line rectangle, and if a character is included (in this case, the ratio of processing S17 is “0”), the target character region includes If it is determined that a character is included and no character is included in the single line rectangle (in this case, the ratio of processing S17 is “1”), the character region of interest does not include the character. It is judged.

  Here, the content of the specific process (2nd noise determination process) in process S16 is demonstrated. In this processing, as shown in FIG. 9, the noise determination unit 48 selects one of the row rectangles to be processed as a target row rectangle (S21), and selects the row rectangle database stored in the storage unit 12. With reference to this, the number of basic rectangles included in the target row rectangle is counted (S22). As a result of the counting, the process branches into a case where the number of basic rectangles is “1”, a case where it is “2”, and a case where it is “3” or more (S23), and the number of basic rectangles is “1”. Is obtained, a list of identifiers of the basic rectangles included in the target line rectangle is acquired, the coordinate information of the basic rectangles included in this list is read from the basic rectangle database of the storage unit 12, and the basic rectangles included in this list are read out. The width and height and the product (that is, the area) are calculated. Then, it is determined whether or not the area is equal to or smaller than a predetermined area threshold (S25). If the area is equal to or smaller than the area threshold, it is determined that no character is included in the target line rectangle, The result is stored in the storage unit 12 (S26). Then, it is checked whether or not there is a row rectangle that has not been selected as the target row rectangle (S27). If there is an unselected row rectangle, one of the unselected row rectangles is selected as the target row rectangle. The process returns to S21 and continues. On the other hand, if there is no unselected row rectangle in the process S27, the process is terminated and the process returns to the process of FIG.

  Further, when the count value in the process S22 is “2”, the area of each basic rectangle is calculated, and the distance between these basic rectangles is calculated. The distance between the basic rectangles can be calculated as the distance between the centers of the basic rectangles, for example. Then, it is determined whether the distance is larger than a predetermined distance threshold value or the ratio of the areas of the two basic rectangles is larger than a predetermined area ratio threshold value (S31), and the distance is set to a predetermined distance. If it is larger than the threshold value, or if the ratio of the areas of the two basic rectangles is larger than the predetermined area ratio threshold value, the process proceeds to step S26 and (X) processing is continued.

  Furthermore, when the count value in process S22 is “3” or more, it is determined whether or not the count value (number of basic rectangles) exceeds a predetermined maximum number (S32). Determines that no character is included in the target line rectangle, and the process proceeds to step S26 (X). This is in consideration of the fact that it is usually impossible to include more than 100 characters in a line, and may be set as a fixed value or adjusted based on the width of the line of interest rectangle. May be. If the number of basic rectangles does not exceed the predetermined maximum number in step S32, the area of each basic rectangle is further calculated to determine whether the calculated maximum area value exceeds the predetermined maximum area value. Judgment is made (S34). If it is determined that the predetermined maximum area value is exceeded, it is determined that no character is included in the target line rectangle, and the process proceeds to step S26 (X). This maximum area value may also be determined as a fixed value, or may be adjusted based on at least one of the width and height of the target row rectangle (for example, the smaller of them).

  Further, when it is determined in the process S34 that the predetermined maximum area value is not exceeded, the area ratio of the two basic rectangles related to each combination is further determined for two combinations of the basic rectangles (at least one combination arbitrarily extracted). It is determined whether it is larger than a predetermined area ratio threshold value (S35). If the ratio of the areas of the two basic rectangles is larger than the predetermined area ratio threshold value, the process proceeds to step S26 and the process continues ( X).

  When the ratio of the areas of the two basic rectangles is not larger than the predetermined area ratio threshold value in this process S35, it is determined that the character is included in the target line rectangle, and the determination result is stored in the storage unit 12. Then, the process proceeds to process S27.

  In the process S25, when the area exceeds the area threshold value, and in the process S31, the distance is equal to or smaller than the predetermined distance threshold value, and the ratio of the areas of the two basic rectangles is a predetermined area ratio. If it is less than or equal to the threshold value, the process proceeds to the process S32 (or process S34) and the process is continued.

  Further, in the processing S35, the processing is performed for each combination, but in order to reduce the processing load, for example, the average value (average area) of each basic rectangle, the minimum value (minimum area), the maximum value ( The maximum area) may be calculated, and the average area and the minimum area, the ratio of the average area and the maximum area, or the ratio of the minimum area and the maximum area may be compared with the area ratio threshold value.

  As described above, the noise determination unit 48 reconfirms whether or not each row rectangle truly includes a character based on the properties (area, area ratio, distance, etc.) of the basic rectangle included in each row rectangle. To do.

  The first noise determination process of the noise determination unit 48 is not limited to the example described here. For example, the width and height of each row rectangle is used here as the property of the row rectangle, but using the coordinate information of the row rectangle (statistics such as an average value and standard deviation) together with or instead of these, Also good. According to this, as shown in FIG. 10, when the positions of the line rectangles included in the character area vary, it is determined that the character area does not include a character (is a noise), The character area of interest is deleted from the character area database in the storage unit 12.

  The character part specifying unit 49 reads the character area database that has undergone the processing of the noise determination unit 48 from the storage unit 12, and the character area included in the character area database (the coordinate information of the character area itself) or the character area database A black pixel portion (character area coordinate information and bitmap information including a black pixel portion) is specified as a character portion, and information (character portion specifying information) for specifying the character portion is stored in the storage unit 12. The control unit 11 may delete the basic rectangle relationship database and the row rectangle relationship database stored in the storage unit 12 at this time.

  As described above, the layout processing unit 24 according to the present embodiment demarcates character areas step by step from a character to a line and from a line to an area, and includes a character string based on the state of the line in the delimited character area. If it is determined that the character string is not included, it is further determined whether the character is included based on the state in the line (character unit). However, the layout processing in the present embodiment is not limited to this, and other well-known layout processing may be used.

  One characteristic of this embodiment is that a character portion is extracted using layout analysis in so-called T / I separation processing. As a result, the character portion included in the pattern candidate region is also extracted by the layout analysis process, and the accuracy of extracting the character portion can be improved.

  On the other hand, the layout processing unit 24 performs a layout analysis process also on a portion (character / line drawing circumscribed rectangle) defined as a character / line drawing. The layout processing unit 24 determines layout frames (including at least rectangles (basic rectangles) circumscribing each character) obtained as a result of the layout analysis processing, and stores information (coordinate information and the like) of these layout frames. 12.

  Note that a line rectangle obtained by relating the basic rectangle in the horizontal or vertical direction may also be defined within the character line drawing circumscribed rectangle, and the coordinate information of the line rectangle may be stored in the storage unit 12 together.

  The layout processing unit 24 of the control unit 11 refers to the character part specifying information stored in the storage unit 12 and the character / line drawing circumscribing rectangle (or the layout frame as a layout processing result thereof), and the image to be processed For the entire data, coordinate information of a rectangular area including characters in the image data is generated. Specifically, the layout processing unit 24 combines the rectangle defined by the coordinate information of the character part specifying information and the rectangle defined by the character / line drawing circumscribing rectangle information (or the layout frame as a result of the layout processing). A character line drawing area is generated. In other words, in the present embodiment, the pattern candidate area and the character / line drawing area are defined separately, and therefore the area specified as the character / line drawing area may exist in the pattern candidate area. For this reason, these areas are combined here to make the overlapping area one character / line drawing area.

  Then, the layout processing unit 24 generates a unique area identifier (hereinafter referred to as label data) for each combined character / line drawing part, and coordinates information (vertex) for defining the label data and the corresponding character / line drawing area. The information is stored in the storage unit 12 as a character / line drawing area database.

[5. Same color area separation unit]
The same color area separation unit 25 performs a process of dividing each of the character / line drawing areas stored in the character / line drawing area database of the storage unit 12 into an area including only the character / line drawing parts of the same color. The same color region separation unit 25 reads the coordinate information of the basic rectangle (rectangle defined from both the pattern candidate region and the character / line drawing region) stored in the storage unit 12 during the layout process. Then, a candidate for a representative value (representative color) is determined among the pixel values included in each basic rectangle defined by the coordinate information in the image data (original image data) to be processed.

  As shown in FIG. 10, the same color region separation unit 25 in the present embodiment includes a color number determination unit 51, a continuity determination unit 52, a representative color determination unit 53, and a limited colorization unit 54. It is configured.

  The number-of-colors determination unit 51 refers to the coordinate information of the basic rectangle (corresponding to the basic region of the present invention) stored in the storage unit 12 and selects the target rectangle as the target basic rectangle in order, while paying attention to the original image data. The number of colors is counted based on the pixel value in the basic rectangle. Specifically, the color number determination unit 51 counts the number of types of pixel values, that is, the number of colors, of the pixels in the target basic rectangle in the original image data.

  The process of counting the number of colors includes a representative color candidate associating unit 61, a pixel number calculating unit 62, a determining unit 63, and a new representative color candidate determining unit 64, as shown in FIG. The representative color determining unit 65 is included.

  The representative color candidate associating unit 61 reads out a set of representative color candidates from the storage unit 12, and associates one of the representative color candidates with respect to each pixel in the basic rectangle to be processed (referred to as a basic rectangle of interest). I do. Here, a new representative color candidate is added and updated in the representative color candidate set by a process described later. Initially, the representative color candidate includes a plurality of predetermined representative color candidates. I do not care.

  That is, the representative color candidate associating unit 61 sequentially selects each pixel in the target basic rectangle as a target pixel one by one, and a color space between the pixel value of the target pixel and the pixel value of each representative color candidate Calculate the distance above. Among them, the representative color candidate with the shortest distance is stored in the storage unit 12 as a representative color candidate relation database in association with information (for example, coordinate information) for specifying the pixel of interest.

  The pixel number calculation unit 62 counts and outputs the number of appearances in the representative color candidate relationship database stored in the storage unit 12 for each representative color candidate. The determination unit 63 determines whether or not the number of appearances exceeds a predetermined threshold, and determines that the character in the basic rectangle is one color when the number exceeds the predetermined threshold. If the number of appearances does not exceed a predetermined threshold, it is determined that the characters in the basic rectangle are not one color (multiple colors), and the process is terminated.

  Here, the predetermined threshold value may be set to a predetermined value in advance, or based on the total number of pixels in the basic rectangle to be processed (or the number of pixels (significant pixels) constituting a pixel block such as a character). It may be determined. For example, 0.8 × N, which is 80% of the total number of pixels N, may be set as the predetermined threshold value.

  Further, when it is determined that the character in the basic rectangle is not one color, the determination unit 63 counts the number of determinations, and ends the process when the number of determinations reaches a predetermined maximum number.

  When it is determined that the character or line drawing in the basic rectangle is not one color, the new representative color candidate determination unit 64 refers to the representative color candidate relationship database and selects a plurality of representative color candidates in descending order of appearance. . For example, the top two representative color candidates with the number of appearances are selected. Then, the new representative color candidate determination unit 64 determines at least one new representative color based on the selected representative color candidate, includes the determined new representative color in the set of representative color candidates, and represents the representative color candidate association unit 61. Is instructed to repeat the process. Here, the new representative color is a statistical calculation result (specifically, an average pixel value or the like) of the selected representative color candidate.

  Alternatively, the new representative color candidate determination unit 64 may be configured as follows. That is, the top two representative color candidates with the highest number of appearances are selected, at least one new representative color is determined based on the selected representative color candidate, and the selected two representative color candidates and the new representative color candidate are newly represented. As a set of color candidates, the representative color candidate associating unit 61 is instructed to repeat the process.

  Furthermore, instead of the top two, for example, the representative color candidates are sequentially selected in descending order, the sum of the selected number of appearances is calculated, and the value obtained by dividing the sum by the total number of pixels N is a predetermined ratio. At this time, at least one new representative color may be determined based on the selected representative color candidate, and the determined new representative color may be included in the set of representative color candidates.

  When the determination unit 63 finishes the process, the representative color determination unit 65 determines a representative color with reference to the determination result. Specifically, when the determination unit 63 determines that the character is one color, a histogram of the pixel values is generated, and the pixel value that is the mode value is determined as the representative color. The representative color determination unit 65 stores this representative color as representative color information in the storage unit 12 in association with information defining a basic rectangle.

  In this way, the number-of-colors calculation unit 51 determines the number of colors of specific pixels included in each basic rectangle and the representative color information.

  The continuity determination unit 52 checks whether the number of colors determined by the color number determination unit 51 is equal to or greater than a predetermined threshold value. If the number of colors is equal to or greater than a predetermined threshold value, the continuity feature amount of pixel values and the inversion feature amount are calculated for each pixel of the target basic rectangle with the target basic rectangle as a processing target, Based on the continuity feature quantity and the inverted feature quantity, it is checked whether or not the pixels in the target basic rectangle are in gradation.

  Specifically, the continuity determination unit 52 performs the following process as a process of calculating the continuity feature value and the inverted feature value. That is, the continuity feature value and the inverted feature value are reset to “0”, and individual pixels included in the target basic rectangle are sequentially selected as the target pixel. Of the pixels adjacent to the pixel of interest (eight pixels from A to H around the pixel of interest P in FIG. 12), the distance between the pixel values of the pixel of interest (such as the sum of squares of the differences between the components) is predetermined. The number of pixels below the continuity determination threshold is added to the continuity feature amount, and the number of pixels whose distance is greater than or equal to the continuity determination threshold is added to the inversion feature amount.

  Then, the continuity determination unit 52 compares the continuity feature value and the inverted feature value, and determines whether or not the pixel group in the target basic rectangle is gradation. For example, if the absolute value of the difference between the continuity feature value and the reverse feature value is greater than a predetermined gradation determination threshold value, it is determined that the image is gradation.

  In this way, the color number calculation unit 51 and the continuity determination unit 52 allow the pixel group included in each basic rectangle to be (1) one color, (2) a plurality of colors that are not gradations, and (3 ) Classified as gradation. The control unit 11 stores the classification result in the storage unit 12 as a color number classification database in association with the identifier of each basic rectangle.

  One characteristic of the present embodiment is that the number of colors considered significant for each pixel group in the basic rectangle is obtained as in the process of counting the number of colors. Thus, for example, the original image data is obtained by scanning a paper document, the edge of the original line segment becomes dull, and the edge portion and the central portion of the line segment have a slightly different color. Even in this case, it is determined that one color is significant as a line segment.

  Further, in this embodiment, since it is determined whether or not to be processed based on the number of colors in each basic rectangle, gradation characters and multicolor characters are not limited colors.

  The representative color determination unit 53 refers to the row rectangle database stored in the storage unit 12 and sequentially selects each row rectangle as a target row rectangle. Then, an identifier of at least one basic rectangle included in the target row rectangle is extracted, and it is checked whether or not there is a representative pixel value associated with each extracted identifier by referring to the representative pixel value database. If there is a representative pixel value, the representative pixel value is read out. Then, the representative color determination unit 53 determines the representative colors of the basic rectangle determined to be one color within the target row rectangle.

  A specific process for determining the representative color will be described below. Here, the color components (a, b, c) in which the representative pixel value P for the i-th basic rectangle is set by three values a, b, c. c may be any color component such as L * a * b * or RGB) and is expressed as Pi (Pia, Pib, Pic). Further, the control unit 11 calculates the standard deviation σi (σia, σib, σic) of each color component of the pixel value in the basic rectangle (the basic rectangle with which the representative pixel value is associated) in the original image data. .

  The representative color determining unit 53 determines whether the i-th basic rectangle and j (i and j are not equal to each other) and the values of Pi, σi, Pj, and σj in the color space. Are generated in accordance with whether or not the two geometric shapes intersect in the color space (there is an overlapping portion), and the representative pixel value for the i-th basic rectangle, j It is determined whether the representative pixel values for the th basic rectangle are the same color. The presence / absence of the intersection of geometric shapes should be a predetermined uniform color condition.

  Specifically, a rectangular parallelepiped or an elliptic sphere can be used as the geometric shape. That is, when the rectangular parallelepiped (Pia ± α × σia, Pib ± α × σib, Pic ± α × σic) and (Pja ± α × σja, Pjb ± α × σjb, Pjc ± α × σjc) overlap, i It may be determined that the representative pixel value for the th basic rectangle and the representative pixel value for the j th basic rectangle are the same color. In addition, when an elliptical sphere having a diameter of each value of σi centered on Pi and an elliptical sphere having a diameter of each value of σj centered on Pj, a representative pixel value for the i-th basic rectangle, It may be determined that the representative pixel values for the jth basic rectangle are the same color.

  Furthermore, when calculating the standard deviation, instead of the color components (a, b, c), three axes (a ′, b ′, c ′) obtained by widely known principal component analysis are used as the respective components. These standard deviations in the axial direction are calculated, and a rectangular parallelepiped (Pia ± α × σia ′, Pib ± α × σib ′, Pic ± α × σic ′) and (Pja ± α × σja ′, Pjb ± α × σjb) ′, Pjc ± α × σjc ′), it may be determined that the representative pixel value for the i-th basic rectangle and the representative pixel value for the j-th basic rectangle are the same color.

  In these, α is a parameter indicating the ease of determination as a uniform color, and may be changed according to the distance between the basic rectangles to be compared, for example (for example, the value of α in the adjacent basic rectangles). Or the like to make it easy to determine that the color is uniform), or a predetermined constant value.

  The representative color determination unit 53 specifies at least one basic rectangle group that is determined to be a uniform color by the above processing for a combination of two basic rectangles extracted from the target row rectangle. Then, for each identified basic rectangle group, pixels included in the basic rectangle group are extracted, and a color-specific row region including the extracted pixels is generated. As a result, for each pixel that is associated with one representative value that is a pixel in the row rectangle, color-specific row region information including information that associates the representative value with information that reproduces the pixel is generated. The data is stored in the storage unit 12 in association with the row rectangle identifier.

  For each color row area information stored in the storage unit 12, the limited colorization unit 54 limits the pixel value in each color row area to a predetermined color (for example, n colors) or less and sets each pixel value. Set.

  Next, the operation of the same color region separation unit 25 will be described with reference to FIG. Specifically, the character region is arranged in two lines as shown in FIG. 13A, and a circle with gradation is included at the beginning of each line, and a red character (indicated by a broken line) is included in a part of the character. Virtually enclosed part) is included. FIG. 13A shows a state in which a row rectangle is defined for each row.

  The color number determination unit 51 determines the number of colors for each character. The circle with gradation is determined to include a plurality of colors, and the other character portions are determined to be black or red. The continuity determination unit 52 excludes a circle with gradation from the processing target, determines a black representative color for a black character, and determines a red representative color for a red character. The representative color determined by the continuity determination unit 52 may be a color determined to be the same black, for example, or the pixel value itself may be different.

  The representative color determination unit 53, for each basic rectangle for a character determined to be black in each row rectangle, includes a color-specific row rectangle (FIG. 13B) including pixels (black pixels) in these basic rectangles. (A), (c)) and the basic rectangle for the character determined to be red, the color-specific row rectangles including the pixels (red pixels) in these basic rectangles ((b) in FIG. 13B). ), (D)).

  Then, the limited colorization unit 54 selects one color for each color row rectangle based on the pixel value in the color row rectangle (the number of color determination units 51 is not m for one color pixel group but m for a pixel group of m colors). When the representative pixel value is determined, a limited color of the same value as m or less) is determined. For example, for a color-by-color row rectangle including black pixels, one pixel value corresponding to black is defined as a limited color, and for a color-by-color rectangle including red pixels, one pixel value corresponding to red is defined as a limited color. Determine as In addition, among the pixels in the row rectangle for each color, a pixel determined as a limited color is a black pixel, and the other pixels are binarized as a white pixel. As a result, the pixel values in the basic area of the image data are in a limited color state.

  In addition, while the binarized image in the color-specific row rectangle is scanned in the raster scan order, the binarized image is subjected to run-length compression such as MMR (Modified Modified Read) to generate character line drawing compressed data. . The same color region separation unit 25 associates the row rectangle identifier with the color-by-color row rectangle obtained by the pixels included in the row rectangle, and further associates each color-by-color row rectangle with the limited color information defined for the color rectangle. In association therewith, it is stored in the storage unit 12 as character / line drawing plane data. In addition, the limited color information may be held as identifier information (hereinafter referred to as a color tag) representing each pixel value, instead of the pixel value itself.

  Further, the basic rectangle that is determined by the number-of-colors determination unit 51 as (2) a plurality of colors and not gradation is not limited color, and the value of the pixel in the basic rectangle is used as it is or by a method such as ZIP. It is compressed and included in the character / line drawing plane data together with the information (coordinate information) for defining the basic rectangle, and stored in the storage unit 12. Further, (3) for the basic rectangle determined to be gradation, the value of the pixel in the basic rectangle is compressed as it is or by a method such as ZIP, and the character (along with coordinate information) for defining the basic rectangle It may be included in the line drawing plane data and stored in the storage unit 12. (3) The basic rectangle determined to be gradation may be included in the picture plane data and stored in the storage unit 12. Further, (2) for a basic rectangle that is determined to be a plurality of colors and not gradation, a process of grouping the significant pixels in the pixel group as a pixel group for each pixel value is performed, and a representative color is determined for each pixel group. It is good as well. In this case, the representative color determination unit 65 selects a pixel that is not yet grouped into any pixel group among the significant pixels as a target pixel, and is also grouped into any pixel group among the significant pixels. Pixels other than the target pixel are sequentially selected as target pixels one by one. Then, the pixel value of the target pixel is compared with the pixel value of the target pixel to determine whether or not the target pixel belongs to the same pixel group (a pixel group determined to have the same color). For example, a distance in the color space between the pixel value of the target pixel and the pixel value of the target pixel is calculated, and when the distance is smaller than a predetermined threshold, it is determined that they belong to the same pixel group. In this case, for each pixel group, an average value of the pixel values determined to belong to the same pixel group is determined as the pixel group representative pixel value. Then, this set of pixel group representative pixel values may be stored in the storage unit 12 as representative color information in association with information specifying the basic rectangle. In this case, a color-specific basic rectangle is generated for each pixel group, and is included in a different color-by-color row rectangle and color-specific region for each color-basis rectangle.

  In addition, the color number determination unit 51 may determine that there is one color without associating with the representative color candidate when the width of the pixel block in the target basic rectangle is smaller than the predetermined width threshold value. Good. That is, if the pixel block is composed only of thin line portions such as thin characters, the number of significant pixels (pixels constituting the pixel block) is small relative to the total number of pixels in the basic rectangle even if it is actually one color. This is because determination by the determination unit 63 becomes difficult.

  Whether the width of the pixel block in the basic rectangle of interest is smaller than a predetermined width threshold value is determined as follows. That is, the target basic rectangle is sequentially scanned for each line in the horizontal direction and the vertical direction, the number of significant pixels (pixels constituting the pixel block) is continuously counted, and the result of the count in each line is counted. Of these, the largest one may be the width of the pixel block in the basic rectangle of interest.

  In consideration of the case where the pixel values in the basic rectangle vary, the color number determination unit 51 may determine the representative pixel value after performing the smoothing process. Here, the smoothing process includes a process of sequentially specifying each pixel in each basic rectangle as a target pixel, and setting an average value of the value of the target pixel and the value of a pixel adjacent thereto as the value of the target pixel.

  Further, only the significant pixel may be selected as the target pixel during the smoothing process. In the smoothing process, when calculating the average value, the average value may be calculated with reference to only the values of the pixels constituting the character. This prevents the representative color of the character from being influenced by the background color by referring to the pixel value of the portion other than the character.

  Here, the representative value determined after the smoothing process may be corrected. That is, the same color region separation unit 25 of the present embodiment corrects the luminance of the determined representative value candidate, and determines the corrected value as the representative value. Here, the luminance can be corrected using, for example, a tone curve (correction function) as shown in FIG. The tone curve shown in FIG. 14 is an output value (value after correction, that is, a value determined as a representative value) for the input value (luminance of the representative color candidate before correction) from the minimum value MIN to the first threshold value TH1. Brightness) is the minimum value MIN, and the output value from the second threshold TH2 (where TH2> TH1) to the maximum value MAX is set to the maximum value MAX. The tone curve has a center value MID between the maximum value MAX and the minimum value MIN (for example, when the maximum value is “255” and the minimum value is “0”, the MID is “128”). The output value for this may be set to be substantially MID.

  That is, the same color region separation unit 25 corrects the representative value candidate (represented by YCbCr in the present embodiment) with the tone curve of FIG. Luminance (Y ′) is determined, and a value specified by this Y ′ and representative value candidate color difference components Cb and Cr is determined as a representative value.

  If the representative color candidate is expressed in a color space that does not include a luminance component, such as RGB, the above processing is performed after conversion to a color space value that includes a luminance component, such as L * a * b * or YCbCr. Should be done.

  Further, although only the luminance is corrected here, the color difference component may also be corrected. Specifically, the same color region separation unit 25 performs correction to reduce the number of gradations regarding the luminance component value of the representative color candidate value when each color difference component of the representative color candidate satisfies a predetermined condition, The corrected value may be determined as the representative value.

  Specifically, as shown in FIG. 15, the color difference components (a *, b *) of the representative color candidates expressed in the color space of L * a * b are calculated from the center values of the corresponding color difference component value ranges. When the condition that it is within the predetermined range (inside the circle defined by THa and THb in FIG. 15) is satisfied, for example, the luminance component L expressed in 256 gradations is converted to 4 gradations or 8th floor. The tone is reduced to a predetermined gradation such as a tone. In this case, the value of the color difference component may be set to the center value. Here, the predetermined ranges THa and THb for each component may be the same value or different values.

  By this processing, when the character color is gray (including black), a representative value that reproduces the original color of the character color is set. For example, when the character color is black, the color difference component and the luminance component are originally “0”. However, depending on the characteristics of the scanner and the encoding format of the original image data (for example, JPEG), the color difference component may be “ In some cases, the luminance component is not “0” or the luminance component is not “0”. Therefore, by performing the processing relating to the color difference component shown here, the representative value can be set to the original black color.

  As described above, according to the present embodiment, the representative color candidate of the target image area is determined based on the pixel values in the basic rectangle or the row rectangle as the target image area defined in the image data to be processed. Then, the luminance is corrected to determine the representative color.

  Here, after performing the smoothing process, the correction process is performed to determine the representative value. However, after performing the above correction process for each pixel by reversing the process order, A representative value may be determined by performing a smoothing process and calculating a histogram. Although the case of characters has been described here, the same processing is performed for line drawings.

  With this smoothing and correction process, in this embodiment, even if there is a variation in the pixel values that make up a character or line drawing, the influence of the variation is reduced and there is no sense of incongruity with the color of the original image data. A representative color can be determined.

  Further, in the above description, the limited color processing is performed for each row rectangle and for each representative color. However, for the limited colors determined for each row rectangle by color, the same processing as in the representative color determination unit 53 is performed. Extract the pixels in the color-specific row rectangles that are associated with the limited colors that are determined to be uniform colors by the process of, and select the rectangle that contains the extracted pixels and has the same size as the rectangle that defines the character / line drawing part. It may be generated as separate character / line drawing partial data (color-specific area information), and a limited color may be further defined for this.

  Further, instead of generating the color-specific row rectangles, the representative color determining unit 53 directly relates to the basic pixel values that are basic rectangles included in one character line drawing portion and are determined to be uniform colors. Extracts the pixels in the rectangle, generates a rectangle with the same size as the rectangle that defines the character / line drawing part as the character / line drawing part data (color-specific area information), and restricts this A color may be defined.

  When generating these color-specific character / line drawing part data, the character / line drawing part data (color-specific character / line drawing part data) associated with the identifier of the character / line drawing part that is the source of the color-specific character / line drawing part data and the information for specifying the limited color, respectively. The internal pixel value may be binarized and may be MMR encoded) as character / line drawing plane data.

[6. Cavity processing section]
The hole-filling processing unit 26 extracts a region corresponding to the pattern candidate region from the original image data, and detects a pixel of a character (a pixel constituting the character) detected by the processing of the layout processing unit 24 from this region. The pattern partial image data is generated by removing.

  Then, each pixel of the pattern partial image data is scanned in the raster scan order, and if the pixel of interest selected by the scanning is not removed, the pixel value of the pixel of interest remains as it is and the pixel of the pixel of interest The value is stored in the work memory of the storage unit 12 as the previous pixel value. If another pixel value is already stored as the previous pixel value, the stored content is overwritten.

  If the pixel of interest selected by scanning is a removed pixel, the pixel value of the pixel of interest is set to the immediately preceding pixel value stored. As a result, the pixel value of the removed portion becomes the same as the previous pixel value in the raster scan order, and the compression efficiency can be improved in many compression processes.

  Then, the pattern partial image data after this processing is performed is stored in the storage unit 12 as pattern plane data.

[7. Compression processing unit]
The compression processing unit 27 performs JPEG compression on the pattern plane data stored in the storage unit 12 to generate compressed pattern plane data. In addition, the compression processing unit 27 concatenates the character / line drawing plane data stored in the storage unit 12 and the compressed picture plane data to generate a series of data.

  Specifically, the series of data can be PDF (Portable Document Format) data. That is, an instruction to generate a bitmap generated by decompressing the compressed picture plane data (a bitmap of the picture plane data), and each character or line drawing included in the character / line drawing plane data is drawn on the bitmap of the picture plane data. PDF data including an instruction to be executed.

  Here, the instruction to draw each character or line drawing included in the character line drawing plane data is to read out the character line drawing compressed data, the representative color information related to this and the coordinate information such as the basic rectangle one by one, For the set of the above, the character line drawing compressed data is expanded to generate a binarized image, the color of the black pixel of the binarized image is set as the representative color, and the basic rectangle etc. on the bitmap of the picture plane data This is an instruction for transparent composition at the position set as the coordinate information. Here, “transparent composition” means that pixels other than black pixels (pixels set as representative colors) in the binarized image are not overwritten, but only pixels set as representative colors are overwritten.

  The compression processing unit 27 stores the generated PDF data in the storage unit 12 or outputs the PDF data to the image output unit 14 to send it to an external device.

[Operation]
Since the image processing apparatus according to the present embodiment has the above-described configuration, it operates as follows. Here, a document including a character portion (T1, T2) as shown in FIG. 16A, a photograph portion (P), and a map portion (M) as a line drawing portion is input from the image input unit 13, and this A case where image data of a document is a processing target will be described as an example. In the example of FIG. 16A, a part (T2) of the character part is superimposed on the photograph part (P). The map portion (M) includes a road map and characters. Here, although shown in black and white for the sake of convenience, the road map and characters in the map portion are actually represented in different colors, and the photograph may be in color.

  The preprocessing unit 21 converts the pixel value of the image data into a value in a predetermined color space (YCbCr). The pattern candidate portion specifying processing unit 22 binarizes the image data (original image data), and generates image data from which the character region (T1) and the diagram (M) are removed by removing the small area. (FIG. 16B). At this time, most of the characters and the road map are removed (they may remain partially), but the characters superimposed on the photograph portion are in a state specified as the pattern candidate portion as it is.

  The character / line drawing extraction processing unit 23 binarizes the original image data and extracts the character / line drawing part by a method such as specifying a small area part. At this time, the original image data is divided into a plurality of regions, and a binarization threshold value is adaptively determined for each region obtained by the division (a method disclosed in Patent Document 3). By performing binarization, characters can be extracted from a colored portion such as a map (FIG. 16C).

  The layout processing unit 24 performs a layout analysis process in the pattern candidate part, and extracts a character part (T2) remaining in the pattern candidate part. The same color region separation unit 25 calculates the number of colors and the continuity of colors included in the character portion or diagram portion extracted by the character / line drawing extraction processing unit 23 or the layout processing unit 24. If the number of colors is one, a representative color is determined. One of the characteristic features of the present embodiment is that the representative color of the character part is included in the area obtained in the layout analysis process from the individual character step by step through the line rectangle to the entire character part. Information that is determined according to the relationship and reproduces the pixels in the rectangle for each representative color (bitmap itself, bitmap compression result, etc.) is generated.

  In other words, in the present embodiment, the control unit 11 uses layout analysis processing to define information that defines a region (a region corresponding to at least the character portion (T1, T2)) including a plurality of pixel blocks determined as individual characters. Generated. Then, each area defined by the generated information is processed as follows.

  First, one of the areas is selected, and for each significant pixel (pixel constituting the pixel block) included in the selected area, the pixel value is compared with a predetermined representative color candidate. The value is associated with one of the representative color candidates. For example, significant pixels in the area (the total number of pixels is 19) have pixel values as shown in FIG. 17A (here, for the sake of simplicity, it is assumed that the gray scale values are from 0 to 15). If the representative color candidate is determined to be (0, 5, 10, 15), the value “2” in FIG. 17A is the representative color candidate value “0” (which is the closest value to 2). The value “3” is associated with the representative color candidate value “5” (which is the closest value to 3), and the value “15” is associated with the representative color candidate value “15” (FIG. 17B). )).

  Further, according to the association result shown in FIG. 17B, the number of pixels associated with the value “0” is 5, 13 for “5”, and 1 for “15”. Also, the value “5” that is a candidate of the above does not exceed a predetermined threshold (for example, 15 that is 80% of the total number of pixels of 19). Therefore, the control unit 11 extracts “0” and “5” which are the top two candidates, calculates an average value “2.5” thereof, and, for example, a new representative of the value “2” as a value close thereto. Color candidates are generated and included in the representative color candidates. As a result, the representative color candidates become (0, 2, 5, 10, 15). Then, for each of the original pixel values (FIG. 17A), processing for associating the representative color candidates again is performed. In addition, here, in the case where it is possible to relate to either the original representative color candidate or the newly added representative color candidate, it is assumed to be related to the newly added representative color candidate.

  Alternatively, three of the top two candidates “0” and “5” and “2” having a value close to the average value “2.5” are set as new representative color candidates (0, 2, 5). Also good.

  Then, as shown in FIG. 17C, the control unit 11 associates the pixel values “2” and “3” with the representative color candidate “2” as shown in FIG. Since the number of pixels associated with the color candidate “2” is 18 and exceeds 80% of the total number of pixels, this pixel block is determined to be one color, and its representative pixel value is “2”. Determine. Also, binarized information (information for reproducing the pixel) is generated by setting all significant pixels in the region as black pixels and other pixels as white pixels, and the representative pixel value and information for reproducing the pixels are generated. The data are stored in the storage unit 12 in association with each other.

  Further, for each row region, the control unit 11 associates the basic regions associated with the representative pixel values that are close to each other among the basic regions included in the row region, and for each set of the basic regions associated with each other. One representative pixel value is further determined. In this way, information on the row region for each representative pixel value (information on the row region for each color) is generated. Further, the control unit 11 generates a color-specific region by associating the color-specific row regions associated with the representative pixel values close to each other among the color-specific row regions included in each character part (T1, T2). Then, one representative pixel value is further determined from the set of the color-specific row regions for each color-specific region.

  In this way, data including information specifying the color-specific area is generated as character / line drawing plane data. Further, at this time, the character portion composed of a plurality of colors of a predetermined number or more (for example, a character with gradation) is not subjected to the limited color processing, and is compressed as it is or by another method such as ZIP. Included in plain data.

  Furthermore, here, when there is continuity in color, for example, it may be set so that it is also excluded from the character portion and treated as a picture area later. As for the line drawing part (M), coordinate information for defining the area may be separately generated.

  The burial processing unit 26 generates image data obtained by removing the character portion (T2) inherent in the pattern candidate portion (FIG. 16D). Originally, the character portion T2 is white as in the character shape. However, in FIG. 16D, the rectangular portion including the character shape is shown as white for easy understanding of the drawing. FIGS. 16D and 16E show only the pattern image portion. Then, the hole filling processing unit 26 sets the value of the removed pixel to the nearest pixel value (the pixel value that has not been removed) in the scan line order (FIG. 16E), and generates the picture plane data. To do.

  The compression processing unit 27 performs JPEG compression on the pattern plane data, generates PDF data in combination with the character line drawing plane data, and outputs this to the image output unit 14. The image output unit 14 outputs the PDF data to an external device.

  Here, the compression processing unit 27 may perform a process of reducing the image size (reduction process) on the design plane data before JPEG compression to further improve the compression rate.

1 is a configuration block diagram illustrating an example of an image processing apparatus according to an embodiment of the present invention. It is a functional block diagram showing the processing content performed by the control part of the image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram showing the example of the processing content of the pattern candidate part specific process part. FIG. 6 is a flowchart illustrating a processing example of an attribute determination unit 34. It is explanatory drawing showing the process example of the pattern candidate part specific process part. 4 is a functional block diagram illustrating an example of processing contents of a layout processing unit 24. FIG. 7 is an explanatory diagram illustrating a processing example of a layout processing unit 24. FIG. FIG. 10 is a flowchart illustrating a processing example of a noise determination unit 48. FIG. 10 is a flowchart illustrating a processing example of a noise determination unit 48. 6 is a functional block diagram illustrating an example of processing contents of the same color region separation unit 25. FIG. 5 is a functional block diagram illustrating an example of processing contents of a color number determination unit 51. FIG. It is explanatory drawing showing the process example of the continuity determination part. 11 is an explanatory diagram illustrating a processing example of the same color region separation unit 25. FIG. It is explanatory drawing showing the example of the tone curve utilized in the same color area separation part 25. FIG. FIG. 11 is an explanatory diagram illustrating processing conditions for correction processing in the same color region separation unit 25. It is explanatory drawing showing the process example of the image processing apparatus which concerns on embodiment of this invention. 6 is an explanatory diagram illustrating an operation example of a color number determination unit 51. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Control part, 12 Memory | storage part, 13 Image input part, 14 Image output part, 21 Pre-processing part, 22 Design candidate part specific processing part, 23 Character line drawing extraction processing part, 24 Layout processing part, 25 Same color area separation part, 26 burial processing unit, 27 compression processing unit, 31, 41 binarization processing unit, 32, 42 connected pixel extraction unit, 33 feature amount calculation unit, 34 attribute determination unit, 35 non-pattern region processing unit, 36 background region coating Crushing part, 37 design candidate area creating part, 38 expansion / contraction part, 43 basic rectangle defining part, 44 first separator detecting part, 45 line rectangular defining part, 46 second separator detecting part, 47 character area defining part, 48 noise determination Part, 49 character part specifying part, 51 color number determining part, 52 continuity determining part, 53 representative color determining part, 54 limited color forming part, 61 representative color candidate associating part, 62 pixel number calculating part, 63 determining part, 64 New Representative color candidate determination unit, 65 Representative color determination unit.

Claims (7)

Means for demarcating a basic region including a pixel block determined as an individual character from at least a part of image data to be processed;
For each pixel constituting the pixel block included in the defined basic region, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set, and after the limited color Color number determination means for performing a color number determination process for determining the number of colors of the pixel block based on the appearance frequency of the representative pixel value candidate in the pixel block;
Including
An image processing apparatus, wherein predetermined image processing is executed based on the determination result of the number of colors. The image processing apparatus according to claim 1,
When the number-of-colors determination unit determines that the number of colors of the pixel block is plural, the plurality of representative pixel value candidates are specified in order of appearance frequency, and a new representative pixel value is determined based on the specified representative pixel value candidates. An image processing apparatus characterized by generating a candidate, including the candidate in a set of representative pixel value candidates, and performing the color number determination process again. The image processing apparatus according to claim 1,
When the number-of-colors determination unit determines that the number of colors of the pixel block is plural, the plurality of representative pixel value candidates are specified in order of appearance frequency, and a new representative pixel value is determined based on the specified representative pixel value candidates. An image processing apparatus that generates candidates, sets the specified representative pixel value candidate and the new representative pixel value candidate as new representative pixel value candidates, and performs the color number determination process again. The image processing apparatus according to claim 1, wherein:
Information on continuity of pixel values is generated for the pixel block determined to have a plurality of colors by the color number determination means, and the pixel block becomes a gradation based on the information on continuity of the pixel value. Further comprising means for determining whether or not
An image processing apparatus that changes the content of image processing on the pixel block based on the result of the determination. The image processing apparatus according to any one of claims 1 to 4, wherein:
The number-of-colors determination unit generates information about the width of the pixel block, and determines that the pixel block is expressed by one color when the information about the width of the pixel block is less than a predetermined width threshold value. An image processing apparatus. Using a computer,
Defining a basic region including a pixel block that is determined as an individual character from at least a part of the image data to be processed;
For each pixel constituting the pixel block included in the defined basic region, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set, and after the limited color Based on the appearance frequency of the representative pixel value candidate in the pixel block, let the number of colors determination processing to determine the number of colors of the pixel block,
A predetermined image processing is executed based on the determination result of the number of colors. On the computer,
A procedure for defining a basic region including a pixel block determined as an individual character from at least a part of image data to be processed;
For each pixel constituting the pixel block included in the defined basic region, the pixel value of the pixel is limited to one of the representative pixel value candidates included in the representative pixel value candidate set, and after the limited color A procedure for performing a color number determination process for determining the number of colors of the pixel block based on the appearance frequency of the representative pixel value candidates in the pixel block;
A procedure for performing predetermined image processing based on the determination result of the number of colors;
An image processing program for executing

Images