JP2005210650A - Image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device by which picture portions to be nominated can be specified while reducing processing load. <P>SOLUTION: Labeling process of image data to be processed is executed. Significant pixels joined together are extracted as joined pixels and nonpicture regions are eliminated from among the joined pixels. With respect to image data after the nonpicture regions have been eliminated, portions existing as a background for remaining joined pixels are specified as background portions. Then, the background portions specified are eliminated to produce information for specifying the picture regions. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that separates a character part and a picture part from image data and performs predetermined processing.

  Raster image data (hereinafter simply referred to as “image data” unless otherwise distinguished) has many image elements with different properties such as character (text) parts and natural picture parts (design parts). Can be included. Due to the difference in properties of these image elements, it is often preferable to perform different image processing for each image element, for example, compression by a different method is suitable for compression processing.

Therefore, conventionally, image processing called so-called T / I separation has been researched and developed. Conventionally, as a method of T / I separation, for example, a processing target image is binarized, a continuous region of black pixels is defined, and when the size of the defined region is lower than a predetermined threshold value, the region is included in the region. There is a method of determining that an included black pixel represents a character (Patent Document 1).
JP 2003-8909 A JP 2002-175532 A

  However, in the above-described conventional method, the area of the character / line drawing is simply identified by the size of the connected pixel, and the portion that is not the character / line drawing is handled as a design, so that the background is treated as a design. There was a point.

  Therefore, a method of extracting the pattern portion from the original multi-valued image data using information related to the color has been considered. However, when multi-value image data is used, there is a problem that the amount of data becomes enormous and the processing load increases.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus capable of specifying a portion to be a candidate for a pattern while reducing a processing load.

  The present invention for solving the problems of the conventional example described above is an image processing apparatus, which performs labeling processing on image data to be processed, and extracts significant pixel portions connected to each other as connected pixels. , A means for removing a non-picture area from the connected pixels, and for the image data after removing the non-picture area, a portion that is a background with respect to the remaining connected pixel part is identified as a background part, And a design area specifying means for generating information for specifying the design area by removing the specified background portion.

  In addition, the image data after removing the non-picture area is subjected to expansion processing for setting pixels in the vicinity of significant pixels as significant pixels, and in the image data after the expansion processing, pixels in the vicinity of insignificant pixels are not significant. The image area specifying means further includes a background portion for the remaining connected pixel portion in the image data after the expansion / contraction processing. It is also possible to generate the information specifying the pattern area by removing the specified background portion.

  Further, the expansion / contraction means converts the image data after removing the non-picture region into run length data, performs the expansion processing on the converted run length data, and executes the run length after the contraction processing. These data may be converted into bitmap data and output.

  Furthermore, the present invention for solving the problems of the above-described conventional example is an image processing device, and each pixel constituting image data to be processed is a background pixel that satisfies a predetermined background pixel condition, Means for classifying as significant pixels not satisfying the predetermined background pixel condition, and extracting a connected portion of the significant pixels separated from each other by an edge of the background pixel or image data as a connected pixel region; For each pixel, means for generating, as significant pixel distance information, a distance from the significant pixel to the adjacent significant pixel to the background pixel, and size relationship information related to the size of the selected region of interest. Generating means, significant pixel distance information for each significant pixel included in the selected attention area, and a picture area of the attention area based on the size relationship information It is characterized in that it comprises, means for identifying.

  Further, the present invention for solving the problems of the conventional example described above includes means for performing a labeling process on image data to be processed and extracting significant pixel portions connected to each other as first connected pixels, and the connected pixels. A means for removing a non-picture area, a means for extracting a portion where insignificant pixels are connected to each other as image connection data after removing the non-picture area, and a second connected pixel. And a means for specifying a second connected pixel in contact with the boundary of the image data as a background part and specifying an area other than the specified background part as a picture area.

  In addition, an image processing method according to an aspect of the present invention uses a computer to perform a labeling process on image data to be processed, extracts significant pixel portions connected to each other as connected pixels, and selects non-connected pixels from the connected pixels. For the image data after removing the picture area and removing the non-picture area, the part that is the background with respect to the remaining connected pixel part is specified as the background part, and the specified background part is removed. The method is characterized by generating information for specifying a picture area.

  According to another aspect of the present invention, there is provided an image processing method using a computer, wherein each pixel constituting image data to be processed includes a background pixel that satisfies a predetermined background pixel condition, and the predetermined background pixel. Classified as significant pixels that do not satisfy the conditions, and extracted a connected portion of the significant pixels partitioned by the background pixel or the edge of the image data as a connected pixel region, and for each of the significant pixels, the significant pixel The distance from the pixel to the adjacent significant pixel to the background pixel is generated as significant pixel distance information, size relationship information related to the size of the selected attention area is generated, and the selected attention area The design area is specified from the attention area based on the significant pixel distance information for each significant pixel and the size relation information.

  Furthermore, an image processing method according to another aspect of the present invention uses a computer to perform a labeling process on image data to be processed, extracts significant pixel portions connected to each other as first connected pixels, and connects the connected pixels. In the image data after removing the non-picture area and extracting the non-picture area, a portion where insignificant pixels are connected to each other is extracted as a second connected pixel, and the image is extracted from the second connected pixels. A second connected pixel in contact with the data boundary is specified as a background part, and an area other than the specified background part is specified as a pattern area.

  Furthermore, an image processing program according to another aspect of the present invention includes a procedure for performing, on a computer, a labeling process on image data to be processed, and extracting significant pixel portions connected to each other as connected pixels; Among these, the procedure for removing the non-picture area and the image data after removing the non-picture area are identified as a background part with respect to the remaining connected pixel part, and the identified background part And a procedure for generating information for specifying a picture area.

  Furthermore, an image processing program according to another aspect of the present invention provides a computer with each pixel constituting image data to be processed, a background pixel that satisfies a predetermined background pixel condition, and the predetermined background pixel condition. For each of the significant pixels, and a procedure for extracting a connected portion of the significant pixels separated from each other by the background pixel or the edge of the image data as a connected pixel region. A procedure for generating a distance from a significant pixel to a background pixel by following an adjacent significant pixel as significant pixel distance information, a procedure for generating size relationship information related to the size of the selected region of interest, The design area is identified among the attention areas based on the significant pixel distance information for each significant pixel included in the selected attention area and the size relation information. Is characterized by executing the instructions that, the.

  Furthermore, an image processing program according to another aspect of the present invention includes a procedure for causing a computer to perform a labeling process on image data to be processed and to extract significant pixel portions connected to each other as first connected pixels, and the connection A procedure for removing a non-picture region from the pixels, a procedure for extracting a portion where non-significant pixels are connected to each other as image connection data after removing the non-picture region, and the second connection pixel Among the pixels, the second connected pixel that is in contact with the boundary of the image data is specified as a background part, and a procedure for specifying an area other than the specified background part as a picture area is executed.

  As shown in FIG. 1, the image processing apparatus according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an image input unit 13, and an image output unit 14. The control unit 11 operates in accordance with a program stored in the storage unit 12 and performs each image processing described later. The contents of this image processing will be described in detail later.

  The storage unit 12 holds a program executed by the control unit 11. The storage unit 12 also operates as a work memory that stores various data generated during the process of the control unit 11. Specifically, the storage unit 12 can be implemented as a computer-readable recording medium and a device that writes data to or reads data from the recording medium (for example, a hard disk device or a memory device).

  The image input unit 13 is, for example, a scanner, and outputs image data obtained by optically reading a document to the control unit 11. Here, it is assumed that the value of each pixel is expressed in an RGB (red, green, blue) color space in the image data output from the image input unit 13. The image output unit 14 outputs image data in accordance with an instruction input from the control unit 11. For example, the image output unit 14 outputs the image data to an image forming unit (printer or the like) or transmits it to an external device via a network. Is what you do.

  Next, the content of the process of the control part 11 is demonstrated. As functionally shown in FIG. 2, the control unit 11 of the present embodiment sets the image data input from the image input unit 13 as a processing target, and performs predetermined preprocessing on the image data that is the processing target. A pre-processing unit 21 that performs the pattern processing, a pattern candidate portion specifying processing unit 22 that specifies a portion that is a candidate for a pattern portion (design candidate portion), a character / line image extraction processing unit 23 that extracts a character / line image, and a layout process. The layout processing unit 24, the same color region separation unit 25, a hole filling processing unit 26, and a compression processing unit 27 are included.

  Hereinafter, each of these parts will be described in detail.

[1. Pre-processing section]
In the preprocessing unit 21, the value of each pixel of the image data (processing target image data) input from the image input unit 13 is converted from RGB to YCbCr (a value composed of luminance and color difference). Specifically, the conversion can be performed using the following equation (1). Here, the value of each component of RGB is assumed to be a value from 0x00 (“0x” indicates a hexadecimal number) to 0xFF. Further, the pre-processing unit 21 may correct the gradation of each pixel value based on the luminance and saturation of the background area. However, the gradation correction process is not always necessary.

[2. Design candidate area identification processing unit]
The pattern candidate portion identification processing unit 22 performs a process of identifying an area estimated as a pattern area from the image data expressed in the YCbCr color space output from the preprocessing unit 21 as a pattern candidate area. Hereinafter, the processing method of the pattern candidate portion identification processing unit 22 will be specifically described.

[2a. First method]
As shown in FIG. 3A, the pattern candidate portion specifying processing unit 22 includes a binarization processing unit 31, a connected pixel extraction unit 32, a feature amount calculation unit 33, an attribute determination unit 34, and a non-pattern region. The processing unit 35, the background region painting unit 36, and the pattern candidate region creation unit 37 are configured.

  The binarization processing unit 31 copies and stores image data expressed in the YCbCr color space output from the preprocessing unit 21 in the storage unit 12. Then, referring to the value of each pixel of the copied image data (design area specifying image data), the luminance component of the pixel value is compared with a predetermined binarization threshold value, and the luminance component is Pixels that are equal to or greater than the binarization threshold are set to black (value “1”) as significant pixels. Also, pixels whose luminance component is less than the binarization threshold are set to white (value “0”) as insignificant pixels, and the pattern area specifying image data is converted into binary image data.

  The connected pixel extraction unit 32 extracts an area (connected area) where black pixels (value “1”) are connected from the binary image data. For the extraction of the connected region, a process widely known as a labeling process can be used. When the labeling process is used, each connected area is given a different label identifier and stored in the storage unit 12.

  The feature amount calculation unit 33 calculates a predetermined scale feature amount for each of the connected regions extracted by the connected pixel extraction unit 32. Here, the scale feature amount includes the rectangular area determined in relation to the connected region, the density of black pixels in the rectangle (the number of black pixels in the rectangle divided by the number of all pixels in the rectangle) ) Etc. For example, the feature amount calculation unit 33 performs processing as follows. That is, the feature amount calculation unit 33 defines a rectangle circumscribing each connected region (a circumscribed rectangle) as a related region for each connected region. Specifically, for the circumscribed rectangle, among the pixels included in the connected area, the X coordinate of the pixel with the smallest X coordinate (the leftmost pixel in the image area specific image data) is defined as Xmin, and the Y coordinate is also the smallest. X coordinate and Y coordinate of the one with the largest X coordinate and the one with the largest Y coordinate are set to Ymin. , Xmax and Ymax are defined as rectangles having diagonal lines from (Xmin, Ymin) to (Xmax, Ymax). Then, the area A of the circumscribed rectangle is calculated as (Xmax−Xmin) × (Ymax−Ymin). The area A is equal to the number of all pixels included in the circumscribed rectangle. On the other hand, the feature amount calculation unit 33 counts the number of black pixels in the circumscribed rectangle and divides the number by the number of all the pixels (that is, the value of the area A) to obtain the density D.

  In this way, the feature amount calculation unit 33 uses, for each connected region, the area A (corresponding to the area feature amount) of the circumscribed rectangle and the black pixel density D (corresponding to the density feature amount) as the scale feature of the connected region. Calculate as a quantity. Then, the identifier (for example, label identifier) of each connected area and the scale feature amount are associated with each other and stored in the storage unit 12 as a scale feature amount database.

  The attribute determination unit 34 determines whether each connected region is a pattern candidate region while referring to the scale feature amount database stored in the storage unit 12. As a specific example, the attribute determination unit 34 performs a process based on the premise that a character has a certain size or more and the black pixel density inside the character should be relatively small. That is, an area (small area) less than an area (so-called minimum area) corresponding to the minimum font size (for example, 6 points) used in a general document can be determined as a picture area that does not include characters. By removing these small areas, the character portion can be specified.

  In addition, since the characters are basically composed of lines, even a complex character will not be a black pixel in a certain proportion or more of the pixels in the region. Therefore, even if the rectangle has an area larger than the minimum area, it can be determined that the pattern is a pattern when the proportion of the number of black pixels in the rectangle (that is, the black pixel density) is relatively large.

  Furthermore, in the present embodiment, for example, when the area is equal to or larger than the maximum font size (for example, 24 points) used in a general document (ie, the maximum area), it is basically determined that no character is included. it can. However, such a region may be a diagram (graph etc.) as well as a picture. Therefore, the black pixel density is also used here, and it is determined that the image has a pattern only when the black pixel density is relatively large among rectangles having an area larger than the maximum area. This is because a graph such as a graph is composed of lines, so that the density of black pixels is estimated to be relatively low.

  That is, in the present embodiment, for each connected region, it is determined which of the plurality of classes the area feature amount belongs to, and the area of the density threshold value that is predetermined for each class. The density threshold value related to the class to which the feature quantity belongs is compared with the density feature quantity to generate information (classification information referred to in the present invention) indicating whether or not the pattern is a pattern.

  Specifically, the attribute determination unit 34 of the present embodiment includes a first area threshold value α1 representing the minimum area, a second area threshold value α2 representing the maximum area (where α1 <α2), First and second density threshold values (ρ1, ρ2) are determined in advance, and the determination is performed as follows using these threshold values.

  That is, as shown in FIG. 4, the attribute determination unit 34 selects one of the connected areas not yet selected in this process as a target area from the connected areas included in the scale feature amount database (S1). Then, the scale feature quantity associated with the region of interest is read, and whether or not the area A included in the scale feature quantity is less than the first area threshold value α1 (A <α1 or not). Check (S2). Here, if A <α1, the region of interest is determined as a non-picture region, and the identifier of the region of interest (for example, a label identifier) and the determination result (information indicating that the region is a non-pattern region) are stored in association with each other. It is stored in the unit 12 (S3), and it is checked whether or not there is a connected area not yet selected in this process in the scale feature database (S4). If there is an unselected connected area (if Yes) Then, the process returns to the process S1 to continue the process. Further, if there is no unselected connected region in the process S4 (if No), that is, if the determination is completed for all the connected regions, the process is ended.

  If it is determined in step S2 that A <α1 is not satisfied, whether or not the area A of the attention area exceeds the second area threshold value α2 (whether A> α2) is checked (S5). If A> α2, it is checked whether or not the black pixel density D, which is one of the scale feature quantities of the region of interest, is less than the first density threshold ρ1 (whether D <ρ1) ( S6) If D <ρ1, it is determined that the region of interest is a picture region, and an identifier of the region of interest (for example, a label identifier) and a result of the determination (information indicating that the region is a pattern region) are stored in association with each other. The data is stored in the unit 12 (S7), and the process proceeds to S4 (Y). Furthermore, if D <ρ1 is not satisfied in process S6, the process proceeds to process S3 (X) (that is, determined as a non-picture area) and the process is continued.

  In step S5, if A> α2, if not, the black pixel density D, which is one of the scale feature amounts of the region of interest, is less than the second density threshold value ρ2 (D <ρ2). (S8), if D <ρ2, the process proceeds to process S3 (X) (that is, it is determined as a non-picture area) and the process continues. If D <ρ2, the process proceeds to process S7 ( Z) (that is, it is determined as a picture area) and the processing is continued. The first and second density threshold values ρ1 and ρ2 are determined to be values suitable for extracting line drawings and the like by experiments and values suitable for extracting characters and the like.

  Further, at least one of the area feature amount or the area threshold value may be corrected based on the resolution of the image data to be processed. For example, when the area threshold values α1 and α2 are values set at the resolution r0, when the resolution of the image data to be processed is r, the first area threshold value α1 is set to (r / r0). ) × α1 and the second area threshold value α2 is corrected to (r / r0) × α2.

  The non-design area processing unit 35 refers to the determination result for each connected area that is generated by the attribute determination unit 34 and stored in the storage unit 12, and is determined to be a non-design area in the design area specifying image data. The black pixels included in the connected area are converted into white pixels, and the image area specifying image data in which the non-picture area and the background are white pixels is generated.

  The background area filling unit 36 sets white pixels connected to the boundary of the pattern portion to predetermined values (white, black, etc.) for the pattern area specifying image data in which the non-pattern area and the background are white pixels. Fill with any color other than Here, a process of painting a closed region (inside or outside surrounded by a closed curve) as is widely known is used. The pattern candidate area creation unit 37 sets the color of the pixels that have not been painted with the predetermined value by the background area painting unit 36 to black. A portion set as a black pixel by the pattern candidate area creation unit 37 is handled as a pattern candidate area.

  The operation of the pattern candidate portion specifying processing unit 22 by this method will be described. Hereinafter, a case where the image data illustrated in FIG. 5A is a processing target will be described as an example. For convenience of illustration, FIG. 5A shows the whole as a diagram, but FIG. 5A shows a photograph portion P, a text portion T, a diagram portion G, and a map. And a plate part M such as Further, in the map M, there are cases where characters or the like partially overlapping with the line segments constituting the map are described (X). For example, letters indicating road names are often shown overlapping other roads.

  The pattern candidate portion specifying processing unit 22 binarizes this (FIG. 5B), extracts connected pixels, and performs a small area removal process based on the feature amount for each connected pixel. Then, the pattern area specifying image data is in a state where the text portion (T) is removed as shown in FIG. In FIG. 5, since the black pixel density of the diagram portion G is relatively high, the diagram portion G remains without being removed. In addition, the characters that overlap the line segment in the map portion M are extracted as connected pixels together with the overlapping line segment. For this reason, the area of the rectangle surrounding the connected pixel is increased, and is not removed as a small region, but remains (X in FIG. 5C).

  The pattern candidate portion specifying processing unit 22 fills the white pixel portion B, which is the background, with a predetermined color (FIG. 5D), and further black pixels other than the portion painted with the predetermined color. And As a method of painting the background part here, use the general paint processing that paints the area that is not partitioned by line segments from the four corners of the image data (upper left, upper right, lower left, lower right corners). Can be taken. Then, the pattern candidate part specifying unit 22 returns the predetermined color part to a white pixel. Then, as shown in FIG. 5 (e), pattern area specifying image data is obtained in which the pattern candidate part is a black pixel and the other part is a white pixel.

  According to this process, the pattern part excluding the background part can be specified by a low-load process using the binarized data.

[2b. Second method]
Next, another processing method of the pattern candidate portion specifying processing unit 22 will be described. As shown in FIG. 3 (b), the processing by the pattern candidate portion specifying processing unit 22 is functionally a binarization processing unit 31, a connected pixel extraction unit 32, a feature amount calculation unit 33, and an attribute determination unit. 34, a non-pattern region processing unit 35, an expansion / contraction unit 38, a background region painting unit 36, and a pattern candidate region creation unit 37. Note that portions that perform the same operations as those already described are denoted by the same reference numerals, and detailed description thereof is omitted.

  If the outline is explained in advance, the pattern candidate part specifying processing unit 22 that performs this processing method includes the black pixels included in the connected area determined as the non-pattern area in the pattern area specifying image data by the non-pattern area processing unit 35. Expansion / contraction processing is performed on the image area specifying image data after the image is converted into white pixels.

  The expansion / contraction unit 38 is included in the pattern area specifying image data after the black pixels included in the connected area determined as the non-pattern area in the pattern area specifying image data by the non-pattern area processing unit 35 are converted into white pixels. The selected pixels are sequentially selected as the target pixel.

  Then, the expansion / contraction unit 38 performs processing (expansion) when the pixel of interest is a black pixel if there is at least one black pixel in the vicinity of the pixel of interest (four neighborhoods of upper, lower, left, and right pixels or eight neighborhoods of the surrounding eight pixels). Process) to expand the black pixel portion and select the next pixel of interest. The selection order may be the order in which lines are scanned from top to bottom and the lines are scanned from left to right (so-called raster scan order).

  When the expansion / contraction unit 38 completes the expansion processing for all the pixels, the expansion / contraction unit 38 again starts the contraction processing while sequentially selecting each pixel as the target pixel. In the contraction process, if there is at least one white pixel in the vicinity of the pixel of interest (near 4 or 8), the pixel of interest is set as a white pixel. Then, when the contraction process is completed for all the pixels, the process of the expansion / contraction unit 38 ends.

  In this case, the background area filling unit 36 sets the white pixels in the background part connected to the boundary of the pattern part to a predetermined value (with respect to the pattern area specifying image data processed by the expansion / contraction part 38). Any color other than white or black) will be painted.

  By performing the expansion / contraction process in this manner, the halftone dot region is easily distinguished from the background region and recognized as a pattern region.

  Further, the expansion / contraction unit 38 has the pattern area specifying image data obtained by converting the black pixels included in the connected area determined to be the non-pattern area in the pattern area specifying image data by the non-pattern area processing unit 35 into white pixels. May be converted into run-length data and then the above process may be performed, and after the process, the data may be returned to the bitmap data and output.

  Here, the run-length data is converted into data consisting of one piece of information for specifying the pixel value and the number of consecutive times instead of the series of pixel values when the same pixel value appears successively in the raster scan order. Is.

  In the description of the first and second methods thus far, the background region painting unit 36 paints the background portion with a predetermined color, but instead of this, the following may be used. That is, when a labeling process is used to extract a connected region, a label (background identification) that is larger than the maximum label value obtained at the time of extraction (for example, a value obtained by adding 1 to the maximum label value). (Label) may be set by replacing the label of the pixel in the background portion. In this case, the pattern candidate area creation unit 37 may generate the pattern area specific image data shown in FIG. 5E by setting the pixels not attached with the background specific label to black pixels.

  According to this process, the pattern part excluding the background part can be specified by a low-load process using the binarized data. In addition, by performing the expansion / contraction process, it is possible to specify a portion constituted by halftone dots as a design.

[2c. Third method]
Furthermore, the process by the pattern candidate portion specifying processing unit 22 may determine whether or not each labeled area touches the boundary after the labeling process, thereby determining whether or not the pattern is a pattern. Specifically, as shown in FIG. 3C, the processing example in this case is functionally a binarization processing unit 31, a connected pixel extraction unit 32, a feature amount calculation unit 33, and an attribute determination unit 34. And a non-design region processing unit 35, an inversion labeling processing unit 41, a boundary determination unit 42, a region removal unit 43, and a design candidate region creation unit 44. The binarization processing unit 31, the connected pixel extraction unit 32, the feature amount calculation unit 33, the attribute determination unit 34, and the non-picture area processing unit 35 perform the same processing as already described. Detailed description thereof is omitted.

  The inversion labeling processing unit 41 performs labeling processing (different for each connected region) on the white pixel portion of the data output from the non-pattern region processing unit 35 (pattern region specific image data in which the non-pattern region and the background are white pixels). Process to give a label). Specifically, if the data output by the non-picture area processing unit 35 is as shown in FIG. 6A, the result of the labeling process is as shown in FIG. 6B. In FIG. 6B, a portion to which a label is attached is shown in black, and a portion to which no label is attached is shown in white. In FIG. 6B, the label value is indicated by a circled number. Note that for the upper left snowman, the upper right bus destination display portion, and the two headlight portions, the label values are shown outside the box for the sake of illustration.

  The boundary determination unit 42 checks whether or not each connected region (or its circumscribed rectangle) to which each label is attached is in contact with the boundary (end) of the image data, and generates a list of labels in contact. Specifically, in the case of the example shown in FIG. 6B, since only the connected region of the label “1” is not in contact with the boundary of the image data, only the label “1” is included in the list.

  The area removing unit 43 removes the labels for the other connected areas while leaving the connected areas of the labels included in the list. As a result, as shown in FIG. 6C, the area other than the label “1” is not given a label. In FIG. 6C as well, the part with the label is shown in black, and the part without the label is shown in white.

  The picture candidate area creation unit 44 generates and outputs image data in which the connected area to which the label is attached is a black pixel and the connected area to which the label is attached is a white pixel as image data of the candidate pattern area. Then, as shown in FIG. 6D, pattern area specifying image data is obtained in which the part that is a pattern candidate is a black pixel and the other part is a white pixel.

  In the present embodiment, even if the character part is erroneously determined as a pattern in the process of specifying the pattern area candidate, the character part is further extracted by the layout process performed later, so that the extraction accuracy of the pattern candidate area is relatively high. It can be low. The control unit 11 stores coordinate information (hereinafter referred to as “design candidate area definition information”) that defines each of the identified design candidate areas in the storage unit 12.

  According to this process, the pattern candidate part excluding the background can be specified by a simple process. Also, in this process, so-called hollow portions (in the example of FIG. 6 such as the inside of a snowman, the windshield of the bus, the headlamp portion, etc.) surrounded by the pattern portion are used as the pattern candidate portion without being the background. Will be included.

[2d. Other examples of non-picture area processing]
In addition, instead of the processing of the binarization processing unit 31, the connected pixel extraction unit 32, the feature amount calculation unit 33, the attribute determination unit 34, and the non-picture area processing unit 35 in the above description, the processing shown in FIG. May be used. That is, as shown in FIG. 7, this processing by the pattern candidate portion specifying processing unit 22 includes a binarization processing unit 71, a labeling processing unit 72, a size information calculation unit 73, and a significant pixel distance information calculation unit 74. And a determination unit 75 functionally.

  Here, the binarization processing unit 71 refers to the pixel values of a part (for example, the four corners) of the processing target image data output from the preprocessing unit 21, and sets the binarization threshold value based on the average value of the luminance. Determine. Then, the processing target image data is copied on the storage unit 12, and for each pixel of the copied image data, “white (background pixel)” is set for a pixel whose luminance is higher than the predetermined binarization threshold, and the luminance is low. The pixel value of “black (significant pixel)” is assigned to the pixel to generate binarized image data. The condition that the luminance is higher than the binarization threshold here corresponds to the background pixel condition of the present invention.

  The labeling processing unit 72 extracts, from the binarized image data, a black pixel connection portion, which is partitioned from each other by an edge of the image data or a background pixel, as a connection pixel region, and is unique to each connection pixel region. Set the identifier (label). Specifically, a data area for storing the same number of identifiers as the number of pixels in the binarized image data is secured in the storage unit 12, and the significant pixel is located at a position corresponding to each significant pixel in the data area. Stores the label of the connected pixel region to which it belongs.

  That is, the labeling processing unit 72 performs a so-called labeling process, specifies a pixel group (connected pixel region) in which adjacent significant pixels are connected to form one lump, and the connected pixel region A label is set for each significant pixel included.

  For each label given by the labeling processing unit 72, the size information calculation unit 73 selects a connected pixel region corresponding to the label as the attention region, and calculates information (size value) related to the size of the attention region. Then, it is stored in the storage unit 12 as size relation information in association with the label.

  Specifically, the information related to the size here is information regarding the number of significant pixels in the attention area and the size of a predetermined geometric shape related to the attention area (for example, coordinate information defining a rectangle circumscribing the attention area) The width, height, area, etc. of the rectangle obtained from

  The significant pixel distance information calculation unit 74 generates, for each significant pixel in the binary image data, the distance from the significant pixel to the adjacent significant pixel to the background pixel as significant pixel distance information. There are various methods for calculating the significant pixel distance information. One example will be described below.

  The significant pixel distance information calculation unit 74 secures an area for storing bitmap data having the same size as the binary image data in the storage unit 12, and a pixel at coordinates (i, j) on the binary image data. Is directly obtained as f0 (i, j). And the data of the position corresponding to the said coordinate (i, j) on the bitmap data ensured in the memory | storage part 12 are set to f0 (i, j). Thereby, the binary image data is first copied as it is onto the bitmap data.

  Next, in the bitmap data, the data at the position corresponding to the pixel at the upper left corner, that is, the data at the position (0, 0) is sequentially scanned in the raster scan order, and the position ( Update the data in i, j) to f1 (i, j).

  When the update processing for all the pixels is completed, the significant pixel distance information calculation unit 74 further reverses the raster scan (in the lower right) in order from the data at the position (0, 0) with respect to the updated bitmap data. The data (f1 (i, j)) at the position (i, j) is updated to f2 (i, j) by the following equation (3) while scanning from the pixel to the upper left pixel one line at a time) To do.

  As a result, the bitmap data shown in FIG. 8B is generated based on the binary image data shown in FIG. In equations (2) and (3), min [x, y,...] Means that the smallest value is selected from the values in parentheses (in []).

  The significant pixel distance information calculation unit 74 further selects each of the connected pixel regions specified by the labeling processing unit 72 as a region of interest one by one from the bitmap data corresponding to the pixels belonging to the region of interest, A storage unit is selected as the significant pixel distance information by selecting the maximum value and associating the label associated with the region of interest with the selected maximum value (so-called feature amount related to the width of the pixel block constituting the region of interest). 12.

  The determination unit 75 refers to the connected pixel specified by the labeling processing unit 72 while referring to the significant pixel distance information generated by the significant pixel distance information calculation unit 74 and the size relationship information generated by the size information calculation unit 73. It is classified by determining whether each of the areas belongs to a “character / line drawing” or “picture” category.

  Specifically, the determination unit 75 reads the size value associated with each label from the size relation information stored in the storage unit 12. And by the size value for each label, the feature quantity stored in the storage unit 12 as significant pixel distance information in association with the corresponding label is divided, and for each label, how much the feature quantity is relative to the size value It is determined whether the connected pixel region corresponding to the label belongs to the category “character / line drawing” or “picture”. In general, in the case of a character line drawing, the feature amount is smaller than the size value (configured by relatively thin lines). Therefore, when the feature amount is smaller than a predetermined threshold with respect to the size value, the determination unit 75 determines that the connected pixel region corresponding to the label is a character line drawing, and the feature amount is predetermined with respect to the size value. When the threshold value is larger than the threshold value, it is determined that the connected pixel region corresponding to the label is a picture.

  In addition, for each connected pixel area, the determination unit 75 determines whether the connected pixel area belongs to the “character / line image” or “picture” category only when the size value exceeds a predetermined threshold value. It may be determined.

  That is, the determination unit 75 refers to the size value associated with each label, and when the size value does not exceed a predetermined noise threshold, the connected pixel region corresponding to the label is set to “noise”. judge.

  When the size value exceeds a predetermined noise threshold value, the threshold value is determined by a predetermined function related to the size value, and the threshold value is compared with the feature amount associated with the label. To do. When the feature amount exceeds the threshold value, the connected pixel area corresponding to the label is determined as “picture”, and otherwise (when the feature amount does not exceed the threshold value). The connected pixel area corresponding to the label is determined as “character line drawing”.

  Here, the predetermined function may be a function that gradually approaches a predetermined logarithmic function from a predetermined minimum value to a predetermined limit value. In this way, a determination is made as shown in FIG. 9 according to the feature amount with respect to the size value.

  Then, the determination unit 75 stores the determination result for each label (for each connected pixel region) in the storage unit 12 and ends the process. In addition, about the label determined as noise, you may make it not store the determination result. The determination unit 75 generates and outputs image data in which the connected pixel region determined to be a pattern is black, and the other is a white pixel.

  This image data can be handled in the same way as the data output by the non-picture area processing unit 35 (the picture area specifying image data in which the non-picture area and the background are white pixels). According to the processing described here, a fine line portion or the like is not determined as a pattern, and the accuracy of extracting a pattern candidate portion can be improved.

  Note that the determination unit 75 may output information on the connected pixel region of the portion determined to be a character / line image to the character / line image extraction processing unit 23 described below.

[2e. Example of background removal processing]
The pattern candidate part specifying processing unit 22 may further perform the background removal process on the part other than the pattern candidate area in the original image data using the pattern candidate area defining information generated by these processes. This background removal processing is, for example, as follows.

  That is, first, a histogram of luminance (Y if YCbCr) is generated, and on the calculated histogram, a luminance value having a frequency exceeding a predetermined threshold is searched from the low luminance (dark) side. The luminance value is acquired and is set as THD1.

  Then, THD1 ′ = k · THD1 + c and THD2 = d · THD1 are generated using predetermined parameters k, c, and d. These parameters are set so that k × THD1 + c <d × THD1 for an arbitrary THD1.

  Next, for an input value from 0 to THD1 ′, the output value is 0, for an input value x such that x ≧ THD2, the output value is x, and for THD1 ′ ≦ x <THD2, the coordinates ( An LUT that generates an output value corresponding to a straight line connecting THD1 ′, 0) to (THD2, THD2) is generated.

  Then, luminance correction using the generated LUT is performed on the original image data other than the pattern candidate region, and the image data after the luminance correction is stored in the storage unit 12.

  That is, in the present embodiment, a pattern candidate region may be generated, and the background removal processing may be performed on a portion other than the pattern candidate region.

[3. Character line drawing extraction processing unit]
The character / line drawing extraction processing unit 23 extracts the character / line drawing part from the image data expressed in the YCbCr color space output from the preprocessing part 21 (or the image data after luminance correction output from the pattern candidate part specifying processing part 22). Perform the extraction process. For this processing, for example, processing as disclosed in Patent Document 2 can be used. Then, the character / line drawing extraction processing unit 23 generates a rectangle (character / line drawing circumscribed rectangle) surrounding each character or line drawing part, and stores the coordinate information in the storage unit 12.

  In addition, when the pattern candidate portion specifying processing unit 22 outputs information specifying a portion (concatenated pixel region) determined to be a character / line image, the character / line drawing extraction processing unit 23 uses the information to identify each portion. A rectangle (character / line drawing circumscribing rectangle) surrounding them may be generated for the characters and line drawing parts, and the coordinate information thereof may be stored in the storage unit 12.

[4. Layout processing section]
The layout processing unit 24 reads, from the storage unit 12, the pattern candidate area defining information generated by the pattern candidate portion specifying processing unit 22 and the coordinate information of the character / line drawing circumscribing rectangle generated by the character / line drawing extraction processing unit 23.

  The layout processing unit 24 performs layout using different determination conditions for each of the pattern candidate area defined by the pattern candidate area definition information and the character / line drawing area defined by the coordinate information of the circumscribed rectangle of the character / line drawing. Analyze.

  Specifically, in the present embodiment, layout analysis processing is performed in the pattern candidate area defined by the pattern candidate area definition information read from the storage unit 12, and further character portions are extracted from the pattern candidate area. Then, a portion excluding the extracted character portion is defined as a pattern region, and is used for subsequent processing.

  In the present embodiment, the layout processing unit 24 demarcates a character area step by step from a character to a line and from a line to a region, and determines whether a character string is included based on the state of the line in the delimited character area. If it is determined that the character string is not included, it is further determined whether the character is included based on the in-line (character unit) state. Then, the storage unit 12 stores information on a rectangle circumscribing a portion determined to be each character (basic rectangle information) and information on a rectangle circumscribing a row in which characters are arranged in one line (row rectangle information). Further, the layout processing unit 24 specifies a character area (character area coordinate information itself) or a black pixel portion in the character area (character area coordinate information and bitmap information including the black pixel portion) as a character portion. Then, information for specifying the character part (character part specifying information) is stored in the storage unit 12.

  Note that the layout processing in the present embodiment is not limited to this, and other well-known layout processing may be used.

  One characteristic of this embodiment is that a character portion is extracted using layout analysis in so-called T / I separation processing. As a result, the character portion included in the pattern candidate region is also extracted by the layout analysis process, and the accuracy of extracting the character portion can be improved.

  On the other hand, the layout processing unit 24 performs a layout analysis process also on a portion (character / line drawing circumscribed rectangle) defined as a character / line drawing. The layout processing unit 24 determines layout frames (including at least rectangles (basic rectangles) circumscribing each character) obtained as a result of the layout analysis processing, and stores information (coordinate information and the like) of these layout frames. 12.

  Note that a line rectangle obtained by relating the basic rectangle in the horizontal or vertical direction may also be defined within the character line drawing circumscribed rectangle, and the coordinate information of the line rectangle may be stored in the storage unit 12 together.

  The layout processing unit 24 of the control unit 11 refers to the character part specifying information stored in the storage unit 12 and the character / line drawing circumscribing rectangle (or the layout frame as a layout processing result thereof), and the image to be processed For the entire data, coordinate information of a rectangular area including characters in the image data is generated. Specifically, the layout processing unit 24 combines the rectangle defined by the coordinate information of the character part specifying information and the rectangle defined by the character / line drawing circumscribing rectangle information (or the layout frame as a result of the layout processing). A character line drawing area is generated. In other words, in the present embodiment, the pattern candidate area and the character / line drawing area are defined separately, and therefore the area specified as the character / line drawing area may exist in the pattern candidate area. For this reason, these areas are combined here to make the overlapping area one character / line drawing area.

  Then, the layout processing unit 24 generates a unique area identifier (hereinafter referred to as label data) for each combined character / line drawing part, and coordinates information (vertex) for defining the label data and the corresponding character / line drawing area. The information is stored in the storage unit 12 as a character / line drawing area database.

[5. Same color area separation unit]
The same color area separation unit 25 performs a process of dividing each of the character / line drawing areas stored in the character / line drawing area database of the storage unit 12 into an area including only the character / line drawing parts of the same color. The same color region separation unit 25 reads the coordinate information of the basic rectangle (rectangle defined from both the pattern candidate region and the character / line drawing region) stored in the storage unit 12 during the layout process. Then, a representative value (representative color) candidate is determined from pixel values included in each basic rectangle defined by the coordinate information in the image data (original image data) to be processed.

  As shown in FIG. 10A, the same color region separation unit 25 in the present embodiment is configured to include a representative color determination unit 51, a same color region information generation unit 52, and an inclusion image creation unit 53. ing. In the following description, the pixels in the basic rectangle may be all the pixels included in the basic rectangle on the original image data, or are determined as pixels constituting a character or a line drawing in the basic rectangle. It may be a pixel.

  The representative color determination unit 51 refers to the coordinate information of the basic rectangle stored in the storage unit 12 and selects at least one representative pixel based on the pixel value in the target basic rectangle while sequentially selecting them as the target basic rectangle. Determine the value. Here, the representative pixel value is determined by generating a histogram (occurrence frequency) of pixel values in the target basic rectangle in the original image data, and one pixel value appearing at the highest frequency in the histogram, or a predetermined pixel value. There is a method in which at least one pixel value that appears at a frequency exceeding a threshold value (for example, 1/3 of the number of pixels in the target basic rectangle) is used as a representative pixel value. The representative color determination unit 51 stores the representative pixel value determined in association with the information specifying the target basic rectangle (identifier issued uniquely to the target basic rectangle) in the storage unit 12 as a representative pixel value database.

  The same color area information generation unit 52 groups representative pixel values stored in the representative pixel value database for each color determined to be the same. Specifically, the same color area information generation unit 52 sets representative pixel values not yet grouped among representative pixel values stored in the representative pixel value database as target representative pixel values. Different representative pixel values that have not yet been grouped are sequentially selected as comparison pixel values. If there is a comparison pixel value that can be determined to be the same color as the target representative pixel value, the at least one comparison pixel value and the target representative pixel value are determined as one group. Then, the same color area information generation unit 52 extracts the basic rectangle identifiers associated with the representative pixel values in the group from the representative pixel value database, generates a list of basic rectangle identifiers, One group representative pixel value determined based on the comparison pixel value is determined. The same color area information generation unit 52 then issues a unique group identifier, stores this group identifier, the group representative pixel value, and the generated list of basic rectangle identifiers as the same color area information database. Stored in the unit 12. Note that the group representative pixel value determination method may be, for example, a pixel value of a statistical calculation result (for example, an average) between a target representative pixel value and a comparison pixel value determined to be the same.

  On the other hand, when there is no comparison pixel value that can be determined to be the same color as the representative pixel value of interest, a group including only the representative pixel value of interest is generated. That is, the same color area information generation unit 52 extracts the identifier of the basic rectangle associated with the target representative pixel value from the representative pixel value database, and determines the target representative pixel value as the group representative pixel value as it is. The same color area information generation unit 52 issues a unique group identifier, associates the group identifier, the group representative pixel value, and the generated basic rectangle identifier with each other in the storage unit 12 as the same color area information database. Store.

  In this way, the same color area information generation unit 52 defines, for each grouped representative pixel value (group representative pixel value), the representative pixel value (group representative pixel value) and a basic area corresponding to the representative pixel value. The same color area information including the information associated with (here, the identifier of the basic area) is generated and stored in the storage unit 12 as the same color area information database.

  The included image creation unit 53 retrieves the identifier of the basic area included in the representative pixel value database while referring to the representative pixel value database. Further, with reference to the basic rectangle database, coordinate information (information defining each basic area) associated with the identifier of each extracted basic area is acquired. Then, the included image creation unit 53 generates a region that includes these basic regions. Specifically, the inclusion image creation unit 53 includes the smallest X coordinate value Xmin, the Y coordinate value Ymin, the largest X coordinate value Xmax, and the Y coordinate among the X and Y coordinate values indicated by the acquired coordinate information. Search for the value Ymax. Then, information is generated that defines a rectangle having a diagonal line (Xmin, Ymin) and (Xmax, Ymax) (a rectangle circumscribing each included basic rectangle).

  The inclusion image creation unit 53 extracts the pixels in each included basic region from the original image data, and generates this based on the background pixel values determined from the pixel values at the four corners of the original image data, for example. Binarization is performed using a threshold value, and the binarized image data is generated as inclusion image data.

  This inclusion image data is bitmap information obtained by binarizing the value of each pixel in the basic region included in the inclusion image data. Therefore, the inclusion image data includes at least one representative. Information indicating the position of the pixel to be set in the pixel value (or group representative pixel value) is included. The inclusion image creation unit 53 may compress information indicating the position of the pixel, such as bitmap information obtained by binarization, by a predetermined method (for example, MMR, JBIG2, etc.).

  Here, only the basic rectangle, that is, the character is targeted, but the representative pixel value of the rectangular portion (that is, the line drawing portion) that is not defined as the basic rectangle in the layout processing unit 24 among the character line drawing circumscribed rectangle is also set. It is also possible to group the rectangles that are determined to be the same color together with the basic rectangle, include the basic rectangle and the line drawing portion, and binarize the pixels included in these to generate included image data.

  The control unit 11 includes data including in its structure the inclusion image data generated by these processes, the same color area information database, and coordinate information indicating the position of the inclusion image data on the original image data. Output as line drawing plane data. When reproducing the original image data, the character / line drawing plane data generated in this way is decompressed first to reproduce the bitmap of the included image data, and is included in the same color area information database. For each group, the pixel value of the black pixel (on pixel) on the reproduced bitmap corresponding to the basic rectangle in the group is specified, and the value of the specified pixel is assigned to the group of the group Set to representative pixel value.

  Further, as shown in FIG. 10B, the same color region separation unit 25 of the control unit 11 includes a representative color determination unit 51, an identical color region information generation unit 52, an included image creation unit 53, and an inclusion by color. The image creation unit 54 and the comparison unit 55 are included, and the representative color determination unit 51, the same color area information generation unit 52, and the inclusion image creation unit 53 perform the above-described operation and include the color-specific inclusion image creation unit. 54 generates inclusion image data for each group, and the comparison unit 55 includes character / line drawing plane data (first image data) including the inclusion image data (color inclusion regions) for each group, and the inclusion image data. And the character / line drawing plane data (second image data) including the same color area information database, and selects the smaller one of the first image data and the second image data, and selects the selected data. Character line drawing It may be output as the lane data.

  In addition, the same color area information generation unit 52 determines whether two representative pixel values can be determined to be the same as the sum of squares of differences of the two representative pixel values for each component (predetermined value between representative pixel values). If the amount relating to the distance in the color space is smaller than a predetermined threshold value, it may be determined that they are the same.

  Next, the operation of the same color region separation unit 25 of the present embodiment will be described with reference to FIG. Here, an image as shown in FIG. 11A will be specifically described as an example. In the example shown in FIG. 11A, in the character string “Yesterday was rainy” on the first line, the letters “Yesterday” were red characters and the characters “Has rainy” were represented by black characters. , The whole string of the next line “Today is cloudy” is represented in black. Further, an example is shown in which there is a character string “It was a story about the weather” at a position apart from the character strings of the two lines, and only the characters “story” are red characters. In FIG. 11A, for convenience of illustration, the red character portion is surrounded by a broken line. This enclosed broken line is not actually represented.

  In this case, a rectangle (basic rectangle) circumscribing the pixel block of each character is defined by the processing of the layout processing unit 24 and the like, and the representative color determining unit 51 determines a representative color for each basic rectangle. The representative color determined by the representative color determining unit 51 is set to the same color (pixel value) for each character, for example, two characters “Yesterday” and “Day” of “Yesterday”. However, the pixel values are not always the same, and representative pixel values that are relatively close to each other but different from each other may be determined. That is, the representative color determining unit 51 determines a representative pixel value close to red for the basic rectangle circumscribing the three characters “Yesterday”, “Day”, and “Story”, and each basic character circumscribing other characters. For the rectangle, a representative pixel value close to black is determined.

  The same color area information generation unit 52 defines basic rectangles associated with representative pixel values determined to be the same as a group. As a result, one group representative pixel value (also defined as a value close to red) is associated with the information defining the basic rectangle circumscribing the three characters “Yesterday”, “Day”, and “Story”. One group representative pixel value (also defined as a value close to black) is associated with information that defines a basic rectangle that is grouped and circumscribes other characters, and is grouped.

  The inclusion image creation unit 53 generates a rectangle that includes all of the target basic rectangles (regardless of the group), and converts the pixel value in each basic rectangle into binary image data (first image). 2 image data) (FIG. 11B).

  Then, the same color area information database including at least one set of information defining the basic rectangle and the group representative pixel value is associated with the included image data, and is output as character line drawing plane data.

  On the other hand, the color-specific inclusion image creation unit 54 generates inclusion image data (color-specific inclusion image data) for each group (FIG. 11C). Here, the pixel values in the color-by-color inclusion image data need not be binarized. In the case of binarization, the group representative pixel values of the group corresponding to the color-by-color inclusion image data are associated and included in the character / line drawing plane data. Then, the comparison unit 55 selectively outputs image data having a small data size among the character / line drawing plane data (first image data) generated by the color-specific inclusion image creation unit 54 and the second image data, The output result is stored in the storage unit 12 as character / line drawing plane data.

  Here, the coordinate information that defines the basic rectangle included in the same color region information database may represent a position on the original image data, or represents a position in the included image data (for example, Coordinate information representing a relative position from the lower right coordinates of the included image data).

  The same color area information database does not necessarily include information that defines all basic rectangles. For example, among the groups, the group representative pixel value of the group having the maximum number of basic rectangles (referred to as the maximum group) is associated with the included image data shown in FIG. May be removed from the same color area information database.

  In this case, the original image data generation side first reproduces the bitmap of the inclusive image data, and associates the pixel values of the black pixels (pixels that are turned on) with the inclusive image data. Set to the group representative pixel value (the group representative pixel value of the largest group), and then turn on for each group in the same color area information database in the area corresponding to the basic rectangle included in each group on the bitmap. For the pixels that are, the group representative pixel value of each group is reset.

  Here, the group representative pixel value of the maximum group is set, but the group representative pixel value closest to a predetermined color (for example, black) is associated with the inclusion image data shown in FIG. The group information related to the representative pixel value may be removed from the same color area information database.

  Further, the representative color determination unit 51 may determine the representative pixel value after performing the smoothing process in consideration of the case where the pixel values in the basic rectangle vary. Here, the smoothing process includes a process of sequentially specifying each pixel in each basic rectangle as a target pixel, and setting an average value of the value of the target pixel and the value of a pixel adjacent thereto as the value of the target pixel.

  Furthermore, at the time of this smoothing process, only the pixels constituting the character in the basic rectangle (for example, the portion that becomes a black pixel by the binarization process) may be selected as the target pixel. In the smoothing process, when calculating the average value, the average value may be calculated with reference to only the values of the pixels constituting the character. This prevents the representative color of the character from being influenced by the background color by referring to the pixel value of the portion other than the character.

  Here, the representative value determined after the smoothing process may be corrected. That is, the same color region separation unit 25 of the present embodiment corrects the luminance of the determined representative value candidate, and determines the corrected value as the representative value. Here, the luminance can be corrected using, for example, a tone curve (correction function) as shown in FIG. The tone curve shown in FIG. 12 is an output value (value after correction, that is, a value determined as a representative value) with respect to an input value (luminance of a representative value candidate before correction) from the minimum value MIN to the first threshold value TH1. Brightness) is the minimum value MIN, and the output value from the second threshold TH2 (where TH2> TH1) to the maximum value MAX is set to the maximum value MAX. The tone curve has a center value MID between the maximum value MAX and the minimum value MIN (for example, when the maximum value is “255” and the minimum value is “0”, the MID is “128”). The output value for this may be set to be substantially MID.

  That is, the same color region separation unit 25 corrects the representative value candidate (represented by YCbCr in the present embodiment) with the tone curve of FIG. Luminance (Y ′) is determined, and a value specified by this Y ′ and representative value candidate color difference components Cb and Cr is determined as a representative value.

  If the representative value candidate is expressed in a color space that does not include a luminance component, such as RGB, the above processing is performed after conversion to a value in a color space that includes a luminance component such as L * a * b * or YCbCr. Should be done.

  Further, although only the luminance is corrected here, the color difference component may also be corrected. Specifically, the same color region separation unit 25 performs correction to reduce the number of gradations regarding the luminance component value of the representative color candidate value when each color difference component of the representative value candidate satisfies a predetermined condition, The corrected value may be determined as the representative value.

  Specifically, as shown in FIG. 13, the color difference components (a *, b *) of the representative value candidates expressed in the color space of L * a * b are respectively calculated from the center values of the corresponding color difference component value ranges. Of the luminance component L expressed in 256 gradations, for example, in 4 gradations or 8th floor, when the condition of being within the predetermined range (inside the circle defined by THa and THb in FIG. 13) is satisfied. The tone is reduced to a predetermined gradation such as a tone. In this case, the value of the color difference component may be set to the center value. Here, the predetermined ranges THa and THb for each component may be the same value or different values.

  By this processing, when the character color is gray (including black), a representative value that reproduces the original color of the character color is set. For example, when the character color is black, the color difference component and the luminance component are originally “0”. However, depending on the characteristics of the scanner and the encoding format of the original image data (for example, JPEG), the color difference component may be “ In some cases, the luminance component is not “0” or the luminance component is not “0”. Therefore, by performing the processing relating to the color difference component shown here, the representative value can be set to the original black color.

  As described above, according to the present embodiment, the representative color candidate of the target image area is determined based on the pixel values in the basic rectangle or the row rectangle as the target image area defined in the image data to be processed. Then, the luminance is corrected to determine the representative color.

  Here, after performing the smoothing process, the correction process is performed to determine the representative value. However, after performing the above correction process for each pixel by reversing the process order, A representative value may be determined by performing a smoothing process and calculating a histogram. Although the case of characters has been described here, the same processing is performed for line drawings.

  With this smoothing and correction process, in this embodiment, even if there is a variation in the pixel values that make up a character or line drawing, the influence of the variation is reduced and there is no sense of incongruity with the color of the original image data. A representative color can be determined.

[6. Cavity processing section]
The hole-filling processing unit 26 extracts a region corresponding to the pattern candidate region from the original image data, and detects a pixel of a character (a pixel constituting the character) detected by the processing of the layout processing unit 24 from this region. The pattern partial image data is generated by removing. At this time, for example, the background portion may be included in the pattern portion image data.

  Then, each pixel of the pattern partial image data is scanned in the raster scan order, and if the pixel of interest selected by the scanning is not removed, the pixel value of the pixel of interest remains as it is and the pixel of the pixel of interest The value is stored in the work memory of the storage unit 12 as the previous pixel value. If another pixel value is already stored as the previous pixel value, the stored content is overwritten.

  If the pixel of interest selected by scanning is a removed pixel, the pixel value of the pixel of interest is set to the immediately preceding pixel value stored. As a result, the pixel value of the removed portion becomes the same as the previous pixel value in the raster scan order, and the compression efficiency can be improved in many compression processes.

  Then, the pattern partial image data after this processing is performed is stored in the storage unit 12 as pattern plane data.

[7. Compression processing unit]
The compression processing unit 27 performs JPEG compression on the pattern plane data stored in the storage unit 12 to generate compressed pattern plane data. In addition, the compression processing unit 27 concatenates the character / line drawing plane data stored in the storage unit 12 and the compressed picture plane data to generate a series of data.

  Specifically, the series of data can be PDF (Portable Document Format) data. That is, an instruction to generate a bitmap generated by decompressing the compressed picture plane data (a bitmap of the picture plane data), and each character or line drawing included in the character / line drawing plane data is drawn on the bitmap of the picture plane data. PDF data including an instruction to be executed.

  Here, the instruction to draw each character or line drawing included in the character line drawing plane data is to read out the character line drawing compressed data, the representative color information related to this and the coordinate information such as the basic rectangle one by one, For the set of the above, the character line drawing compressed data is expanded to generate a binarized image, the color of the black pixel of the binarized image is set as the representative color, and the basic rectangle etc. on the bitmap of the picture plane data This is an instruction for transparent composition at the position set as the coordinate information. Here, “transparent composition” means that pixels other than black pixels (pixels set as representative colors) in the binarized image are not overwritten, but only pixels set as representative colors are overwritten.

  The compression processing unit 27 stores the generated PDF data in the storage unit 12 or outputs the PDF data to the image output unit 14 to send it to an external device.

[Operation]
Since the image processing apparatus according to the present embodiment has the above-described configuration, it operates as follows. Here, a document including a character part (T1, T2) as shown in FIG. 14A, a photograph part (P), and a map part (M) as a line drawing part is input from the image input unit 13, and this A case where image data of a document is a processing target will be described as an example. In the example of FIG. 14A, a part (T2) of the character part is superimposed on the photograph part (P). The map portion (M) includes a road map and characters. Here, although shown in black and white for the sake of convenience, the road map and characters in the map portion are actually represented in different colors, and the photograph may be in color.

  The preprocessing unit 21 converts the pixel value of the image data into a value in a predetermined color space (YCbCr). The pattern candidate portion specifying processing unit 22 binarizes the image data (original image data), and generates image data from which the character region (T1) and the diagram (M) are removed by removing the small area. (FIG. 14B). Further, the pattern candidate portion specifying processing unit 22 specifies and removes a background region (a portion adjacent to the boundary of the image data) from this image data by painting or the like, and specifies a pattern candidate portion. At this time, most of the characters and road map are removed (may be partially left), but the characters superimposed on the photograph portion are included in the pattern candidate portion as they are.

  The character / line drawing extraction processing unit 23 binarizes the original image data and extracts the character / line drawing part by a method such as specifying a small area part. At this time, the original image data is divided into a plurality of regions, and a binarization threshold value is adaptively determined for each region obtained by the division (a method disclosed in Patent Document 2). By performing binarization, characters can be extracted from a colored portion such as a map (FIG. 14C).

  The layout processing unit 24 performs a layout analysis process in the pattern candidate part, and extracts a character part (T2) remaining in the pattern candidate part. The same color region separation unit 25 determines a representative color for each character or line drawing portion of the character portion or line drawing portion extracted by the character line drawing extraction processing unit 23 or the layout processing unit 24, and is determined to be the same color. Group characters and line drawings related to representative colors. Then, image data including the character portion or the diagram portion extracted by the character / line drawing extraction processing unit 23 or the layout processing unit 24 is generated, and the image data is binarized (or converted to gray scale). It is sufficient if the size of the pixel value information can be reduced), and further compression processing is performed to include image data. As shown in FIG. 15, a group representative color for each group and a basic rectangle included in the group (which circumscribes each character) Character / line drawing plane data including coordinate information defining a rectangle), inclusion image data, and coordinate information specifying the position of the inclusion image data on the original image data is generated.

  The burial processing unit 26 generates image data obtained by removing the character portion (T2) inherent in the pattern candidate portion (FIG. 14D). Originally, the character portion T2 is outlined as in the character shape, but in FIG. 14D, the rectangular portion including the character shape is illustrated in white to make the drawing easier to see. FIGS. 14D and 14E show only the pattern image portion. Then, the hole filling unit 26 sets the value of the removed pixel to the nearest pixel value (the pixel value that has not been removed) in the scan line order (FIG. 14E), and generates the picture plane data. To do.

  The compression processing unit 27 performs JPEG compression on the pattern plane data, generates PDF data in combination with the character line drawing plane data, and outputs this to the image output unit 14. The image output unit 14 outputs the PDF data to an external device.

  Here, the compression processing unit 27 may perform a process of reducing the image size (reduction process) on the design plane data before JPEG compression to further improve the compression rate.

  Note that the background image area excluding the image part included in the pattern plane and the image part included in the character / line drawing data from the original image data is subjected to hole filling processing in the same manner as the pattern image plane, and a method such as run length is used. It may be compressed and included in the PDF data.

  On the side receiving this PDF data, the base image data is generated by expanding the run-length compression of the background image area, the JPEG compression of the pattern plane data is expanded and enlarged, and the base image data is opaquely synthesized. Further, for each image data obtained by expanding the character / line drawing plane data, image data in which a group representative color is set to a significant pixel of the included image data is generated and specified by the coordinate information associated with the included image data. The original image data may be reproduced by synthesizing the generated image data at a certain position.

1 is a configuration block diagram illustrating an example of an image processing apparatus according to an embodiment of the present invention. It is a functional block diagram showing the processing content performed by the control part of the image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram showing the example of the processing content of the pattern candidate part specific process part. FIG. 6 is a flowchart illustrating a processing example of an attribute determination unit 34. It is explanatory drawing showing the process example of the pattern candidate part specific process part. It is explanatory drawing showing another example of a process of the pattern candidate part specific process part. It is a functional block diagram showing the example of a part of process in the pattern candidate part specific process part 22. FIG. It is explanatory drawing showing the process example of a significant pixel distance information calculating part. It is explanatory drawing showing the process example of a significant pixel distance information calculating part. 6 is a functional block diagram illustrating an example of processing contents of the same color region separation unit 25. FIG. 11 is an explanatory diagram illustrating a processing example of the same color region separation unit 25. FIG. It is explanatory drawing showing the example of the tone curve utilized in the same color area separation part 25. FIG. FIG. 11 is an explanatory diagram illustrating processing conditions for correction processing in the same color region separation unit 25. It is explanatory drawing showing the process example of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing an example of character line drawing plane data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Control part, 12 Memory | storage part, 13 Image input part, 14 Image output part, 21 Pre-processing part, 22 Design candidate part specific processing part, 23 Character line drawing extraction processing part, 24 Layout processing part, 25 Same color area separation part, 26 burial processing unit, 27 compression processing unit, 31, 71 binarization processing unit, 32 connected pixel extraction unit, 33 feature amount calculation unit, 34 attribute determination unit, 35 non-pattern region processing unit, 36 background region filling unit 37, 44 Design candidate region creation unit, 38 Expansion / contraction unit, 41 Reverse labeling processing unit, 42 Boundary determination unit, 43 Region removal unit, 51 Representative color determination unit, 52 Same color region information generation unit, 53 Included image creation unit 54 color-specific inclusion image creation unit, 55 comparison unit, 72 labeling processing unit, 73 size information calculation unit, 74 significant pixel distance information calculation unit, 75 determination unit.

Claims (11)

Means for performing a labeling process on the image data to be processed, and extracting significant pixel parts connected to each other as connected pixels;
Means for removing a non-picture area from the connected pixels;
For the image data after removing the non-picture area, the background part is specified as the background part for the remaining connected pixel parts, and the specified background part is removed to specify the picture area A pattern area specifying means for generating information;
An image processing apparatus comprising: The image processing apparatus according to claim 1.
The image data after removing the non-picture area is subjected to expansion processing for setting pixels in the vicinity of significant pixels as significant pixels, and in the image data after the expansion processing, pixels in the vicinity of insignificant pixels are set as insignificant pixels. Further includes expansion and contraction means for performing the contraction process,
The pattern area specifying means, for the image data after the expansion / contraction processing, specifies a background part as a background with respect to the remaining connected pixel part, removes the specified background part, An image processing apparatus that generates information for specifying a picture area. The image processing apparatus according to claim 2,
The expansion / contraction means converts the image data after removing the non-pattern area into run-length data, and performs the expansion process on the converted run-length data.
An image processing apparatus, wherein the run-length data after the contraction processing is converted into bitmap data and output. Each pixel constituting the image data to be processed is classified into a background pixel that satisfies a predetermined background pixel condition and a significant pixel that does not satisfy the predetermined background pixel condition, and an end of the background pixel or image data Means for extracting a connection portion between the significant pixels, which are partitioned from each other by a unit, as a connection pixel region;
For each of the significant pixels, means for generating a distance from the significant pixel to the background pixel following the adjacent significant pixel as significant pixel distance information;
Means for generating size relationship information relating to the size of the selected region of interest;
Means for identifying a pattern area of the attention area based on the significant pixel distance information for each significant pixel included in the selected attention area, and the size relationship information;
An image processing apparatus comprising: Means for performing a labeling process on the image data to be processed, and extracting significant pixel portions connected to each other as first connected pixels;
Means for removing a non-picture area from the connected pixels;
Means for extracting, as the second connected pixel, a portion in which insignificant pixels are connected to each other for the image data after removing the non-picture area;
Means for specifying a second connected pixel in contact with a boundary of image data as a background portion among the second connected pixels, and specifying a region other than the specified background portion as a pattern region;
An image processing apparatus comprising: Using a computer,
Perform a labeling process on the image data to be processed, extract significant pixel parts connected to each other as connected pixels,
Removing a non-picture area from the connected pixels;
For the image data after removing the non-picture area, the background part is specified as the background part for the remaining connected pixel parts, and the specified background part is removed to specify the picture area An image processing method characterized by generating information. Using a computer,
Each pixel constituting the image data to be processed is classified into a background pixel that satisfies a predetermined background pixel condition and a significant pixel that does not satisfy the predetermined background pixel condition, and an end of the background pixel or image data A connected portion between the significant pixels partitioned by each other as a connected pixel region;
For each of the significant pixels, a distance from the significant pixel to the adjacent significant pixel to the background pixel is generated as significant pixel distance information.
Generating size relationship information related to the size of the selected region of interest;
An image processing method characterized in that a pattern area is specified among attention areas based on significant pixel distance information for each significant pixel included in the selected attention area and the size relation information. Using a computer,
A labeling process is performed on the image data to be processed, and significant pixel portions connected to each other are extracted as first connected pixels.
Removing a non-picture area from the connected pixels;
For the image data after removing the non-picture area, a portion where insignificant pixels are connected to each other is extracted as a second connected pixel,
An image processing method characterized in that, among the second connected pixels, a second connected pixel in contact with a boundary of image data is specified as a background part, and an area other than the specified background part is specified as a pattern area. On the computer,
A procedure for performing a labeling process on the image data to be processed and extracting significant pixel parts connected to each other as a connected pixel;
Removing a non-picture area from the connected pixels;
For the image data after removing the non-picture area, the background part is specified as the background part for the remaining connected pixel parts, and the specified background part is removed to specify the picture area A procedure for generating information,
An image processing program for executing On the computer,
Each pixel constituting the image data to be processed is classified into a background pixel that satisfies a predetermined background pixel condition and a significant pixel that does not satisfy the predetermined background pixel condition, and an end of the background pixel or image data A procedure for extracting a connection portion between the significant pixels separated from each other by a unit as a connection pixel region;
For each of the significant pixels, a procedure for generating the distance from the significant pixel to the background pixel by tracing the adjacent significant pixel as significant pixel distance information;
Generating size relationship information related to the size of the selected region of interest;
A procedure for identifying a picture area of the attention area based on the significant pixel distance information for each significant pixel included in the selected attention area, and the size relationship information;
An image processing program for executing On the computer,
A procedure of performing a labeling process on the image data to be processed, and extracting significant pixel portions connected to each other as first connected pixels;
Removing a non-picture area from the connected pixels;
A procedure for extracting a portion where insignificant pixels are connected to each other as the second connected pixel for the image data after removing the non-picture area;
Among the second connected pixels, a procedure for specifying a second connected pixel in contact with a boundary of image data as a background portion and specifying an area other than the specified background portion as a design area;
An image processing program for executing

Images