JP2006054712A - Image processing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To layer document data efficiently, and to obtain high-quality and high-compressibility image data. <P>SOLUTION: An image processing unit 10 executes various programs stored in a storage 2 by a control unit 1. The image processing unit 10 generates intermediate data from document data for generating multilayered compression image data based on the intermediate data by executing a printer driver program PRG3. The intermediate data are expressed as a set of edges for each object; and startpoint coordinates, endpoint coordinates, color information, and attribute information are described in each edge. The image processing unit 10 generates FG and BG images from the intermediate data based on the information, and executes edge padding processing of the BG image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique for creating layered image data based on drawing objects such as characters, graphics, and images constituting a document and performing image processing on each layer.

  Analyzes a document containing multiple drawing objects such as “natural images” such as photographs, “characters”, and “figures / line drawings”, and compresses each drawing object separately, resulting in high image quality but with a high compression ratio There is a technology for converting the image data into high image data. As an example of such a technique, there is a technique described in Patent Document 1 (hereinafter referred to as “conventional technique”). In the prior art, first, a binary image is generated from a document represented by a multi-valued image, and a character region is extracted using the binary image. Then, the character area of the multi-valued image is painted with the surrounding color of the character to obtain a background multi-valued image representing the background area. In this way, the character area and the background area are separated from the multi-valued image, the character area is compressed by the MMR (Modified Modified Read) method, and the background area is compressed by the JPEG (Joint Photographic Experts Group) method. In this compression process, mosquito noise generated at the boundary of the character area is suppressed, and image data with high image quality and high compression rate is obtained.

However, several problems are found in this prior art.
First, in the prior art, as described above, a character region is specified based on a binary image. For this reason, in order to obtain the background multi-value image, it is necessary to refer to both the binary image indicating the character region and the multi-value image indicating the character peripheral color, and there is a problem that the processing takes time.
Second, although the conventional technology is suitable for multilayering and compressing image data read by a scanner or the like, document data created by computer document creation software, or page description of this document. It is unsuitable for multilayering data described in a language or the like (hereinafter, these data are collectively referred to as “document data”). This is because, in the prior art, in order to make document data multi-layered, it is necessary to generate rasterized multi-valued images and binary images, which is wasteful in processing. . Further, since the rasterized image data has a data capacity much larger than that of the document data, there is a problem that a lot of physical resources such as a memory are required.

JP 2003-244447 A

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for efficiently multilayering document data and obtaining image data with high image quality and high compression rate.

In order to achieve the above-described object, the present invention represents a document, attribute information indicating the types of objects constituting the document, storage means for storing a correspondence relationship between each layer divided into a plurality from the upper level to the lower level, and the document A conversion unit that converts document data into image information; and the object included in the image information is assigned to a plurality of layers in accordance with the correspondence relationship stored in the storage unit, and layered image data representing an image of each layer is generated. A layered image generation unit that performs processing, and the layered image generation unit includes an upper layer or a lower layer of the specific layer when there is a missing area in which an object does not exist in the image of the specific layer to be processed In the above, a hole-filled image generated based on an object existing in an area corresponding to the position of the missing area and an object existing in the specific layer is Only formed in the region to provide an image processing apparatus for generating a layered image data of the particular layer.
According to such an image processing device, when there is a missing area in a specific layer, a hole-filled image is generated based on an object existing in an upper layer or a lower layer of the specific layer. Rate image data can be obtained.

In a more preferred aspect of the image processing apparatus of the present invention, the image information is represented by a set of edge lines obtained by subdividing the image in a certain direction, and the edge lines are the objects included in the edge lines. Is represented by a set of edge information including at least one of the start point coordinates, end point coordinates, color information, or attribute information.
More preferably, the layered image generation means may assign each of the edge information to any one of the plurality of layers based on the correspondence relationship stored in the storage means.
In this way, the image data can be represented by a set of edge information, and the document data can be efficiently multi-layered and the missing area can be filled in a certain layer.

In a more preferred aspect of the image processing apparatus according to the present invention, the layered image generation means may be configured such that when an object exists in an area corresponding to the position of the missing area in the upper layer, the layer in the specific layer The hole-filling image is formed in the missing area based on the object in the area adjacent to the missing area.
Alternatively, the layered image generation means does not form the hole-filled image when an object exists in an area corresponding to the position of the missing area in the lower layer.
In this way, it is possible to appropriately form a hole filling image.

At this time, the layered image generation means can form the hole-filled image based on the start point coordinates, the end point coordinates, and the color information of an object existing in an area adjacent to the missing area. is there.
Alternatively, the layered image generation unit forms the hole-filled image based on the start point coordinates, the end point coordinates, the color information, and the attribute information of an object existing in an area adjacent to the missing area. It is possible.

In a more preferred aspect of the image processing apparatus of the present invention, the layered image generation means is such that color information of objects existing in a region adjacent to the missing region has a constant color value. In this case, the hole-filling image having the color value is formed.
Alternatively, the layered image generation means may provide the hole-filled image having an average value of the color values of the plurality of objects when objects existing in the plurality of regions adjacent to the missing region have color information of different color values. Form.
In this way, it is possible to form a hole-filled image without a sense of incongruity between the area adjacent to the missing area.

Further, in a more preferred aspect of the image processing apparatus of the present invention, the layered image generation means has color information of different color values for objects existing in a plurality of areas adjacent to the missing area, and When both of these objects have attribute information of an object having a drawing effect that changes the color value in stages, the above-described hole-filling image having a color value that changes in stages is formed.
More preferably, when the layered image generating means forms the hole-filled image having a color value that changes in a stepwise manner in the missing area, the layered image generating unit is configured to change the first number of objects existing in the first area adjacent to the missing area. The hole-filling image having a color value that changes stepwise from a color value of 1 to a second color value of an object existing in a second area adjacent to the missing area may be formed.
In this way, even if the area adjacent to the missing area is a so-called gradation image, it is possible to form a hole-filled image without a sense of incongruity.

In a more preferred aspect of the image processing apparatus of the present invention, the layered image generation unit has attribute information of an object having a color value that periodically changes an object that exists in an area adjacent to the missing area. In some cases, the hole-filled image having a color value that changes periodically is formed.
In this way, it is possible to form a hole-filled image without a sense of incongruity even if the region adjacent to the missing region is a pattern image in which the color value changes periodically.

In a more preferred aspect of the image processing apparatus of the present invention, the layered image generation means includes an object in an area corresponding to the position of the missing area in the upper layer and is adjacent to the area. When an area where no object exists is included in a plurality of areas, a filling image is not formed near the boundary between the missing area and the area corresponding to the area where the object of the upper layer does not exist in the specific layer. .
In this way, it is possible to suppress the occurrence of noise when layered image data is compressed.

Further, the image processing apparatus according to the present invention, as a more preferable aspect, performs compression processing in which compression processing is performed on each layer image of the layered image data generated by the layered image generation unit using a compression method determined for each layer. Means.
This makes it possible to compress the layered image data and reduce the data capacity.

Further, the image processing apparatus according to the present invention may be configured such that, as a more preferable aspect, when the layered image generation unit forms the hole-filled image in the missing area of the specific layer, the compression processing performed on the image of the specific layer An image to be generated is determined by referring to the compression method.
In this way, it is possible to form a hole-filled image only in a layer to which a compression method suitable for forming a hole-filled image is applied.

Next, an embodiment of the present invention will be described.
(1) First Embodiment (1-1) Configuration FIG. 1 is a block diagram showing a hardware configuration of an image processing apparatus 10 according to a first embodiment of the present invention. The image processing apparatus 10 is a personal computer, for example, and includes a control unit 1, a storage unit 2, an input / output I / F 3, a display unit 4, and an operation unit 5. The control unit 1 includes an arithmetic device such as a CPU (Central Processing Unit) and various memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 2 is a large-capacity nonvolatile storage device such as a hard disk, for example, and stores an OS program PRG1 that is basic software. The storage unit 2 stores a document creation program PRG2, a printer driver program PRG3, and a tag correspondence table TBL. The OS program PRG1 describes a procedure for executing basic processing of each unit of the image processing apparatus 1. The document creation program PRG2 describes a procedure for creating document data in accordance with the operation contents by the operator. The printer driver program PRG3 describes a procedure for interpreting a printing drawing command and generating multi-layer compressed image data. The tag correspondence table TBL describes the association between tag information and layer images, details of which will be described later. The input / output I / F 3 is connected to an external device such as a computer or a printer, and exchanges data with the external device. The display unit 4 is a liquid crystal display, for example, and displays an image based on data supplied by the control unit 1. The operation unit 5 is, for example, a keyboard or a mouse, and supplies a signal corresponding to the operation content of the operator to the control unit 1.

FIG. 2 is a functional block diagram showing a functional configuration of the image processing apparatus 10. In this functional block diagram, the arrows in the figure represent the flow of data and various control information generated by processing executed by the image processing apparatus 10.
As shown in the figure, the image processing apparatus 10 realizes the functions of an OS 11, a document creation application 12, and a printer driver 13. These functions are realized by the control unit 1 executing a program stored in the storage unit 2 of the image processing apparatus 10.
With such a configuration, the image processing apparatus 10 outputs document data created by the operator as multilayer compressed image data. Specifically, when the operator gives a print instruction via the operation unit 5, the document creation application 12 interprets the document data, converts it into a drawing command defined by the OS 11, and supplies this to the OS 11. The OS 11 converts this drawing command into a printing drawing command interpretable by the printer driver 13, and supplies this to the printer driver 13. The printer driver 13 interprets the print drawing command and generates multilayer compressed image data. The multilayer compressed image data is output (printed) on recording paper by being supplied to an external device such as a printer.

Here, the multilayer compressed image data generated by the image processing apparatus 10 of the present embodiment will be described. Multi-layered compressed image data refers to classifying images into multiple layers of images (hereinafter referred to as “layer images”) based on the type of drawing object included in the document data. This is image data obtained by integrating the compression processes using different compression methods. In the present embodiment, the multilayer compressed image data includes an FG (Foreground) image and a BG (Background) image as layer images. When the multilayer compressed image data is drawn, the FG image is drawn in front of the BG image. That is, the BG image is masked by the FG image.
For convenience of explanation, in the following explanation, it is assumed that there is a “null image” in addition to the above-described FG image and BG image as a layer image. Here, the “null image” is defined as a layer image representing a blank area, that is, an area where a drawing object to be formed does not exist on the image. In other words, the “null image” is not a data that is actually generated, but is conceptual only.

Next, the function of the printer driver 13 will be described in more detail.
FIG. 3 is a functional block diagram showing the functional configuration of the printer driver 13. Also in this functional block diagram, the arrows in the figure represent the flow of data and various control information generated by the processing executed by the image processing apparatus 10.
As shown in the figure, the printer driver 13 includes an I / F unit 131, an intermediate data generation unit 132, an FG image generation unit 133, an FG image compression unit 134, a BG image generation unit 135, The functions of the BG image compression unit 136 and the format unit 137 are realized. The I / F unit 131 includes a function called from the OS 11 and receives a print drawing command from the OS 11. The intermediate data generation unit 132 interprets the print drawing command and converts it into intermediate data that is an expression format of document data in the printer driver 13. The intermediate data generation unit 132 supplies the intermediate data to the FG image generation unit 133 and the BG image generation unit 135 every time intermediate data corresponding to one page of document data is generated. The FG image generation unit 133 generates an FG image, and the FG image compression unit 134 compresses the FG image by a predetermined compression method (for example, MMR method) and supplies the compressed FG image to the format unit 137. Further, the BG image generation unit 135 generates a BG image, and the BG image compression unit 136 compresses the BG image by a predetermined compression method (for example, JPEG method) and supplies the compressed BG image to the format unit 137. The format unit 137 integrates these compressed layer images into a file and outputs the file.

Here, the intermediate data generated in the intermediate data generation unit 132 will be described.
FIG. 4 is a schematic diagram showing the structure of intermediate data. In the present embodiment, the intermediate data is composed of the same number of edge lines as the number of scanning lines n in the main scanning direction (here, x direction) when a document is rasterized, and each edge line includes a plurality of edge lines. It is image information represented as an edge row by edges. In this edge, information indicating what kind of drawing object exists in each edge line is described.

  FIG. 5 is a schematic diagram showing the structure of an edge. As shown in the figure, each edge includes a start point coordinate (Sx), an end point coordinate (Ex), color information (C), and tag information (T). Yes. The start point coordinate Sx is the x coordinate value of one end of the edge, and the end point coordinate Ex is the x coordinate value of the other end of the edge. The tag information is attribute information representing the type (attribute) of the drawing object constituted by the edges. In the present embodiment, when there are a plurality of drawing objects such as “characters” and “graphics” on one scanning line, one edge is generated each time the type of the drawing object changes. ing.

  In this embodiment, four types of “text (Tx)”, “graphics (Gr)”, “image (Im)”, and “null (Nl)” are defined as tag information. The “text” tag indicates that an edge having this attribute is a text object. Here, the text object is so-called text information such as characters and symbols. The “graphics” tag represents that an edge having this attribute is a graphics object. Here, the graphics object is a so-called vector image in which graphics and images are expressed in a vector format. The “image” tag represents that an edge having this attribute is an image object. Here, the image object is a so-called raster image in which figures and images are expressed in a bitmap format. The “null” tag indicates that an edge having this attribute is a null image, that is, an area where no drawing object is formed. The identification numbers “0”, “1”, “2”, and “3” are assigned to the tags “text”, “graphics”, “image”, and “null”, respectively. Inside the image processing apparatus 10, tag information is identified by this identification number.

The tag information is associated with any layer image of an FG image, a BG image, or a null image. This correspondence relationship is described in the tag correspondence table TBL. By referring to the tag correspondence table TBL, the image processing apparatus 10 can obtain the correspondence between the attributes of the respective drawing objects and the types of layer images.
FIG. 6 is a schematic diagram showing the tag correspondence table TBL in the present embodiment. As shown in the figure, each tag information is associated with any one layer image, and a drawing object indicated by each tag information is formed on the associated layer image. In the present embodiment, the “text” tag is associated with “FG image”, the “graphic” tag and the “image” tag are associated with “BG image”, and the “null” tag is associated with “null image”.

  In the color information C, the type of information described differs depending on the tag attribute of the edge. When the tag information is “text” or “graphics”, the color information C includes information on the color of the drawing object constituted by the edges, for example, red (R), green (G), blue (B) 8 It is described by RGB values of bits (0 to 255). When the tag information is “image”, the color information C describes the address of the memory area in which the raster image corresponding to the edge length is stored. When the tag information is “null”, the color information C describes an RGB value of (R, G, B) = (255, 255, 255), that is, an RGB value representing “white”.

(1-2) Operation Based on the above-described configuration, the control unit 1 of the image processing apparatus 10 realizes the functions shown in FIGS. 2 and 3 and processes data in the flow shown in these drawings. . The image processing apparatus 10 of this embodiment is characterized by the functions of the FG image generation unit 133 and the BG image generation unit 135. Therefore, specific operations in the FG image generation unit 133 and the BG image generation unit 135 will be described in detail below. Then, after describing the operation in each unit, a specific description will be given by exemplifying certain document data.
Note that the functions of the FG image generation unit 133 and the BG image generation unit 135 described below are realized when the control unit 1 of the image processing apparatus 10 executes the printer driver program PRG3. Therefore, the main operation of the FG image generation unit 133 and the BG image generation unit 135 is primarily the control unit 1. However, for convenience of explanation here, for example, “FG image generation unit 133 is not processed in the scanned edge line. The description will be made with the FG image generation unit 133 or the BG image generation unit 135 as the subject of operation, such as “determine whether an edge exists”.

(1-2-1) Operation of FG Image Generation Unit 133 FIG. 7 is a flowchart showing an operation of the FG image generation unit 133 generating an FG image from intermediate data. This operation is started when intermediate data for one page of document data is supplied from the intermediate data generation unit 132 to the FG image generation unit 133. First, when intermediate data is supplied to the FG image generation unit 133, the FG image generation unit 133 scans an edge line corresponding to the upper end portion of the page (step S11). Next, the FG image generation unit 133 determines whether there is an unprocessed edge in the scanned edge line (step S12). This “unprocessed” means that raster development processing (step S15) described later has not been performed. Here, when an unprocessed edge exists (step S12; YES), the FG image generation unit 133 extracts tag information from the unprocessed edge (step S13). Then, the FG image generation unit 133 refers to the extracted tag information and determines whether or not this edge is an FG image (step S14).

  The tag information inside the edge only describes the type of drawing object at that edge. Therefore, the FG image generation unit 133 cannot determine whether or not the image is an FG image only by referring to the tag information. Therefore, at this time, the FG image generation unit 133 refers to the tag correspondence table TBL at the same time. As shown in FIG. 6, the tag correspondence table TBL describes the correspondence between tag information and layer images. In this embodiment, since only the “text” tag is associated with the FG image, the FG image generation unit 133 may determine whether the tag information is “text” at this time.

  If the FG image generation unit 133 determines that the unprocessed edge is an FG image, in other words, the tag information of the unprocessed edge is a “text” tag (step S14; YES), the FG image generation is performed. The unit 133 rasterizes the edge data as an FG image (step S15), and executes the processing from step S12 again. On the other hand, when the FG image generation unit 133 determines that the unprocessed edge is not an FG image, in other words, the tag information of the unprocessed edge is “graphic”, “image”, or “null” ( Step S14; NO), the FG image generation unit 133 performs the processing from Step S12 again without performing the processing of Step S15.

  When the FG image generation unit 133 determines that there is no unprocessed edge in the scanned edge line in the determination in step S12 described above (step S12; NO), the FG image generation unit 133 determines this edge. The processing in the line is terminated, and it is determined whether there is an unprocessed edge line to be processed next (step S16). If the FG image generation unit 133 determines that there is an edge line to be processed next (step S16; YES), the processing target is moved to the edge line and the processing from step S11 is repeated (step S17). If it is determined that there is no edge line to be processed next (step S16; NO), this operation is terminated.

(1-2-2) Operation of BG Image Generation Unit 135 FIG. 8 is a flowchart showing an operation of the BG image generation unit 135 generating a BG image from intermediate data. This operation is started when intermediate data for one page of document data is supplied from the intermediate data generation unit 132 to the BG image generation unit 135. First, when intermediate data is supplied, the BG image generating unit 135 scans an edge line corresponding to the upper end portion of the page (step S21). Next, the BG image generation unit 135 determines whether there is an unprocessed edge in the scanned edge line (step S22). This “unprocessed” means that raster development processing (step S25) described later has not been performed. If there is an unprocessed edge (step S22; YES), the BG image generation unit 135 extracts tag information from the unprocessed edge (step S23). Then, the BG image generation unit 135 determines whether this edge is a BG image with reference to the extracted tag information (step S24).

  Here, in the same manner as in step S14 described above, the BG image generation unit 135 determines whether or not the unprocessed edge is a BG image by referring to the tag correspondence table TBL. Specifically, the BG image generation unit 135 determines that the unprocessed edge is a BG image, in other words, the tag information of the unprocessed edge is “graphic”, “image”, or “null”. If so (step S24; YES), the BG image generation unit 135 rasterizes the edge data as a BG image (step S25), and executes the processing from step S22 again. On the other hand, when the unprocessed edge is not a BG image, in other words, when the BG image generation unit 135 determines that the tag information of the unprocessed edge is a “text” tag (step S24; NO), the BG image generation unit 135 Executes the edge filling process (step S26). This edge filling process will be described after the description of this operation.

  When the BG image generation unit 135 determines that there is no unprocessed edge in the scanned edge line in the determination in step S22 described above (step S22; NO), the BG image generation unit 135 determines that this edge The processing in the line is terminated, and it is determined whether there is an unprocessed edge line to be processed next (step S27). If it is determined that there is an edge line to be processed next (step S27; YES), the BG image generation unit 135 moves the processing target to the edge line and repeats the processing from step S21 (step S28). If it is determined that there is no edge line to be processed next (step S27; NO), this operation is terminated.

  Here, the edge hole filling process will be described. Edge filling processing is a drawing object (here referred to as FG object) that becomes an FG image in document data is adjacent to or surrounded by a drawing object (here called BG object) that becomes a BG image. In such a case, a region substantially corresponding to the FG object in the BG image of the intermediate data is painted based on the color information of the neighboring BG object.

  FIG. 9 is a diagram of certain document data DOC1 exemplified for explaining the edge filling process. In the figure, a hatched area a1 represents a graphics object, and an A-shaped area a2 surrounded by the graphics object represents a text object. The other area a3 of the document data DOC1 is a null object without a drawing object. Here, it is assumed that the area a1 is red, that is, is uniformly filled with RGB values of (R, G, B) = (255, 0, 0).

  When such document data DOC1 is converted into multi-layer compressed image data by the image processing apparatus 10 of this embodiment, the area a1 is classified as a BG image and the area a2 is classified as an FG image. The BG image at this time is an image in which no drawing object exists in an area corresponding to the area a2 of the document data DOC1, as shown in FIG. When such a BG image is subjected to compression processing using discrete cosine transform (DCT) such as JPEG, image quality deterioration called mosquito noise occurs as the compression rate increases. Since mosquito noise is caused by an error of a high frequency component in DCT, it occurs in a portion where a color change suddenly occurs like a contour portion of a character. That is, when compression processing such as JPEG is performed on the BG image, mosquito noise is likely to occur at the boundary of the A-shaped region corresponding to the region a2.

As described above, since the BG image is masked by the FG image, in the BG image, no matter what processing is performed on the region where the drawing object exists in the FG image, the influence does not reach the multilayer compressed image data. . Therefore, if the process of filling the A-shaped area of the BG image with the same color as that of the area a1, the color change does not occur in the A-shaped area of the BG image, so that the mosquito noise is not affected without affecting the multilayer compressed image data. Can be suppressed. Further, by performing such processing, it is possible to compress the BG image at a higher compression rate.
The process of “filling the area masked by the FG image in the BG image” is the basic principle of the edge filling process of this embodiment.

Next, a specific operation in the edge hole filling process in step S26 will be described.
FIG. 11 is a flowchart showing edge filling processing in the present embodiment. Describing along the drawing, first, the BG image generation unit 135 determines whether or not the processing target edge is an edge of an FG image (step S261). This is because, as can be seen from the example of FIG. 9, if the FG image is not masked in the region to be subjected to the hole filling process, the hole filling process cannot be performed in the first place. In step S24, it is determined whether or not the processing target edge is a BG image, and when the determination result is negative, the determination in step S261 is performed. Either an FG image or a null image. If the edge to be processed is an edge of a null image, the BG image generation unit 135 returns to the operation of the flowchart of FIG. 8 without performing anything.

  On the other hand, when this determination result is affirmative, that is, when the edge to be processed is an edge of an FG image (step S261; YES), the BG image generation unit 135 has already performed this edge filling process on this edge. It is determined whether or not it has been performed (step S262). In this processing, when the edges of the FG image are continuous, the processing can be performed collectively on these edges, and thus such a step is added. Then, if this edge filling process has already been performed on the edge to be processed (step S262; YES), this process need not be performed, and the BG image generation unit 135 does not perform anything and performs FIG. Return to the operation of the flowchart.

On the other hand, if the determination result is affirmative (step S262; YES), the BG image generation unit 135 identifies the edge arrangement pattern by referring to the edges at both ends of the processing target edge (step S262). S263).
Here, the edge arrangement pattern refers to a group of edges of an FG image continuous to the edge of the FG image to be processed as one set (hereinafter also referred to as “FG edge group”), and the FG edge group. Represents the types of edges adjacent to the left and right of the. FIG. 12 is a diagram showing all edge arrangement patterns in the present embodiment. FIG. 12 (1) shows a case where both ends of the FG edge group are edges of the BG image, and this is hereinafter referred to as “BG-FG-BG pattern”. FIG. 12 (2) shows a case where the left end of the FG edge group is the edge of the BG image and the right end is the edge of the null image, which is hereinafter referred to as “BG-FG-Nl pattern”. FIG. 12 (3) shows a case where the left end of the FG edge group is the edge of the null image and the right end is the edge of the BG image, and this is hereinafter referred to as an “Nl-FG-BG pattern”. FIG. 12 (4) shows a case where both ends of the FG edge group are edges of a null image, which is hereinafter referred to as “Nl-FG-Nl pattern”. Note that there may be a case where the edge itself does not exist at one or both ends of the FG edge group, but in this case, it is considered that the edge of the null image exists on the side where the edge does not exist.

  When the edge arrangement pattern is identified in this way, the BG image generation unit 135 determines which of these patterns the FG edge group to be processed has. First, the BG image generation unit 135 determines whether or not the edge arrangement pattern is a “Nl-FG-Nl pattern” (step S264). Here, the “Nl-FG-Nl pattern” means that both ends of the FG image formed by the FG edge group are not adjacent to the BG image. In such a case (step S264; YES), since the hole filling process cannot be performed in the first place, the BG image generation unit 135 returns to the operation of the flowchart of FIG. 8 without performing the subsequent process.

  If it is determined that the processing target FG edge group is not the “N1-FG-Nl pattern” array pattern (step S264; NO), then the BG image generation unit 135 determines that the FG edge group is the “BG-FG-BG pattern”. Is determined (step S265). This determination is negative when the arrangement pattern is “BG-FG-Nl pattern” or “Nl-FG-BG pattern”. That is, the determination in step S265 can be rephrased as “whether both ends of the FG edge group are both edges of the BG image”.

If it is determined that both ends of the FG edge group are edges of the BG image (step S265; YES), the BG image generation unit 135 is based on information included in the edges of the BG image that are in contact with both ends of the FG edge group. Then, the area corresponding to the FG edge group is raster-developed as a BG image, thereby “filling” the area (step S266). Specifically, the BG image generation unit 135 performs raster development with reference to the start point coordinates (Sx), end point coordinates (Ex), and color information (C) of the BG images at both ends of the FG edge group. Here, the end point coordinates and color information of the part of the BG image in contact with the left end of the FG edge group to be processed are respectively Ex _l and C _l, and the BG image of the part in contact with the right end of the FG edge group to be processed When the start point coordinates and color information of Sx _r and C _r are respectively, raster development uses “Ex _l +1” as the start point coordinates and “Sx _r +1” as the end point coordinates, and the region in this range is defined as C _l and C _r. This is a process of drawing with the color indicated by the average value of. The drawn colors at this time are (R ₁ + R _r / 2) where the RGB values of C ₁ and C _r are (R ₁ , G ₁ , B ₁ ) and (R _r , G _r , B _r ), respectively. , G ₁ + G _r / 2, B ₁ + B _r / 2). In the following, an image generated at this time is referred to as a “filled image” in the sense that the region corresponding to the FG edge group is “filled”.
When this raster development is completed, the BG image generation unit 135 returns to the operation of the flowchart of FIG.

  Next, when it is determined that one end of the FG edge group is an edge of the BG image (step S265; NO), the BG image generation unit 135 is based on information included in the edges at both ends in contact with the FG edge group. The region corresponding to the FG edge group is raster-developed as a BG image, thereby “filling” the region (step S267). Here, raster expansion is not performed on the entire region corresponding to the FG edge group as in step S266. This is because, in this case, one end of the region corresponding to the FG edge group to be processed is in contact with the null image. If raster development is performed up to the boundary of the null image with respect to the FG edge group in contact with the null image, the boundary of the region corresponding to the FG edge group when the BG image generated by the raster expansion is compressed by the JPEG method May cause noise (block noise).

  FIG. 13 is a diagram exemplarily used to explain the conditions for generating this noise, and this diagram shows the boundary between the character area (FG image) and the blank area (null image). For example, in JPEG compression processing, DCT is performed in units of 8 × 8 pixel blocks. At this time, as shown in FIG. 13A, if the DCT block is selected so as not to include the blank area, the color component of the character area is converted into the blank area by the conversion (compression processing) of this block. There is no contamination. However, as shown in FIG. 13B, if the DCT block is selected so as to include a blank area, the color component of the character area is added to the blank area b by this block conversion (compression process). Will be mixed. In order to prevent generation of a hole-filled image in such a region, in the present embodiment, a range for forming a hole-filled image is determined by the following method.

1. First, the length “FG_Length” of the FG edge group is obtained. Specifically, FG_Length = FG_Ex−FG_Sx + 1. Here, FG_Sx is the start point coordinate of the left edge of the FG edge group, and FG_Ex is the end point coordinate of the right edge of the FG edge group (unit is pixel).
2. When the block length Block_Length in the x direction of the compression method used by FG_Length is larger than that of Block_Length (for example, Block_Length = 8 in the case of JPEG method), only the length of n blocks satisfying FG_Length ≧ (Block_Length × n) A hole-filled image is formed on the side in contact with the edge of the BG image in the region corresponding to the group. The color value of the hole-filled image is the RGB value of one end of the edge of the BG image that is in contact with the FG edge group.
3. When FG_Length <Block_Length, no hole-filling image is generated.

  Here, FIG. 14 and FIG. 15 are diagrams illustrating an example in which a hole-filling image is generated according to the above-described method. First, in FIG. 14, FG_Length = 10 and Block_Length = 8. In such a case, in a region corresponding to the FG edge group, a hole-filled image is generated on the side in contact with the edge of the BG image, that is, a region corresponding to 8 pixels from the right side in the drawing. On the other hand, in FIG. 15, FG_Length = 6 and Block_Length = 8. In such a case, if a hole-filled image is formed in a region corresponding to the FG edge group, noise may be generated in the null image region, so that the hole-filled image is not generated.

When the hole-filling image is generated by any one of the above steps S266 and S267, the BG image generation unit 135 ends this processing and returns to the operation of the flowchart of FIG.
When the edge filling process is completed, the BG image generation unit 135 performs the determination in step S22 described above. The subsequent processing is repeated until there are no more edges and edge lines to be processed.

(1-2-3) Example of Operation From now on, the series of operations of the image processing apparatus 10 described above, particularly the BG image generation processing in the printer driver 13, will be described more specifically with reference to document data. To do. Here, the document data DOC1 in FIG. 9 is used as an example of the document data.
When the operator gives an instruction to print the document data DOC1, the printer driver 13 of the image processing apparatus 10 generates intermediate data. Here, FIG. 16 shows the i-th scan line of the intermediate data, and EL _i is an edge line included in this scan line. In the figure, the edge line EL _i is composed of five types of edges E ₁ , E ₂ , E ₃ , E ₄ , and E ₅ . Edges E ₁ and E ₅ are edges of a blank area, that is, an area where no drawing object exists, and these edges are classified as null images. Edges E ₂ and E ₄ are edges of a region where the graphics object is drawn, and these edges are classified into BG images. The edge E ₃ is an edge of a region where the text object is drawn, and this edge is classified into an FG image.

FIG. 17 shows a specific data structure of these edges. In the figure, RGB values (255, 255, 255), (255, 0, 0) and (0, 0, 0) represent white, red and black, respectively. For each edge, since the end point coordinates of one edge are continuous with the start point coordinates of the next edge, it is understood that the image data of this edge line is formed without any gap by raster development of these edges. . Here, the correspondence between the tag information and the layer image in the present embodiment is defined by the tag correspondence table TBL in FIG. That is, the edges E ₂ and E ₄ are BG images, and the edge E ₃ is an FG image.

FIG. 18 is a diagram illustrating a BG image in a region corresponding to the edge line EL _i of the intermediate data. As shown in the figure, there is a region on the BG image where there are no drawing objects surrounded by drawing objects (graphics objects) at both ends. This area (hereinafter, this area is referred to as “missing area”) corresponds to an area where a drawing object (text object) exists on the FG image. When such a BG image is raster-developed as it is and compression processing is performed using the JPEG method, the noise described above occurs at the boundary of the missing region. Therefore, the printer driver 13 performs edge filling processing in the manner of the flowchart shown in FIG. Hereinafter, specific processing contents in the BG image generation unit 135 of the printer driver 13 will be described with reference to each step of FIG. For details of the processing in each step, refer to (1-2-2) above as appropriate.

The edge arrangement pattern in the missing region is a “BG-FG-BG pattern” as is apparent from FIG. Therefore, the BG image generation unit 135 makes a positive determination in step S265, and executes the process shown in step S266. Here, the BG image generation unit 135 refers to the information of the edges E ₂ and E ₄ adjacent to the missing region, and the start point coordinates are represented by 1200 (1199 + 1) and the end point coordinates are represented by 1399 (= 1400-1). A hole-filled image is formed in which a region to be drawn is drawn in red, that is, a color of (255, 0, 0). When such processing is performed, the “missing area” is in a state where “hole filling” is performed.
Note that such edge filling processing is not performed in the region corresponding to the null image of the intermediate data. This is because, for such a region, the determination in step S261 of the edge filling process is negative, and no filling image is generated.

FIG. 19 is a diagram showing edge lines EL _i before and after the edge hole filling process. FIG. 20 is a diagram showing a BG image before the edge filling process and a BG image obtained by executing the edge filling process over all scanning lines. By performing the above-described edge filling process on the missing area in this way, it is possible to remove the missing area from the BG image. Then, even when this BG image is compressed by the JPEG method and displayed so as to be superimposed on the FG image, it is possible to prevent noise from being generated at the boundary of the A-shaped character region. Furthermore, in the BG image, the information regarding the shape of the FG image is removed by the edge filling process, and therefore, the compression rate when compressed by the JPEG method is also more advantageous than the BG image before the edge filling process. Can be obtained.

  As described above, the image processing apparatus 10 according to the present embodiment is characterized in that image processing is performed using intermediate data. According to the image processing apparatus 10 of the present embodiment, unlike the technique described in Patent Document 1, it is not necessary to fill a character area with reference to a binary image and a multi-valued image, and it is extremely efficient. It is possible to execute image processing. In addition, since the image processing apparatus 10 according to the present embodiment performs image processing using intermediate data, compared with the technique described in Patent Document 1 that refers to a binary image and a multi-valued image, the image processing apparatus 10 has a smaller storage capacity. Image processing is possible. Therefore, by using the technique of the image processing apparatus 10 of the present embodiment, it is possible to expect higher processing speed, smaller apparatus size, and lower cost as compared with the case where the technique described in Patent Document 1 is used.

(2) Second Embodiment Subsequently, a second embodiment of the present invention will be described.
This embodiment is different from the first embodiment described above in that the type of tag information included in the edge is different. Accordingly, the contents of the tag correspondence table and the edge filling process are different, but the components and approximate processing of the image processing apparatus are the same as in the first embodiment. Therefore, the following description will be made with a focus on differences from the above-described first embodiment, and description of components and processes similar to those of the first embodiment will be omitted as appropriate.

  FIG. 21 shows the types of tag information in this embodiment. As shown in the figure, the tag information of this embodiment includes “text (Tx)”, “graphics (Gr)”, “image (Im)”, “null (Nl)” of the first embodiment. "Gradation (Gd)" and "Pattern (Pt)" are defined. The “gradation” tag indicates that an edge having this attribute is a drawing object having a drawing effect that changes its color value stepwise. Here, only a drawing object whose color value changes in the horizontal direction (x direction) as a gradation is considered, but it can of course be applied to other gradations. The “pattern” tag represents a drawing object that forms a pixel pattern that is repeated at a certain cycle with an edge having this attribute. The “gradation” tag and the “pattern” tag are both associated with the BG image.

  By adding such tag information, the processing contents in the edge filling process are different. Specifically, the processing content of step S266 in FIG. 11 is different. In the present embodiment, the BG image generation unit 135 refers to the tag information (T) in addition to the start point coordinates (Sx), end point coordinates (Ex), and color information (C) of the BG images at both ends of the FG edge group. Raster development. If the BG image generation unit 135 refers to the tag information of the BG images at both ends of the FG edge group, and this is a “text”, “graphics”, “image”, or “null” tag, the processing at this time is the first This is the same as the embodiment. However, if the BG image generation unit 135 refers to the tag information of the BG images at both ends of the FG edge group, and this is a “gradation” or “pattern” tag, a process different from that of the first embodiment is performed. . Therefore, here, only the description of step S266 when the tag information is the “gradation” or “pattern” tag will be given, and the others will be omitted.

First, a case where the BG images at both ends of the FG edge group are “gradation” tags will be described. Here, the end point coordinates and color information of the part of the BG image in contact with the left end of the FG edge group to be processed are respectively Ex _l and C _l, and the BG image of the part in contact with the right end of the FG edge group to be processed When the start point coordinates and color information of Sx _r and C _r are respectively, raster development uses “Ex _l +1” as the start point coordinates and “Sx _r +1” as the end point coordinates, and the region in this range is defined as C _l and C _r. This is a process of drawing with a color that changes step by step. Specifically, the number of pixels in the x direction of the FG edge group is p _1, and the RGB values of C _l and C _r are (R _l , G _l , B _l ) and (R _r , G _r , B _{r, respectively.} ) and when, BG image generation unit 135, the left end of the RGB values of the FG edge group and C _l, the right end of the RGB values of the FG edge group and C _r, in between each pixel, R in RGB values, G, B Each value is in steps of “| R ₁ −R _r | / p ₁ ”, “| G ₁ −G _r | / p ₁ ”, “| G ₁ −G _r | / p ₁ ” step by step. Draw as it changes. Here, | n | indicates the absolute value of the numerical value n, and rounding or the like is performed when the values of R, G, and B are not divisible. If such processing is performed, the color of the region corresponding to the FG edge group is drawn so as to change stepwise, and there is no sense of incongruity in the color change between the BG images at both ends.

FIG. 22 is a diagram for comparing the edge filling process of the present embodiment with the edge filling process of the first embodiment. For example, if the intermediate data before processing of a certain edge line is EL ₄ , EL ₅ is obtained by performing the edge filling process of the first embodiment. At this time, the color value of the area corresponding to the missing area in the BG image of the edge line EL ₅ is the average value of the color values at both ends of the FG edge group, and the area is rendered with a uniform color. On the other hand, when the edge hole filling process of this embodiment is performed on this edge line, it becomes EL ₆ and the color value changes step by step even in the region corresponding to the missing region of the BG image, which is uncomfortable with the BG images at both ends. It will be connected without.

Next, a case where the BG images at both ends of the FG edge group are “pattern” tags will be described. Here, the end point coordinates of the part of the BG image that is in contact with the left end of the processing target FG edge group are set as Ex _l, and the start point coordinates of the part of the BG image that is in contact with the right end of the processing target FG edge group are set as Sx _r . Then, this raster development is a process of drawing an area in this range with a pixel pattern that is repeated at a constant cycle, with “Ex _l +1” as the start point coordinate and “Sx _r +1” as the end point coordinate. Specifically, for example, when the BG images at both ends of the FG edge group are pattern images that change from the color value C ₁ to C ₂ at an interval of p ₁ pixels, this corresponds to the FG edge group. The region is drawn so as to form a similar pattern image. If such processing is performed, the color of the region corresponding to the FG edge group is drawn with a pixel pattern that is repeated at a constant cycle, and there is no sense of incongruity in the color change between the BG images at both ends.

FIG. 23 is a diagram for comparing the edge filling process of the present embodiment with the edge filling process of the first embodiment. For example, if the intermediate data before processing of a certain edge line is EL ₇ , EL ₈ is obtained by performing the edge filling process of the first embodiment. At this time, the color value of the region corresponding to the missing region in the BG image of the edge line EL ₈ is the average value of the color values at both ends of the FG edge group, and the region is rendered with a uniform color. On the other hand, when the edge hole filling process of this embodiment is performed on this edge line, it becomes EL ₉ and the color value changes in a certain period even in the region corresponding to the missing region of the BG image, It becomes connected without a sense of incongruity.

(3) Modification Examples The image processing apparatus according to the present invention has been described by exemplifying the first and second embodiments. However, the application of the present invention is not limited to such an aspect, and various modifications are possible. Is possible. An example is shown below.
First, in the above-described embodiment, when edge filling processing is performed, only the left and right edges of the FG edge group to be processed are referred to, but the application of the present invention is limited to such an aspect. Instead, for example, the upper and lower edge lines of the FG edge group to be processed may be referred to. That is, the region referred to in the edge hole filling process may be an edge adjacent to the processing target FG edge group. Further, the edge lines referred to at this time may refer to a plurality of edge lines as well as the edge lines directly in contact with the processing target FG edge group.

In the present invention, the JPEG method using DCT is particularly suitable as the BG image compression method, and this has been described as an example in the above-described embodiments. However, the BG image compression method in the present invention is not limited to the JPEG method, and various compression methods that perform compression processing in units of blocks, such as DCT, can be used.
Similarly, the FG image compression method is not limited to the MMR method, and various compression methods such as an MH (Modified Huffman) method and an MR (Modified Read) method can be used.

In the above-described embodiment, there are two types of layer images, FG images and BG images. Of course, the present invention can be applied even when there are three or more layer images. . When the above-described edge filling process is performed when there are three or more layer images as described above, it is necessary to refer not only to the upper layer of the layer image to be processed but also to the lower layer. . For example, a situation is assumed in which a missing region exists in a certain layer (referred to as L ₁ ) and a drawing object exists in a region corresponding to the missing region in a lower layer (referred to as L ₂ ) of this layer L ₁ . Here, when the layer L ₁ is raster-developed, if the missing area is filled without referring to the lower layer L ₂ , the layer L ₁ is formed in an area corresponding to the missing area on the lower layer L _2. The drawn object is masked by the hole-filled image and is not displayed. Therefore, when performing such processing, the BG image generation unit of the image processing apparatus refers to the lower layer of the layer image that is the processing target, and corresponds to an area in which a hole-filled image is to be formed in the lower layer. If there is a drawing object in the area to be processed, it is necessary to perform an operation such that the edge filling process is not performed.

  In such a case, in the image processing apparatus, it may be stored in what compression method the image generation unit corresponding to each layer performs the compression process. Then, referring to the compression method of the image forming unit of each layer stored when generating the multilayer compressed image data, in the image generating unit of the layer to which the compression method suitable for performing the edge filling process described above is applied Only this edge filling process should be performed.

  In the above-described embodiment, it is described that the document data is created by the document creation application. However, the present invention is not limited to such a mode, and the document data is supplied from an external device. It may be a thing.

1 is a block diagram illustrating a hardware configuration of an image processing apparatus according to a first embodiment of the present invention. FIG. 2 is a functional block diagram illustrating a functional configuration of the image processing apparatus in FIG. 1. FIG. 2 is a functional block diagram showing a functional configuration of a printer driver in the same embodiment. It is the schematic diagram which showed the structure of the intermediate data in the same embodiment. It is the schematic diagram which showed the structure of the edge in the same embodiment. It is the schematic diagram which showed the tag corresponding | compatible table in the same embodiment. 5 is a flowchart illustrating an operation in which an FG image generation unit generates an FG image from intermediate data in the embodiment. 4 is a flowchart illustrating an operation in which a BG image generation unit generates a BG image from intermediate data in the embodiment. It is the figure which illustrated the document data in which edge hole filling processing is performed in the same embodiment. It is the figure which showed the BG image before an edge hole-filling process is performed in the embodiment. It is the flowchart which showed the edge hole filling process in the same embodiment. It is the figure which showed the arrangement pattern of the edge in the same embodiment. It is the figure illustrated in order to demonstrate the conditions which noise generate | occur | produces. It is the figure which showed the edge line at the time of producing | generating the hole-filling image in the same embodiment. It is the figure which showed the edge line when the production | generation of a hole-filling image was not performed in the same embodiment. It is the figure which showed the scanning line of the intermediate data in the same embodiment. It is the figure which showed the data structure of the edge in the same embodiment. In the same embodiment, it is a figure showing the BG image of the area | region corresponded to the edge line of intermediate data. It is the figure which showed the edge line before and behind the edge hole filling process in the same embodiment. It is the figure which showed BG image before performing the edge hole-filling process in the same embodiment, and the BG image obtained by performing this edge hole-filling process over all the scanning lines. The kind of tag information in the 2nd Embodiment of this invention is shown. It is a figure for comparing the edge filling process of the same embodiment with the edge filling process of the first embodiment. It is a figure for comparing the edge filling process of the same embodiment with the edge filling process of the first embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 1 ... Control part, 2 ... Memory | storage part, 3 ... Input-output I / F, 4 ... Display part, 5 ... Operation part, 11 ... OS, 12 ... Document creation application, 13 ... Printer driver, 131 ... I / F part, 132 ... Intermediate data generation part, 133 ... FG image generation part, 134 ... FG image compression part, 135 ... BG image generation part, 136 ... BG image compression part, 137 ... Format part.

Claims (15)

Storage means for storing correspondence information between attribute information indicating the types of objects constituting the document and each layer divided into a plurality from upper to lower;
Conversion means for converting document data representing a document into image information;
Layered image generation means for performing processing for assigning the object included in the image information to a plurality of layers in accordance with the correspondence relationship stored in the storage means, and generating layered image data representing images of these layers,
When there is a missing area where no object exists in the image of the specific layer that is the processing target, the layered image generation unit creates a region corresponding to the position of the missing area in the upper layer or the lower layer of the specific layer. An image processing apparatus for forming a hole-filled image generated based on an existing object and an object existing in the specific layer in the missing region to generate layered image data of the specific layer. The image information is represented by a set of edge lines obtained by subdividing the image in a certain direction,
The image according to claim 1, wherein the edge line is represented by a set of edge information including at least one of start point coordinates, end point coordinates, color information, and attribute information of each object included in the edge line. Processing equipment. The layered image generation means includes
The image processing apparatus according to claim 2, wherein each of the edge information is assigned to any one of the plurality of layers based on the correspondence relationship stored in the storage unit. The layered image generation means includes
When the object exists in an area corresponding to the position of the missing area in the upper layer, the hole filling image is formed in the missing area based on an object in an area adjacent to the missing area in the specific layer. The image processing apparatus according to 1. The layered image generation means includes
The image processing apparatus according to claim 1, wherein the hole-filled image is not formed when an object exists in an area corresponding to the position of the missing area in the lower layer. The layered image generation means includes
The image processing apparatus according to claim 2, wherein the hole-filling image is formed based on the start point coordinates, the end point coordinates, and the color information of an object existing in an area adjacent to the missing area. The layered image generation means includes
The image processing apparatus according to claim 2, wherein the hole-filled image is formed based on the start point coordinates, the end point coordinates, the color information, and the attribute information of an object existing in an area adjacent to the missing area. The layered image generation means includes
The image processing according to claim 6 or 7, wherein when the color information of an object existing in an area adjacent to the missing area is color information having a constant color value, the hole-filled image having the color value is formed. apparatus. The layered image generation means includes
8. The hole-filled image having an average value of the color values of the plurality of objects is formed when objects existing in the plurality of regions adjacent to the missing region have color information of different color values. Image processing apparatus. The layered image generation means includes
Attribute information of an object having a drawing effect in which objects existing in a plurality of areas adjacent to the missing area have color information of different color values, and these objects both change the color value in stages. The image processing apparatus according to claim 7, wherein the hole-filling image having a color value that changes stepwise is formed. The layered image generation means includes
When forming the hole-filling image having a color value that changes stepwise in the missing area, the first color value of the object existing in the first area adjacent to the missing area is changed to the first color value adjacent to the missing area. The image processing apparatus according to claim 10, wherein the hole-filled image having a color value that gradually changes to a second color value of an object existing in the second area is formed. The layered image generation means includes
8. The hole-filling image having a periodically changing color value is formed when an object existing in an area adjacent to the missing area has attribute information of an object having a periodically changing color value. Image processing device. The layered image generation means includes
In the upper layer, when an object exists in an area corresponding to the position of the missing area and an area where no object exists is included in a plurality of areas adjacent to the area, the missing layer includes the missing area. The image processing apparatus according to claim 4, wherein a hole-filled image is not formed in the vicinity of a boundary between a region and a region corresponding to a region where the object in the upper layer does not exist. The image processing apparatus according to claim 1, further comprising: a compression unit that performs compression processing on each layer image of the layered image data generated by the layered image generation unit using a compression method determined for each layer. 15. The image to be generated is determined by referring to a compression method of compression processing performed on the image of the specific layer when the layered image generation unit forms the hole-filled image in the missing region of the specific layer. The image processing apparatus described.

Images