JP4911585B2 - Image processing apparatus, image processing method, program, and information recording medium - Google Patents

Description

  The present invention relates to an image processing apparatus and method, and more particularly, to create a page description file in which commands for creating partial images of each object from a bitmap image including a plurality of objects and drawing the partial images on a page are described. The present invention relates to an image processing apparatus and method.

  There is a scene where a bitmap image including a plurality of objects is desired to be handled in units of objects. For example, there is a one-page bitmap image in which character objects and photo objects are mixed, and only a certain photo object is used as a material for a word profile. Normally, in such a scene, a dedicated image editing software (such as Adobe's PhotoShop) is used to cut out a desired object portion. However, this work is complicated, and such dedicated image editing software is not always available. Therefore, it is desirable to perform the clipping operation on the bitmap image in advance.

  However, it is not sufficient to create a file group of partial images cut out for each object from a bitmap image. This is because it is difficult to find a desired partial image because the positional relationship of each object in the original image is unknown. Therefore, it is preferable to provide a mechanism for displaying each partial image at the same position as that extracted from the original bitmap image. This can be achieved with a page description file (W3C HTML: HyperTextMarkupLanguage, etc.) that contains commands for where to draw each partial image in the page. If this page description file is displayed by general viewer software (such as Microsoft Internet Explorer), the same appearance as the original bitmap image can be obtained, so that a desired object can be easily found. Further, since each object is already divided, copying of the object can be easily realized by a click operation on the desired object.

  A specific example will be described. FIG. 15A is one bitmap image including three objects. A state in which each object is enclosed by a rectangle is shown in FIG. The partial images cut out by this rectangle are shown in FIGS. 16A, 16B, and 16C, and each is made into an image file (here, a JPEG file). FIG. 19A shows the contents of a page description file (here, HTML) in which a command for instructing where to draw the image file is written. FIG. 20A shows the page description file displayed on the viewer (Internet Explorer). In this way, the same appearance as the original bitmap image can be obtained despite being divided into partial images. However, it may not look the same. An example is shown below.

  FIG. 17A is a bitmap image including two objects, and FIG. 17B shows a state in which each object is surrounded by a rectangle. The difference between the two rectangles and the example shown in FIG. That is, the rectangle surrounding the object P includes a part of another object G. Since this part is a hindrance from the viewpoint of using an object as a material, a partial image (FIG. 18A) that does not include the object is converted into an image file. When the page description file shown in FIG. 19B or 19C (only the order of drawing commands is different) is displayed, it looks like FIG. 20B or 20C, respectively. FIG. 20C looks the same as the original image, but FIG. 20B does not look the same. This is because (b) of the same figure draws the partial image that does not include the disturbing part above the other partial image. In general page description language, the second drawing for the same coordinates is applied to the upper side of the first drawing, and the lower drawing data is hidden and cannot be seen. Yes. In the page description file of FIG. 19B, since the drawing command for the partial image that does not include the disturbing part is described later, the part cannot be seen as it is. Conventionally, the drawing order of the partial images is in the raster order (up / left priority) as shown in FIGS. 19A and 19B, and this problem of information loss has occurred.

  There is a conventional technique for solving this problem, such as an alpha channel, but there is a problem in versatility. The alpha channel is auxiliary data provided in addition to the color expression data for each pixel, and is used for image masking and transmission composition. In the above example, the pixel data outside the object is not erased with the alpha value (mask) without erasing the disturbing part, or the disturbing part is erased but the pixel data outside the object is alpha value. To make it completely transparent (transparent synthesis). By doing so, the problem of information loss of the original image can be solved. However, the image format that supports the alpha channel is limited to the one like PNG (Portable Network Graphics), and there is a problem in versatility such that it is not supported by the most common JPEG.

JP 2002-297339 A JP 2002-41498 A

  Accordingly, an object of the present invention is to provide an alpha channel in an image processing apparatus or method for creating a partial image of an individual object from a bitmap image and creating a page description file describing a command for rendering the partial image on a page. It is to eliminate or improve the above-mentioned problem of information loss at the time of image display without using such a technique.

An image processing apparatus according to the invention of claim 1 is provided.
Each pixel detects a cluster of adjacent non-background attribute pixels in a bitmap image having a background attribute or a non-background attribute as an object, and each detected object includes one object and does not constitute the object. Partial image creation processing means for creating a rectangular partial image as background attribute pixels;
A drawing order determination processing means for determining a drawing order of the partial images created by the partial image creation processing means, and a drawing order determined by the drawing order determination processing means for the partial images created by the partial image creation processing means Has a file creation processing means for creating a page description file in which a command to be drawn on the page is described,
The drawing order determination processing means detects a mutual overlap when the partial images created by the partial image creation processing means are drawn on a page, and between two partial images with overlap, an object in the overlap area is detected. The drawing order is determined so that one partial image having a larger number of non-background attribute pixels constituting the image is drawn above the other partial image.
It is characterized by that.
It is characterized by that.

The image processing method according to the invention of claim 2
Each pixel detects a cluster of adjacent non-background attribute pixels in a bitmap image having a background attribute or a non-background attribute as an object, and each detected object includes one object and does not constitute the object. A partial image creation process for creating a rectangular partial image as a background attribute pixel;
A drawing order determination processing step for determining a drawing order of the partial images created by the partial image creation processing step; and a drawing order determined by the drawing order determination processing step for the partial images created by the partial image creation processing step. A file creation process for creating a page description file in which commands to be drawn on the page are described,
The drawing order determination processing step detects a mutual overlap when the partial image created by the partial image creation processing step is drawn on a page, and between two partial images having an overlap, an object in the overlap region is detected. The drawing order is determined so that one partial image having a larger number of non-background attribute pixels constituting the image is drawn above the other partial image.
It is characterized by that.

A program according to a third aspect of the present invention causes a computer to execute each step of the image processing method according to the second aspect of the present invention.

A computer-readable information recording medium according to a fourth aspect of the invention is characterized in that a program for causing a computer to execute each step of the image processing method according to the second aspect of the invention is recorded.

When the page description file obtained by the image processing apparatus or the image processing method according to the present invention (claims 1 and 2) is displayed by general viewer software, the problem of information loss due to overlap between partial images is solved or improved. Is done. That is, the area of the object that cannot be seen due to the overlapping of the partial images is reduced. In addition, the image processing method (claim 2) according to the present invention can be carried out by loading a program according to the present invention or a program recorded on an information recording medium (claims 3 and 4) into a computer and executing it. it can.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an example of an image processing system in which the present invention is implemented. In FIG. 1, 110 is a computer, and 111 is an image editing apparatus. The image editing apparatus 111 is a personal computer having a general configuration, for example.

  In the computer 110, 100 is a CPU, and 102 is a ROM or HDD (hard disk device) that stores a program and data of the CPU. Reference numeral 101 denotes an image processing unit that processes image data at high speed using an ASIC, a DSP, or the like, but this is not essential. Reference numeral 103 denotes a RAM, which is used as a work memory for developing image data to be processed, execution programs, and the like. An image input unit 104 is, for example, a scanner that inputs image data by scanning a paper document, or a digital camera that obtains an electronic image by imaging a space. Reference numeral 105 denotes an interface (Ethernet, telephone line, wireless, etc.) with an external device. Via this interface 105, for example, an electronic image can be input from an external device, data processed by the computer 110 can be transferred to the image editing device 111, and displayed by viewer software (Internet Explorer or the like).

  FIG. 2 is a block diagram showing a functional configuration of the image processing apparatus according to the present invention. FIG. 3 is a flowchart for explaining the contents of image processing in the image processing apparatus according to the present invention. FIG. 3 is also a flowchart for explaining the processing contents of the image processing method according to the present invention. Therefore, the following description of the image processing apparatus also serves as a description of the image processing method.

  In FIG. 2, reference numeral 201 denotes a bitmap image to be processed, which is input to an area allocated on the RAM 103. The bitmap image 201 may be obtained by the image input unit 104 in FIG. 1 or may be obtained by rendering by the CPU 100, but the object of the present invention is a document image. Yes, it is not a natural image as if a camera had captured a space. In particular, it is intended for a document image in which objects such as characters, figures, and photographs are drawn without duplication on a solid background of one color. The gradation may be any of 1 bit / pixel (binary), 8 bit / pixel (grayscale), and 24 bit / pixel (RGB color), but here each component is an RGB color of 8 bits / pixel. It will be explained as being.

  A partial image creation processing unit 202 detects an object from the bitmap image 201 and performs a process of creating a partial image for each detected object. Reference numeral 203 denotes a drawing order determination processing unit that performs processing for determining the order in which the partial images created by the partial image creation processing unit 202 are drawn on the page. A file creation processing unit 204 creates a file group 205 of partial images created by the partial image creation processing unit 202 and draws the partial images on the page in the drawing order determined by the drawing order determination processing unit 203. A process for creating a page description file 206 in which a command to be executed is described is performed. However, the partial image creation processing unit 202 may convert the partial image into a file. The partial image, its file, and page description file are generated in an area allocated on the RAM 103.

  In FIG. 3, step 1, step 2, and step 3 are processing steps by the partial image creation processing unit 202. Step 4 is a processing step by the drawing order determination processing unit 203. Step 5 is a processing step by the file creation processing means 204.

These processing steps can be executed as software processing by the CPU 100 in FIG. 1, executed using the image processing unit 101, or only part of the processing steps can be executed using the image processing 101. It is also possible to execute the steps as software processing by the CPU 100. The form executed only by software processing by the CPU 100 is general because it can be performed by a computer that does not include the image processing unit 101. In any case, a program that causes a computer to execute each processing step shown in FIG. 3 (in other words, a program that causes a computer to function as each processing means 202, 203, and 204 shown in FIG. 2) is described in claim 3 . This corresponds to one embodiment of the invention. A computer-readable information recording (storage) medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor storage element on which such a program is recorded corresponds to an embodiment of the invention described in claim 4. To do.

  Hereinafter, the contents of the image processing will be specifically described along the processing flow shown in FIG.

[Background color detection processing step (step 1)]
This processing step corresponds to preprocessing in the partial image creation processing unit 202. In this processing step, as shown in the flowchart of FIG. 4, a histogram of the pixel values of the bitmap image 201 is obtained (step 11, step 12), and the pixel value of the maximum frequency is set as the background color (step 13). For the most common white background, pixel values such as R = 255, G = 255, and B = 255 are background colors.

[Attribute setting processing step (step 2)]
This processing step also corresponds to preprocessing in the partial image creation processing unit 202. In this processing step, a background or non-background attribute is assigned to all the pixels of the bitmap image 201. Specifically, as shown in the flowchart of FIG. 5, first, as an initialization process, a background attribute is assigned to all the pixels of the bitmap image 201 (step 21). Actually, an attribute recording memory corresponding to the pixels of the bitmap image 201 is prepared one by one, and the pixel attributes are recorded in this memory. Next, it is checked whether or not the pixel value of one pixel of interest in the pit map image 201 is equal to the background color (step 22). When the pixel value is equal, a non-background attribute is given to pixels within a distance of 5 mm including the pixel of interest (step 23). Similar processing is repeated while sequentially moving the target pixel. If all the pixels have been processed (step 24, Yes), this processing step is terminated. In this way, by performing the dilation process (step 23) when assigning the non-background attribute, even if non-background color pixels are not connected, for example, between characters, this can be easily detected as a group of objects. Become.

  If the bitmap image 201 is obtained by scanning a paper document, it is more likely that the pixel value is quantized and the above-described “background color detection processing step” and “attribute setting processing step” are performed. It is preferable to eliminate the influence of

[Partial image generation processing step (step 3)]
This processing step is a central processing step by the partial image creation processing means 202. In this processing step, a cluster of adjacent non-background attribute pixels is detected as an object in a bitmap image in which a background or non-background attribute is assigned to all pixels. Then, for each detected object, a partial image is generated in which a pixel that includes one object and does not constitute the object is a background attribute pixel.

  Specifically, as shown in the flowchart of FIG. 6, the connection of non-background attribute pixels is tracked, and a cluster of connected non-background attribute pixels is detected as an object (steps 31 and 32). Next, the maximum and minimum X and Y coordinates of the smallest rectangle (the circumscribed rectangle) that surrounds the object of interest are detected (step 33), and a partial image in which all pixels are initialized to the background color with the size of the rectangle is created. (Step 34). Next, the pixel value of the non-background attribute pixel constituting the corresponding object of the bitmap image 201 is copied to this partial image (step 35). Thus, a partial image corresponding to one object was obtained. Similar processing is repeated for each object to generate a partial image thereof.

  According to the processing procedure shown in FIG. 6, for example, as shown in FIG. 7A, even if the circumscribed rectangle of one object L overlaps with the circumscribed rectangle of the other object S, The partial image of the object L and the partial image of the object S can be created normally.

[Drawing order determination (step 4)]
This processing step is executed by the drawing order determination processing unit 203. In this processing step, the drawing order of the partial images on the page is determined so as to eliminate or reduce the problem of information loss due to the overlapping of the partial images when the partial images are drawn on the page. Three examples of processing procedures in this processing step are shown below.

First, non-background Oite to the first embodiment detects the overlapping of mutual when drawing the partial image on the page, between the two partial images there is an overlap, which constitutes an object to the overlapping region The drawing order is determined so that one partial image having more attribute pixels is drawn above the other partial image.

  For example, in the example of the partial images of the two objects N and W drawn in the positional relationship as shown in FIG. 8A, the area of the object W included in the overlapping region than the object N The number of background attribute pixels is large. Therefore, in this example, the drawing order is determined so that the partial image of the object W is drawn above the partial image of the object N. When drawing is performed in this drawing order, it is displayed as shown in FIG. 8B, and when drawing is performed in the reverse drawing order, it is displayed as shown in FIG. 8C. Information loss of the original image occurs in any drawing order, but as is clear from the comparison between FIG. 8A and FIG. 8B, the direction along the drawing order determined by this embodiment (FIG. 8 ( b)) is less defective and less problematic.

  An example of the flow of such drawing order determination processing is shown in FIG. First, a partial image that overlaps when rendered on a page is detected (step 51). When the drawing position of the partial image is the same as the position of the object on the original bitmap image, by using the coordinate information of each partial image obtained in the partial image generation processing step (step 3) of FIG. It is possible to determine the presence or absence of overlap between partial images and to specify the overlap region. Further, when a partial image is drawn at a position different from the position of the object in the original bitmap image in the page description file, the overlap between the partial images at the drawing position is checked.

  Next, a pair of overlapping partial images is selected (step 52), an overlapping area (rectangular area) of the two partial images is specified (step 54), and non-background attribute pixels in each partial image in the overlapping area A number is obtained (step 55). Then, the drawing order is assigned to each of the two partial images so that the partial image with the larger number of non-background attribute pixels obtained is drawn above the other partial image (step 56). If the number of non-background attribute pixels is the same, the drawing order may be assigned so that one arbitrarily selected partial image is drawn above the other partial image.

  By performing the same process on all the partial image pairs in which the overlap is detected, when the assignment of the drawing order to all the partial images having the overlap is finished (step 53, Yes), the remaining in which the overlap is not detected The drawing order is assigned to the partial image (step 57). For example, the drawing order can be assigned to the partial images according to the raster order. Note that the drawing order may be assigned to the partial images that do not overlap first, and then the drawing order may be assigned to the partial images that overlap.

Next, a description will be given of the second embodiment. In this embodiment, the area of the objects of the partial images is obtained, and the drawing order is determined so that the partial images having a smaller object area between the partial images are drawn on the upper side. Usually, the smaller the area of the object, the higher the ratio of the image defect area due to overlap to the total area. In other words, it can be said that the smaller the area of the object, the less the defect is allowed. This is the basis for this drawing order determination method. Unlike the drawing order determination method in the first embodiment, such a drawing order determination method can be applied to all objects regardless of whether or not partial images overlap, and it is not necessary to specify an overlapping area between partial images. Therefore, the process is simple.

For example, in the example shown in FIG. 10A, since the area of the object M is smaller than the area of the object L and the area of the object S is smaller, the partial image of the object M is drawn above the partial image of the object L. The drawing order is determined such that the partial image of the object S is drawn above the partial image of the object M.

  FIG. 10B shows an example in which the partial images of the objects L, M, and S that are not overlapped in the original bitmap image are drawn at the positions where the overlap occurs according to the drawing order. Although there is an image defect, the worst situation where a small object is completely hidden behind a large object and cannot be seen is avoided.

FIG. 11 shows an example of the flow of such drawing order determination processing. First, the object area (number of non-background attribute pixels) of each partial image is obtained (step 61). Next, the areas obtained for the partial images are sorted from the largest to the smallest (in descending order) (step 62). Then, a drawing order is assigned to each partial image in accordance with the order after sorting the areas (step 63).

Next, a description will be given of the third embodiment. In this embodiment, it is identified whether the object type of each partial image is a character, a figure, or a photograph. Then, the drawing order of the partial images is determined so that the partial image of the character object is drawn above the partial image of the graphic object and the partial image of the graphic object is drawn above the partial image of the photographic object. In general, a graphic is better than a photograph, and a character is better than a graphic, so the amount of information per unit area is greater. This is the basis for this drawing order determination method.

  In the example of FIG. 12A, the drawing order is such that the graphic object G is drawn above the photo object P and the character object T is drawn further above. FIG. 12B is a drawing in which the partial images of the objects G, P, and T that were not originally overlapped are drawn in this order, and information on each object is transmitted to the person who sees it, although there is an image defect.

  An example of the processing flow for determining the drawing order is shown in FIG. First, the object type is determined for each partial image (step 71). Specifically, for example, as shown in the flowchart of FIG. 14, the number of colors (number of types of pixel values) of the partial image is counted (step 81). If the number of colors is two (step 82, Yes), the object type of the partial image is determined to be a character object (step 84). If the number of colors is more than 2 colors (step 82, No) and within 10 colors (step 83, No), the object type of the partial image is determined as a graphic object (step 85), and if the number of colors exceeds 10 colors (step 85, No) (Step 83, Yes), the object type of the partial image is determined to be a photo object (step 86).

  Next, the drawing order is assigned to the partial image of the photographic object (step 72). When there are a plurality of partial images of a photographic object, for example, the drawing order can be assigned in raster order, or the drawing order can be assigned in descending order of area.

  Next, the drawing order is assigned to the partial image of the graphic object from the next order after the last order assigned in step 72 (step 73). Since drawing is usually performed in an overwritten form, a partial image with a slower drawing order is drawn on the upper side. When there are a plurality of partial images of the graphic object, for example, the drawing order can be assigned in the order of raster, or the drawing order can be assigned in the order of increasing area.

  Finally, the drawing order is assigned to the partial image of the character object (step 74). When there are a plurality of partial images of the character object, for example, the rendering order can be assigned in the order of raster or in descending order of area.

As described above, in general, a graphic is larger than a photograph, and a character is larger than a graphic, and the amount of information per unit area is large. However, since the difference between pictures and figures in the amount of information per unit area is smaller than that between pictures and figures, the photo object and figure object are combined into a picture object. The drawing order may be determined so that the partial images are drawn on the upper side . In this case, the processing flow determines whether the object type of the partial image is a character object or a picture object in step 71 of FIG. That is, the partial image for which the determination result of step 82 in FIG. 14 is No is determined as a pattern object. In step 72 of FIG. 13, the drawing order is assigned to the partial image of the pattern object, step 73 is omitted, and in step 74, the drawing order is assigned to the partial image of the character object.

[Create file (step 5)]
This processing step is executed by the file creation processing means 204. In this processing step, a page description file is created in which a partial image is filed in, for example, the JPEG format, and a drawing command for drawing the partial image on the page in the determined drawing order is created. However, the file creation processing unit 204 may only create a page description file, and the partial image creation may be performed immediately after the partial image creation by the partial image creation processing unit 202.

  The page description file is, for example, a file described in HTML as shown in FIG. In this page description file, the order of drawing commands for each partial image is the same as the drawing order of the partial images. The coordinate position at which the partial image is drawn is specified by the drawing command. The coordinate position may be the same as or different from the coordinate position on the original bitmap image of each partial image.

  Note that a script language such as JavaScript may be added to the page description file. Then, the user can move each object to another coordinate position, collect only necessary objects, or bring objects to be compared closer by a drag and drop operation on the viewer.

  As mentioned above, although it demonstrated to embodiment of this invention, this invention is not limited only to it, Various deformation | transformation is permitted. For example, the process up to “partial image creation” may use a process of separating an object used in encoding such as MPEG4 from the background. A method combining the drawing order determination methods described above can also be used.

1 is a configuration diagram of an image processing system according to an embodiment of the present invention. It is a block diagram which shows the functional structure of the image processing apparatus which concerns on this invention. It is a flowchart which shows the schematic flow of an image process. It is a flowchart of a background color detection process. It is a flowchart of an attribute setting process. It is a flowchart of a partial image generation process. It is explanatory drawing of the object which overlapped in the form where one side is included by the other, and its partial image. It is a figure explaining the drawing order determination method in a 1st Example. It is a flowchart which shows an example of the drawing order determination processing flow in a 1st Example. It is a figure explaining the drawing order determination method in a 2nd Example. It is a flowchart which shows an example of the drawing technique determination process flow in 2nd Example. It is a figure explaining the drawing order determination method in a 3rd Example. It is a flowchart which shows an example of the drawing order determination processing flow in a 3rd Example. It is a flowchart of object type determination of a partial image. It is explanatory drawing of a bitmap image and its object. It is a figure which shows the partial image which cut out each object by the rectangle. It is a figure which shows the example with the overlap of the partial image of the object of a bitmap image. It is a figure which shows the example of the partial image containing only 1 object. It is a figure which shows the example of a page description file. It is a figure which shows the example drawn according to the page description file of FIG.

110 Computer 111 Image Editing Device 201 Bitmap Image 202 Partial Image Creation Processing Unit 203 Drawing Order Determination Processing Unit 204 File Creation Processing Unit 205 Partial Image File Group 206 Page Description File

Claims (4)

Each pixel detects a cluster of adjacent non-background attribute pixels in a bitmap image having a background attribute or a non-background attribute as an object, and each detected object includes one object and does not constitute the object. Partial image creation processing means for creating a rectangular partial image as background attribute pixels;
A drawing order determination processing means for determining a drawing order of the partial images created by the partial image creation processing means, and a drawing order determined by the drawing order determination processing means for the partial images created by the partial image creation processing means Has a file creation processing means for creating a page description file in which a command to be drawn on the page is described,
The drawing order determination processing means detects a mutual overlap when the partial images created by the partial image creation processing means are drawn on a page, and between two partial images with overlap, an object in the overlap area is detected. An image processing apparatus that determines a drawing order so that one partial image having a larger number of non-background attribute pixels constituting the image is drawn above the other partial image. Each pixel detects a cluster of adjacent non-background attribute pixels in a bitmap image having a background attribute or a non-background attribute as an object, and each detected object includes one object and does not constitute the object. A partial image creation process for creating a rectangular partial image as a background attribute pixel;
A drawing order determination processing step for determining a drawing order of the partial images created by the partial image creation processing step; and
A file creation processing step for creating a page description file in which a command for drawing a partial image created by the partial image creation processing step on a page in a drawing order determined by the drawing order determination processing step is described;
The drawing order determination processing step detects a mutual overlap when the partial image created by the partial image creation processing step is drawn on a page, and between two partial images having an overlap, an object in the overlap region is detected. An image processing method is characterized in that a drawing order is determined so that one partial image having more non-background attribute pixels constituting the image is drawn above the other partial image. The program which makes a computer perform each process of the image processing method of Claim 2. A computer-readable information recording medium in which a program for causing a computer to execute each step of the image processing method according to claim 2 is recorded.

Images