JP2007005844A - Coding processor, coding processing method, program and information recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus or a method for generating JPM coding data having high partial access performance. <P>SOLUTION: Original image data are decomposed into a background and a pair of one ore more foregrounds and mask data (steps 1101-1105). JPEG 2000 coding using tile dividing or precinct dividing is performed for the foregrounds and the mask (steps 1106 and 1107). Thus, partial access using a tile or precinct per unit can be realized for the code of the foregrounds or the code of the mask. Partial access of the code of the mask is also realized by dividing the mask etc. into a plurality of partial regions and individually coding each of the partial regions. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus and method for encoding a page image of a mixed document such as a character / line drawing and a photograph, and more particularly, background data obtained by decomposing original image data instead of encoding original image data as it is. And an apparatus and method for encoding at least one foreground data and mask data pair.

  Here, in the present invention, “background data obtained by decomposing original image data, at least one foreground data and mask data pair” means “background data 1 obtained by decomposing original image data, n foreground data i and masks” A method of selecting one of foreground data i and background data i for each pixel in accordance with mask data i in a pair of data i (where n is an integer equal to or greater than 1; i = 1, 2,... N), or The procedure for synthesizing the background data i + 1 by the method of taking the weighted average of the values of the foreground data i and the background data i for each pixel is sequentially repeated from i = 1 to i = n, and finally synthesized background data. The original image data is reproduced as n + 1 ”.

  In the following description and the drawings to be referred to, background data, foreground data, and mask data may be abbreviated as background, foreground, and mask, respectively.

  In general, a document is composed of a mixture of characters / line drawings and images. In recent years, in order to efficiently encode such mixed documents, document pages (original image data) are decomposed into background data, one or a plurality of foreground data and mask data pairs, and the data are encoded separately. A method has been proposed.

  For example, in MRC (mixed raster content), as schematically illustrated in FIG. 1, the original image data is foreground data that is character color information, mask data that is character region information, and background data that is image information. The background data, the foreground data, and the mask data are encoded separately. When reproducing the original image data, the reproduced image is synthesized by selecting the foreground data or the background data for each pixel in accordance with the mask data. Note that the original image data can be decomposed into background data and two or more “pairs of foreground data and mask data”, and the respective data can be encoded.

  With respect to MRC, for example, Patent Document 1 discloses a multi-valued picture image (background) representing a picture part of an original image, a multi-valued character color image (foreground) representing color information of a character / line drawing part of the original image, and the original image. Binary selection data (mask) representing the shape of the character / line drawing of JPEG, Lempel-Ziv, and MMR, respectively. Describes an invention of an image processing apparatus that combines and restores an image by selecting an image for each pixel in accordance with the expanded selection data. According to the description of paragraphs (0003) to (0005) of this patent document, the main object of the present invention is to prevent deterioration of characters and line drawings at a high compression rate.

  In recent years, along with the advent of a new encoding method JPEG2000, JPM (JPEG2000 Multi Layer) has been proposed which enables selection of JPEG2000 as a compression method for foreground data, mask data, and background data of the MRC model. In addition, JPIP (JPEG2000 Interactive Protocol), which is a mechanism for “transmitting and receiving only a code of a desired region in an image encoded with JPEG2000” in a network environment, has been proposed. These will be briefly described below.

  First, JPM will be described. In JPM, original image data is decomposed into “background data (BasePage)” and “a pair of foreground data and mask data” called one or a plurality of layout objects (Layout Objects). JPM background data is handled as an initial page on which a layout object is drawn. These background data, foreground data, and mask data are encoded independently, but JPEG2000 can be selected as the encoding method.

  For example, when reproducing original image data that has been decomposed into a background data, a pair of foreground data 1 and mask data 1, and a pair of foreground data 2 and mask data 2, the background data is converted from the foreground data 1 and the background data according to the mask data 1. 2 is synthesized. Next, background data 3 is synthesized from foreground data 2 and background data 2 in accordance with mask data 2. In this example, the synthesized background data 3 is reproduced image data. Even when the number of foreground data and mask data pairs increases, an image is reproduced by repeating the same synthesis procedure.

As a method of combining the background data and foreground data described above,
(I) a method for selecting foreground data or background data for each pixel;
(Ii) a method of taking a weighted average of values of foreground data and background data for each pixel;
Two are possible.

In the case of the method (i), the mask data is binary data, and the foreground data can be selected at the pixel position where the mask data value is “1” and the background data can be selected at the pixel position “0”. . In the case of the method (ii), for example, a positive value of 8 bits can be given to the mask data, and a weighted average of the values of the foreground data and the background data can be taken for each pixel. That is,
The composite image = (mask value / 255) × foreground + {(255−mask value) / 255} × background can be used to calculate the pixel value of the composite image. Which combination method is used can be designated for each pair of foreground data and mask data, and the designation is described in a header (described later) for each pair.

  Next, JPEG2000 will be described. JPEG2000 is a JPEG successor image encoding method that became an international standard in 2001, and the encoding process is generally performed according to the flow shown in FIG. First, the image is divided into rectangular tiles (number of divisions ≧ 1). Each tile is color-converted into components such as luminance and color difference. The converted component (referred to as a tile component) is divided into four subbands abbreviated as LL, HL, LH, and HH by wavelet transformation. When the wavelet transform (decomposition) is recursively repeated for the LL subband, one LL subband and a plurality of HL, LH, and HH subbands are finally generated.

  Next, each subband is divided into rectangles called precincts (see FIG. 3). The precinct is a subband divided into rectangles, and precincts corresponding to each of the HL, LH, and HH subbands (a total of three) are treated as a group. However, each precinct obtained by dividing the LL subband is treated as a precinct independently. The precinct roughly represents a position in the image. The precinct can be the same size as the subband. A code block is obtained by further dividing the precinct into rectangles (see FIG. 3). Therefore, the physical size order is image ≧ tile> subband ≧ precinct ≧ code block.

  FIG. 4 shows an example of subband division. The prefix numbers LL, HL, LH, and HH indicate the number of wavelet transforms (decomposition level) applied until the coefficient is obtained. FIG. 4 also shows the relationship between the decomposition level and the resolution level.

  After the above division, entropy coding for each subband coefficient (after linear quantization for each subband if an irreversible wavelet transform called 9 × 7 transform is used), for each code block and in bit plane order (Bit-plane encoding) is performed.

  Headers are collected and collected from all code blocks included in the precinct (for example, from the MSB of all code blocks to the third bit plane code) The attached one is called a packet. Since the “part” may be empty, the content of the packet may be empty from the sign. The packet header includes information about codes included in the packet, and each packet can be handled independently. In other words, a packet is a unit of code.

  By collecting packets of all precincts (= all code blocks = all subbands), a part of the code of the entire image (for example, the code of the bit plane from the MSB to the third bit of the wavelet coefficients of the entire image) can be created. But this is called a layer. Since the layer is roughly a part of the code of the bit plane of the entire image, the image quality increases as the number of layers to be decoded increases. That is. A layer is a unit of image quality formed in the bit depth direction. If all the layers are collected, it becomes the sign of all the bit planes throughout the image.

  FIG. 5 shows an example of layers when the number of decomposition levels = 2 and the precinct size = subband size, and packets included in the layers. Since the packet is based on a precinct, when precinct size = subband size, the packet spans HL to HH subbands. In the figure, only some packets are shown surrounded by thick lines.

Now, the operation of arranging packets according to the packet and layer delimiters generated above is called code formation. As described above, which component (symbol C) the packet belongs to,
-Which resolution level (symbol R) belongs to
Which precinct ("location") (symbol P) belongs to
-It has four attributes, which layer (symbol L) belongs to. The packet arrangement means the order in which packets are arranged hierarchically. This arrangement order is called a progression order, and five patterns in FIG. 6 are defined.

For example, in the case of LRCP progression order, the following for loop
for (layer) {
for (resolution) {
for (component) {
for (Precinct) {
Encoding: Place packet
When decoding: Interpret packet attributes}
}
}
}
Thus, the packet is arranged (when encoded) or interpreted (when decoded).

Each packet has a packet header, which
Whether the packet is empty,
Which code block is included in the packet,
The number of zero bit planes for each code block included in the packet,
-The number of coding passes (number of bit planes) of each code block code included in the packet,
The code length of each code block included in the packet,
Is described, but the layer number, resolution number, etc. are not described at all. In order to determine which resolution of which layer the packet is at the time of decoding, the for loop as described above is formed from the progression order described in the COD marker in the main header, and each code included in the packet It is only necessary to determine the break of the packet from the sum of the code lengths of the blocks and see where each packet is handled in the for loop. This means that if only the code length in the packet header is read, the next packet can be detected without decoding the entropy code itself, that is, an arbitrary packet can be accessed.

  As described above, the JPEG 2000 code can be accessed in packet units. This means that only necessary codes can be extracted from the original code to generate a new code, and only a partial code can be decoded from the original code as necessary.

  For example, when a large image on the server is displayed on the client side, a code with only the required image quality, a code with only the required resolution, a code with only the desired location, and a code with only the desired component are received from the server. Can be decrypted. As described above, JPIP (JPEG2000 Interactive Protocol) is a protocol for receiving only necessary codes from JPEG2000 codes in the server, and is currently in the process of standardization.

JPIP proposes that a client can specify an area to be drawn and an image quality from the server. When an area is designated, the server transmits a precinct packet covering the area. That is
for (Precinct) {
Interpret the packet}
In this loop, only necessary precinct packets can be transmitted. If the image quality is specified at the same time,
for (layer) {
...
for (Precinct) {
Interpret the packet}
...
}
In such a loop, only necessary layer packets can be transmitted.

  Protocols for partially accessing such hierarchical images can be found in the past from FlashPix, which is a multi-resolution representation of images, and IIP (Internet Imaging Protocol), which is a protocol for accessing it. . For example, Patent Document 2 is a prior art document related to IIP. Further, for example, Patent Document 3 is a prior art document related to a cache model in JPIP.

Japanese Patent No. 3275807 JP-A-11-205786 JP 2003-23630 A

  Now, when trying to access only a desired part of an image through JPIP with respect to JPM encoded data, it is desirable that only the code of the desired part can be transmitted and received for any of the foreground, mask, and background. However, such partial accessibility is not considered in the conventional encoded data of JPM. For example, the mask data is often binary image data, but conventionally, binary images such as MMR and JBIG widely used in FAX for compressing mask data that is binary image data in JPM. For this reason, partial access was impossible. The same applies to the MRC encoded data.

  Therefore, the present invention generates encoded data with high partial accessibility in an apparatus and method for encoding background data obtained by decomposing original image data and one or more pairs of foreground data and mask data. It is in.

  Again, in the present invention, “background data obtained by decomposing original image data, at least one pair of foreground data and mask data” means “background data 1 obtained by decomposing original image data, n foreground data i and A method of selecting one of the foreground data i and the background data i for each pixel in accordance with the mask data i in a pair of mask data i (where n is an integer equal to or greater than 1; i = 1, 2,... N); Alternatively, the procedure of synthesizing the background data i + 1 by the method of taking the weighted average of the values of the foreground data i and the background data i for each pixel is repeated in order from i = 1 to i = n, and finally the synthesized background The original image data is reproduced as data n + 1 ”.

  There are two approaches that allow partial access. One is to devise an encoding method that allows partial access even if the mask data is binary image data. Therefore, the inventor has noted that JPEG 2000 can be applied to mask data in JPM.

  As described above, tile division and precinct division can be applied in JPEG2000. If JPEG2000 encoding using tile division or precinct division is applied to the mask data, it is possible to access the code of the desired portion. In JPEG2000, encoding without applying wavelet transform is also possible. When the mask data is binary image data, generally, the code size of the mask data can be reduced without applying the wavelet transform. Also, even if the mask data is multi-value image data, for example, when there are only three values, and their absolute values are limited to powers of 0 or 2, if the wavelet transform is not applied, Since the bit planes other than the bit plane are filled with 0, the coding efficiency is generally good.

  Another approach that enables partial access is, for example, a method in which mask data is divided into a plurality of region portions, and these region portions are encoded separately (treated as separate images). Thus, even if the mask data itself is divided into a plurality of area portions (different images), there is no inconvenience because the JPM has a mechanism for integrating the divided area portions at the time of decoding. Note that the encoding method for the mask data in this case is not limited to a method capable of tile division or precinct division such as JPEG2000.

  The present invention is based on the above consideration. Hereinafter, the invention according to each claim will be described in order.

  According to the first aspect of the present invention, background data obtained by disassembling original image data, data acquisition means for acquiring at least one pair of foreground data and mask data, and background data, foreground data and mask data acquired by the data acquisition means And an encoding unit that encodes each of the image data, and the encoding unit performs JPEG2000 encoding using tile division or precinct division on the mask data. According to such a configuration, partial access to mask data is facilitated.

  According to a second aspect of the present invention, there is provided background data obtained by decomposing original image data, data acquisition means for acquiring at least one pair of foreground data and mask data, and background data, foreground data and mask data acquired by the data acquisition means. Encoding means for encoding each of the regions, wherein the encoding means divides the mask data into a plurality of region portions and individually encodes each region portion. . According to such a configuration, partial access to the mask data is facilitated.

  The invention of claim 3 is the encoding processing apparatus according to the invention of claim 1, wherein the encoding means performs masking when the absolute value of each pixel position of the mask data takes only a power of 0 or 2. An encoding processing apparatus is characterized in that wavelet transform is not applied when data is encoded. According to such a configuration, partial access to the mask data is facilitated, and the coding efficiency of the mask data can be increased.

  According to a fourth aspect of the invention, there is provided an encoding processing apparatus according to the first or second aspect of the invention, wherein the encoding means performs JPEG2000 encoding using tile division or precinct division on the foreground data, and foreground data. The encoding processing apparatus is characterized in that the division boundary of the mask and the division boundary of the mask data are matched. With this configuration, partial access to the mask data and foreground data is facilitated.

  The invention of claim 5 is the encoding processing apparatus according to the invention of claim 1 or 2, wherein the encoding means divides the foreground data into a plurality of area portions, and individually encodes each area portion, The encoding processing apparatus is characterized in that a division boundary of foreground data and a division boundary of mask data are matched. With this configuration, partial access to the mask data and foreground data is facilitated.

  Now, the background data (in the narrow sense) in JPM is defined to be composed of single-color pages. Such single-color background data can be easily accessed even if region division is not performed at the time of encoding. However, in the case of MRC that allows background data that is not a single color as illustrated in FIG. 1, a device that facilitates partial access to the background data is desired. The inventions of claims 6 and 7 are based on such consideration, and partial access to the background data is facilitated.

  That is, the invention of claim 6 is the encoding processing apparatus according to the invention of claim 1 or 2, wherein the encoding means performs JPEG2000 encoding using tile division or precinct division on background data, An encoding processing apparatus is characterized in that a division boundary of background data and a division boundary of mask data are matched.

  The invention of claim 7 is the encoding processing apparatus according to the invention of claim 1 or 2, wherein the encoding means divides the background data into a plurality of area portions and codes each area portion individually. The coding processing apparatus is characterized in that the division boundary of the background data and the division boundary of the mask data are matched.

  The inventions of claims 8 and 9 facilitate partial access to all of mask data, foreground data, and background data.

  That is, the invention of claim 8 is the encoding processing apparatus according to the invention of claim 4 or 5, wherein the encoding means performs JPEG2000 encoding using tile division or precinct division on background data, An encoding processing apparatus is characterized in that a division boundary of background data and a division boundary of mask data are matched.

  The invention of claim 9 is the encoding processing apparatus according to the invention of claim 4 or 5, wherein the encoding means divides the background data into a plurality of area parts, and individually encodes each area part, An encoding processing apparatus is characterized in that a division boundary of background data and a division boundary of mask data are matched.

  In JPM, a pair of foreground data and mask data (layout object) is synthesized with one background data, the synthesized image data is regarded as new background data, and the next layout object is synthesized. When there are a plurality of layout objects, it is desirable that all layout objects have the same dividing boundary for partial access.

  Therefore, the invention of claim 10 is the encoding processing device according to the invention of claim 4 or 5, wherein there are a plurality of pairs of foreground data and mask data, and the division boundary of the foreground data of all pairs and the mask data An encoding processing apparatus characterized by matching division boundaries. According to such a configuration, it is possible to improve the partial accessibility to the foreground data and the mask data when there are a plurality of pairs of foreground data and mask data.

  In JPM, mask data and foreground data can be shared between different layout objects. Even when the mask data is divided into a plurality of area portions and each area portion is encoded as separate image data as in the invention of claim 2, the foreground data is encoded as one image data (however, partial access is possible). This can be shared between different mask data. That is, it is not necessary to divide the foreground data into a plurality of separate images, and the code amount of the foreground data does not increase by not dividing the foreground data.

  In view of such points, the invention of claim 11 provides background data obtained by decomposing the original image data, data acquisition means for acquiring at least one foreground data and mask data pair, background data acquired by the data acquisition means, Encoding means for encoding foreground data and mask data, and code forming means for combining the background data, foreground data and mask data generated by the encoding means to form encoded data in a predetermined format; In the encoding means, the mask data is composed of a plurality of partial areas, and each of the partial areas is individually encoded. In the code forming means, the code of the foreground data is converted into a code of the plurality of partial areas of the mask data. It is an encoding processing device characterized in that it is shared between the two. According to such a configuration, it is possible to avoid the trouble of dividing the foreground data into a plurality of separate images without increasing the code amount.

  The inventions of claims 12 to 19 are encoding processing methods corresponding to the contents of the inventions of claims 1, 2, 4 to 9.

  That is, the invention of claim 12 provides background data obtained by disassembling the original image data, a data acquisition step of acquiring at least one foreground data and mask data pair, background data acquired by the data acquisition step, foreground data, and An encoding process for encoding each mask data, and JPEG2000 encoding using tile division or precinct division is performed on the mask data in the encoding step.

  According to a thirteenth aspect of the present invention, there is provided a data acquisition step of acquiring background data obtained by decomposing original image data, at least one pair of foreground data and mask data, and background data, foreground data and mask data acquired by the data acquisition step. An encoding process for encoding each of the areas, and in the encoding process, the mask data is divided into a plurality of area portions, and each area portion is individually encoded. .

  A fourteenth aspect of the invention is an encoding processing method according to the twelfth or thirteenth aspect of the invention, wherein in the encoding step, foreground data is subjected to JPEG2000 encoding using tile division or precinct division, and foreground data is obtained. This encoding processing method is characterized in that the division boundary of the mask and the division boundary of the mask data are made to coincide.

  The invention of claim 15 is the encoding processing method according to the invention of claim 12 or 13, wherein in the encoding step, the foreground data is divided into a plurality of area portions, and each area portion is individually encoded, The encoding processing method is characterized in that the division boundary of foreground data and the division boundary of mask data are matched.

  The invention according to claim 16 is the encoding processing method according to claim 12 or 13, wherein, in the encoding step, the background data is subjected to JPEG2000 encoding using tile division or precinct division, and background data is obtained. This encoding processing method is characterized in that the division boundary of the mask and the division boundary of the mask data are made to coincide.

  The invention of claim 17 is the encoding processing method according to the invention of claim 12 or 13, wherein, in the encoding step, the background data is divided into a plurality of area portions, and each area portion is individually encoded, In this encoding method, the division boundary of the background data and the division boundary of the mask data are matched.

  The invention according to claim 18 is the encoding processing method according to claim 14 or 15, wherein, in the encoding step, the background data is subjected to JPEG2000 encoding using tile division or precinct division, and background data is obtained. This encoding processing method is characterized in that the division boundary of the mask and the division boundary of the mask data are made to coincide.

  The invention of claim 19 is the encoding processing method according to the invention of claim 14 or 15, wherein in the encoding step, the background data is divided into a plurality of area portions, and each area portion is individually encoded, In this encoding method, the division boundary of the background data and the division boundary of the mask data are matched.

  The invention of claim 20 or 21 makes it possible to implement the encoding processing apparatus according to the invention of claims 1 to 9 using a computer. That is, the invention of claim 20 is a program for causing a computer to function as each means of the encoding processing apparatus according to any one of claims 1 to 9. The invention of claim 21 is a computer-readable information recording medium in which a program for causing a computer to function as each means of the encoding processing device according to any one of claims 1 to 9 is recorded. is there.

  As described above, according to the first to nineteenth aspects, encoded data with high partial accessibility can be obtained. That is, according to the first, second, twelfth and thirteenth aspects, partial access to the mask data is facilitated. According to the invention of claim 3, partial access to the mask data is facilitated and the coding efficiency of the mask data can be increased. According to the fourth, fifth, fourteenth and fifteenth inventions, partial access to mask data and foreground data is facilitated. According to the sixth, seventh, sixteenth and seventeenth aspects, partial access to the mask data and the background data is facilitated. According to the eighth, ninth, eighteenth, and nineteenth aspects, partial access to all of the mask data, foreground data, and background data is facilitated. Furthermore, according to the invention of claim 10, it is possible to improve the partial accessibility to the foreground data and the mask data when there are a plurality of pairs of foreground data and mask data. According to the eleventh aspect of the present invention, the code amount is not increased, and the troublesomeness of dividing the foreground data into a plurality of separate images can be avoided.

  Further, by using the program or the information recording medium according to the inventions of claims 20 and 21, the encoding processing device according to the inventions of claims 1 to 9 can be easily realized using a computer.

  FIG. 7 is a block diagram for explaining an embodiment of the present invention. The encoding processing apparatus shown here includes background data obtained by disassembling original image data, data acquisition means 100 that acquires at least one pair of foreground data and mask data, background data acquired by the data acquisition means 100, and foreground data. The encoding means 110 which encodes mask data separately is provided. Since the background data code, foreground data code and mask data code generated by the encoding means 110 are inconvenient to handle independently, they are usually combined into one file of a predetermined format. is there. Therefore, also in the encoding processing apparatus according to the present embodiment, the JPM format is obtained by combining the background data code, the foreground data code and the mask data code generated by the encoding unit 110 and adding a necessary header. Alternatively, a code forming unit 111 that forms encoded data (file) in the MRC format and a code output unit 112 that outputs the encoded data to an external device or writes it to an internal storage device are provided.

  The data acquisition unit 100 according to this embodiment includes a unit 101 for inputting original image data from the outside, the original image data input by the unit 101 as background data, one or more “pairs of foreground data and mask data”. Means 102 for disassembling. Further, the data acquisition means 100 includes means 103 for inputting, from the outside, background data obtained by disassembling the original image data in advance, one or a plurality of foreground data and mask data pairs. Further, the data acquisition means 100 is a means 104 for inputting JPM or MRC encoded data from the outside, and decodes the encoded data input by this means to obtain background data, one or more foreground data and mask data. A means 105 for generating a pair is included. The data acquisition means 100 can take any of a form including only the means 101 and 102, a form including only the means 103, and a form including only the means 104 and 105, and both forms are included in the present invention. Is done.

  FIG. 7 also shows an embodiment of the encoding method according to the present invention, where the data acquisition means 100 is in the data acquisition process, the encoding means 110 is in the encoding process, and the code forming means 111 is in the code forming process. The code output means 112 corresponds to the code output process. The data acquisition process can take any of the three forms as described above.

  Such an encoding processing apparatus or method according to the present invention uses a computer having a general configuration such that a CPU 200, a memory 201, a hard disk device 202, and the like as shown in FIG. It can be realized by one or more programs (application program, device driver, etc.).

  Such a program, that is, a program that causes a computer to function as the data acquisition unit (or step) 100, the encoding unit (or step) 110, the code formation unit (or step) 111, and the code output unit (or step) 112, Usually, it is stored in the hard disk device 202, loaded into the memory 201 when necessary, and executed by the CPU 200, whereby the coding processing device according to the present invention is constructed on the computer, or the coding according to the present invention. A processing method is implemented. Such a program and various information recording (storage) media that can be read by a computer, such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor storage element, on which the program is recorded are also included in the present invention.

  When the means 101 and 102 of the data acquisition means 100 are used, the processing in the computer has the following flow, for example. (1) Original image data stored in the hard disk device 202 is read into the memory 201 by a command from the CPU 200. (2) The CPU 200 reads the original image data on the memory 201, generates background data, one or a plurality of foreground data and mask data pairs, encodes each data, and combines the generated codes with the JPM format or MRC format. (3) The encoded data is written in another area of the memory 201. (4) Encoded data generated on the memory 201 in accordance with an instruction from the CPU 200 is stored in the hard disk device 202.

  A typical format of MRC encoded data (code file) is as shown in FIG. That is, the encoded data of MRC is composed of an entire header indicating that it is an MRC code, one background code and its header, one or more “pairs of foreground code and mask code” and its header. The

  As described above, JPM is an MRC type encoding method that allows JPEG2000 as an encoding method for background data, foreground data, and mask data. The encoded data has the same format as MRC, and includes a header, It consists of a series of following codes.

  FIG. 10 shows a configuration example of JPM encoded data (code file). Since the dotted line portion is an option, a simple description will be given focusing on the solid line portion. In FIG. 10, “JPEG2000 Signature Box” is an overall header indicating that the code belongs to the JPEG2000 family. “File Type Box” is an entire header indicating that the code is in JPM format. “Compound Image Header Box” is an entire header including general information of the code. “Page Collection box” is a table of contents indicating the order of each page when the code consists of multiple pages. “Page Box” is an overall header indicating the page resolution and the like. Here, the page is a canvas for sequentially superimposing (combining) images, and has the same size as the image after the composition is finished. In the case of JPM, “layout object (layout object)” composed of a pair of foreground and mask is sequentially drawn on the page. “Layout Object box” is a header for the foreground and the mask indicating the size and position of the foreground and the mask. The “Media Data box” and “Contiguous Codestream box” are portions including foreground and mask codes. In JPM, the background (BasePage) is handled as an initial page before the layout object is drawn.

  Hereinafter, embodiments of the present invention will be described in detail with reference to several examples. Unless otherwise specified, in the data acquisition means 100, the original image data is input by the means 101, and the original image data is input by the means 102 as background data and one or more layout objects (a pair of foreground data and mask data). ).

  In this embodiment, the original image data is decomposed into background data, a pair of foreground data 1 and mask data 1, and a pair of foreground data 2 and mask data 2 as schematically illustrated in FIG.

  FIG. 12 is a schematic flowchart for explaining the flow of processing in the present embodiment. In the figure, steps 1100 to 1105 are processing steps by the means 102 in the data acquisition means 100, steps 1106 to 1108 are processing steps by the encoding means 110, and step 1109 is a processing step by the code forming means 111. is there. In other words, the means 102 includes means corresponding to steps 1100 to 1105, and the encoding means 110 includes means corresponding to steps 1106 to 1108.

  First, in step 1100, the input original image data is divided into four tiles (see FIG. 11). Note that the number of tile divisions can be increased or decreased. In this embodiment, since the tile is the minimum area that can be divided and accessed for the encoded data, the number of tile divisions may be selected from the size of the minimum area and the size of the original image data.

  In the next step 1101, it is determined whether each pixel of the original image data is a pixel (character pixel) constituting a character (line drawing) or other pixel (non-character pixel). A binary mask data 2 having the same size as the size of the original image data is created by setting the corresponding position value to 1 and the position values corresponding to the other pixels to 0. The pixels of the mask data 2 and the pixels of the original image data have a one-to-one correspondence, and the mask data 2 is tiled at the same division boundary as the original image data (see FIG. 11).

  Here, discrimination between character pixels and other pixels can be performed by a known image area discrimination method. In this embodiment, for example, the following method is used. First, a Sobel filter known as an edge detection operator is applied to each pixel of the original image data. That is, 3 × 3 pixels centered on the target pixel are multiplied by the first weight matrix (Soble operator) shown in FIG. 13, and the sum HS is calculated. Similarly, the second weight matrix shown in FIG. And the sum VS is calculated. Then, the square root √ (HS ^ 2 + VS ^ 2) of the square sum of HS and VS is set as the output value of the filter for the pixel of interest. If the filter output value is equal to or greater than a predetermined threshold th (for example, 30), the pixel of interest is determined to be a character pixel, and 1 is set to the corresponding pixel position of the mask data, otherwise 0 is set to the corresponding pixel position. . Mask data 2 is created by repeating the same processing procedure for all pixels. The character pixel / non-character pixel discrimination result in this step is also used in steps 1102 and 1103.

  In the next step 1102, multi-value foreground data 2 is created by replacing the color of the non-character pixel of the original image data with the color of the character pixel located closest to the non-character pixel. However, the original image data itself is stored. The foreground data 2 has the same size as the original image data, and the pixels correspond one-to-one. The foreground data 2 is also tiled at the same dividing boundary as the original image data (see FIG. 11).

  In the next step 1103, multivalued foreground data 1 is created by replacing the color of the character pixel of the original image data with the color of the non-character pixel located closest to the character pixel. The foreground data 1 has the same size as the original image data, and the pixels correspond one-to-one. The foreground data 1 is also tiled at the same division boundary as the original image data (see FIG. 11).

  In the next step 1104, binary mask data 1 having the same size as the original image data and all pixel values set to 1 is created. The mask data 1 also has a form in which the original image data and the pixels have a one-to-one correspondence and are tiled at the same division boundary as the original image data (see FIG. 11).

  In the next step 1105, multi-value background data in which all pixel values having the same size as the original image data are set to 0 (white) is created.

  Thus, the background data (for reproducing the original image data), the foreground data 1 and mask data 2 pair, and the foreground data 1 and mask data 2 pair obtained by decomposing the original image data are acquired.

  In the next step 1106, JPEG2000 encoding using tile division is performed on the multivalued foreground data 1 and foreground data 2. As schematically shown in the upper part of FIG. 15, each foreground data is divided into four tiles and encoded, but each foreground data is handled as one image data, and thus is generated for each foreground data. There is one code. By such tile division, the codes of the foreground data 1 and 2 can partially access the upper left, upper right, lower left, and lower right of the image. In this encoding, wavelet transform up to decomposition level 3 is performed, but the number of decomposition levels can be increased or decreased.

  In the next step 1107, JPEG2000 encoding using tile division is performed on the binary mask data 1 and mask data 2. As schematically shown in the lower part of FIG. 15, each mask data is divided into four tiles and encoded, but each mask data is handled as one image data, and thus is generated for each mask data. There is one code. By such tile division, the codes of the mask data 1 and 2 can partially access the upper left, upper right, lower left, and lower right of the image. In this encoding, wavelet transform is not performed (decomposition level = 0). However, wavelet transformation can also be performed.

  In the next step 1108, the background data is encoded according to the JPM specification (entropy encoding is not performed, and the background color is designated as a code).

  In the final step 1109, the foreground data 1 and 2, the mask data 1 and 2 and the background code are combined, and a necessary header is added to form JPM format encoded data.

  As described above, in the present embodiment, the tile boundary of the foreground 1 and mask 1 pair (layout object 1), that is, the dividing boundary is the same, and the foreground 1 and mask 1 pair (layout object 1). In addition, the division boundary also matches in the foreground 2 and mask 2 pair (layout object 2).

  In this embodiment, the original image data is tile-divided at step 1100, but tile division may be performed at encoding steps 1106 and 1107, and such a mode is naturally included in the present invention.

  In other words, in the data acquisition unit 100, when the background, foreground, and mask obtained by decomposing the original image data by the unit 103 are directly input, or when encoded data is input by the unit 104 and decoded by the unit 105 In this case, the processing in steps 1101 to 1105 is not necessary.

  Also in this embodiment, the original image data is decomposed into background data 1, foreground data 1, 2 and mask data 1, 2 as schematically shown in FIG. 11, but precinct division is used instead of tile division.

  Since the overall flow of processing in the present embodiment is the same as that in the first embodiment, a description will be given with reference to the flowchart of FIG.

  In this embodiment, the original image data tile division process (step 1100) is not performed.

  Mask data 2 creation processing (step 1101), foreground data 2 creation processing (step 1102), foreground data 1 creation processing (step 1103), and mask data 1 creation processing (step 1104) are not tile-divided. Except for this, the second embodiment is the same as the first embodiment. Background data creation processing (step 1105) is the same as in the first embodiment.

  In JPEG2000 encoding of the foreground data 1 and 2 in step 1106, tile division is not performed, and precinct division as schematically illustrated in the upper part of FIG. 16 is performed (that is, the subband is divided into four precincts). . By such precinct division, the foreground data 1 and 2 can be partially accessed in the upper left, upper right, lower left, and lower right of the image.

  Further, in JPEG2000 encoding of the mask data 1 and 2 in step 1107, tile division is not performed, and precinct division as schematically illustrated in the lower part of FIG. 16 is performed (that is, the subband is divided into four precincts). ) By such precinct division, the codes of the mask data 1 and 2 can partially access the upper left, upper right, lower left, and lower right of the image.

  Steps 1108 and 1109 are the same as those in the first embodiment.

  In this embodiment, as can be understood from FIG. 16, the precinct boundary, that is, the division boundary is the same for all of the foreground data 1 and 2 and the mask data 1 and 2.

  FIG. 17 is a schematic flowchart for explaining the flow of processing in the present embodiment. In the figure, steps 1600 to 1605 are processing steps by the means 102 in the data acquisition means 100, steps 1606 to 1608 are processing steps by the encoding means 110, and step 1609 is a processing step by the code forming means 111. is there. In other words, the means 102 includes means corresponding to steps 1600 to 1605, and the encoding means 110 includes means corresponding to steps 1606 to 1608.

  The processing contents of steps 1600-1605 are the same as the corresponding steps 1100-1105 in FIG.

  In step 1606, JPEG2000 encoding using tile division is performed on the multivalued foreground data 1 and foreground data 2. As schematically shown in the upper part of FIG. 18, each foreground data is divided into four tiles and encoded, but each foreground data is handled as one image data, and thus is generated for each foreground data. There is one code. By such tile division, the codes of the foreground data 1 and 2 can partially access the upper left, upper right, lower left, and lower right of the image. In this encoding, wavelet transform up to decomposition level 3 is performed, but the number of decomposition levels can be increased or decreased.

  In step 1607, the mask data 1 and 2 are each divided into four area portions (images 0, 1, 2, and 3) at the same position as the tile boundary, as schematically shown in the lower part of FIG. Encode parts individually (as independent images) with JBIG. Therefore, four codes are generated for the mask data 1 and four codes are generated for the mask data 2. Thus, with respect to the codes of the mask data 1 and 2, partial access can be made to the upper left, upper right, lower left and lower right of the image.

  In step 1608, the background data is encoded according to the JPM specification (entropy encoding is not performed and a background color is designated as a code).

  In the last step 1609, the codes of the foreground data 1 and 2, mask data 1 and 2 and background data are combined, and a necessary header is added to form encoded data in JPM format. However, in this embodiment, since each mask data is encoded as four independent images, the structure of the layout object is different from those of the first and second embodiments.

  That is, the four area portions (divided images) of the mask data 1 constitute four layout objects by using the tiled foreground data 1 as shared data defined by JPM. The four divided images constitute four layout objects using the foreground data 2 as shared data (claim 11).

  In the case of JPM, an ID is assigned to each layout object, and this ID is described in “Layout Object Header box” shown in FIG. As described above, layout objects are in principle “pairs”. However, if “Separate objects for image and mask components” is specified as “Layout Object Style”, foreground for layout objects with the same ID And mask can be handled separately (that is, an ID is assigned only to the foreground).

  In the “Shared Data Entry box” that is the header of the entire JPM file, the ID of the shared layout object (in this example, the foreground) is entered, and the side of the layout object that wants to share the foreground is “Shared Data Entry box”. Use the “Reference box” to specify the foreground ID and realize sharing.

  In this embodiment, as can be understood from FIG. 18, the division boundaries coincide for all of the foreground data 1 and 2 and the mask data 1 and 2.

  It is also possible to omit step 1600 and perform tile division in step 1606, and such an aspect is also included in the present invention.

  In the present embodiment, a mode in which tile division of foreground data is not performed is possible, and this is also included in the present invention.

  In other words, in the data acquisition unit 100, when the background, foreground, and mask obtained by decomposing the original image data by the unit 103 are directly input, or when encoded data is input by the unit 104 and decoded by the unit 105 Therefore, the processing of steps 1601 to 1605 is not necessary.

  Since the overall flow of the process in the present embodiment is the same as that in the third embodiment, a description will be given with reference to the flowchart of FIG. 17 relating to the third embodiment.

  In this embodiment, the original image data tile division process (step 1600) is not performed.

  Mask data 2 creation processing (step 1601), foreground data 2 creation processing (step 1602), foreground data 1 creation processing (step 1603), and mask data 1 creation processing (step 1604) are not tile-divided. Except for this, the third embodiment is the same as the third embodiment. Background data creation processing (step 1605) is the same as in the third embodiment.

  In JPEG2000 encoding of the foreground data 1 and 2 in step 1606, precinct division as schematically shown in the upper part of FIG. 19 is performed instead of tile division (that is, the subband is divided into four precincts). . By such precinct division, the foreground data 1 and 2 can be partially accessed in the upper left, upper right, lower left, and lower right of the image.

  In step 1607, the mask data 1 and 2 are each divided into four independent area portions (images 0, 1, 2, and 3) as shown in the lower part of FIG. That is, an independent image is encoded by JPIG. The division boundary of the mask data coincides with the precinct boundary of the foreground data 1 and 2. Therefore, the codes of the foreground data 1 and 2 and the mask data 1 and 2 can partially access the upper left, upper right, lower left, and lower right of the image.

  Steps 1608 and 1609 are the same as those in the third embodiment.

  In this embodiment, a mode in which the precinct division of the foreground data is not performed is possible, and this is also included in the present invention.

<Other embodiments>
In the third and fourth embodiments, the foreground data can also be divided into a plurality of area portions, and each area portion can be individually encoded, and such an aspect is also included in the present invention.

  In the first to fourth embodiments, the sizes and boundaries of the foreground and the divided areas of the mask all match. However, it is also possible to make the division sizes different while matching the division boundary between the foreground and the mask.

For example, according to another embodiment of the present invention, the size of the divided area of the mask (or foreground) is an integral multiple of the size of the divided area of the foreground (or mask). That is, as schematically illustrated in FIG. 20, the foreground is divided into 8 tiles and the mask is divided into 4 tiles. As can be seen in this example, the image is divided at a boundary where partial access is possible, but the foreground and mask divided regions need not have the same size or number of divisions. Although an example of tile division is given here, a mode in which precinct division is performed is also possible, and a mode in which the divided areas of the mask are treated as independent images as in the third and fourth embodiments is also possible. is there. Since the overall processing flow of such an embodiment may be the same as that of Embodiment 1 or 3, the description thereof is omitted.
In the above description, there is one mask image, which is divided into a plurality of parts, and each region is encoded independently. However, there is also an aspect in which a plurality of adjacent (or overlapping) regions are assumed as mask images from the beginning, the plurality of regions are individually encoded, and the same foreground is shared among the regions of the plurality of mask images. This is the intention of the invention according to Item 11.

In Examples 1 to 4, the mask covered the entire page. However, as apparent from the role of the mask, it is sufficient that the mask exists only in a place necessary for drawing character pixels, for example, and it is not always necessary to cover the entire page (the entire MRC image). Therefore, according to another embodiment of the present invention, as shown in the upper part of FIG. 21, foreground is divided into 4 tiles and JPEG2000 encoded, but a mask is provided for each area corresponding to each tile area. The image is divided into four images as shown in the lower part of FIG. 21 that cover a place necessary for drawing character pixels, and each divided image is individually JBIG-encoded. Since the overall processing flow of this embodiment may be the same as that of the third embodiment, the description thereof is omitted.
As described above, an aspect in which a plurality of non-overlapping regions are assumed as mask images from the beginning, the plurality of regions are individually encoded, and the same foreground is shared between the mask images is also the invention according to claim 11. It is the intention of

  In the embodiment described above, JPM with a single background is taken as an example. However, when the background is not a single color, it is effective to divide and encode the background so that the background can be partially accessed. Is as described above in relation to the inventions of claims 8 and 9. For example, the original image data corresponds to the foreground, mask, and background shown in FIG. 1 (the background corresponding to the foreground 1 in FIG. 11, the mask corresponding to the mask 2 in FIG. 11, and the foreground 2 in FIG. 11). Foreground), and each can be divided and encoded in the same manner as in the previous embodiment. Examples of the division encoding are schematically shown in FIGS.

  FIG. 22 schematically illustrates an example in which JP2000 encoding is performed using tile division in which the division boundaries are matched with respect to the foreground, the mask, and the background. FIG. 23 schematically shows an example in which the foreground, the mask, and the background are precinct divided and encoded by JPEG2000. In FIG. 24, the foreground and the background are divided into tiles, encoded with JPEG2000, and the mask is divided into four area portions (images 0, 1, 2, and 3) that match the tile boundaries. An example of encoding with is schematically shown. FIG. 25 schematically shows an example in which the foreground and the background are precinct-divided and encoded by JPEG2000, the mask is divided into area portions that match the precinct boundary, and each area portion is individually JBIG-encoded.

  Since the process in such an aspect can be easily inferred from Examples 1 to 4, the description thereof is omitted.

  Moreover, since it is clear that the above description is also the description of the embodiment of the encoding processing method according to the present invention, the description of the encoding processing method will not be repeated.

It is a figure explaining the original image in MRC, a foreground, a mask, and a background. It is a block diagram for demonstrating the flow of the encoding process of JPEG2000. It is a figure which shows the relationship between the image in JPEG2000, a tile, a precinct, and a code block. It is a figure which shows the example of the subband division | segmentation in JPEG2000, and the resolution level of each subband. It is a figure which illustrates the packet in which the layer division | segmentation in JPEG2000 and a layer are contained. It is a figure which shows the progression order of JPEG2000. It is a block diagram for demonstrating embodiment of this invention. It is a block diagram for demonstrating the flow of a process in the case of implementing this invention using a computer. It is a figure for demonstrating the format of MRC encoding data. It is a figure for demonstrating the format of JPM encoded data. It is a figure which shows the example divided | segmented into the background which divided | segmented original image data, the pair of the mask 1 and the foreground 1, and the pair of the mask 2 and the foreground 2. It is a flowchart which shows the flow of the process in Example 1 of this invention. It is a figure which shows a Sobel operator. It is a figure which shows a Sobel operator. FIG. 5 is a diagram schematically illustrating foreground and mask division coding in the first embodiment. It is a figure which shows typically the foreground and the division | segmentation encoding of a mask in Example 2 of this invention. It is a flowchart which shows the flow of the process in Example 3 of this invention. It is a figure which shows typically the division | segmentation encoding of a foreground and a mask in Example 3. FIG. It is a figure which shows typically the foreground and the division | segmentation encoding of a mask in Example 4 of this invention. It is a figure which shows typically the example of the division | segmentation encoding of a foreground and a mask. It is a figure which shows typically the example of the division | segmentation encoding of a foreground and a mask. It is a figure which shows typically the example of the division | segmentation encoding of a foreground, a mask, and a background. It is a figure which shows typically the example of the division | segmentation encoding of a foreground, a mask, and a background. It is a figure which shows typically the example of the division | segmentation encoding of a foreground, a mask, and a background. It is a figure which shows typically the example of the division | segmentation encoding of a foreground, a mask, and a background.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Data acquisition means 110 Encoding means 111 Code formation means 112 Code output means

Claims (21)

Background data obtained by decomposing the original image data, data acquisition means for acquiring at least one pair of foreground data and mask data, and encoding for encoding the background data, foreground data and mask data acquired by the data acquisition means, respectively Means,
An encoding processing apparatus characterized in that the encoding means performs JPEG2000 encoding using tile division or precinct division on mask data. Background data obtained by decomposing the original image data, data acquisition means for acquiring at least one pair of foreground data and mask data, and encoding for encoding the background data, foreground data and mask data acquired by the data acquisition means, respectively Means,
In the encoding means, the mask data is divided into a plurality of area portions, and each area portion is individually encoded.   The wavelet transform is not applied when the mask data is encoded when the absolute value of each pixel position of the mask data takes only a power of 0 or 2 in the encoding means. Encoding processing device.   3. The encoding unit according to claim 1, wherein the encoding unit performs JPEG2000 encoding using tile division or precinct division on the foreground data, and matches the division boundary of the foreground data and the division boundary of the mask data. Encoding processing device.   3. The encoding means according to claim 1, wherein the foreground data is divided into a plurality of area portions, each area portion is individually encoded, and the division boundary of the foreground data and the division boundary of the mask data are matched. The encoding processing apparatus described.   3. The encoding unit according to claim 1, wherein the encoding unit performs JPEG2000 encoding using tile division or precinct division on the background data to match the division boundary of the background data and the division boundary of the mask data. Encoding processing device.   3. The coding means according to claim 1, wherein the encoding means divides the background data into a plurality of area portions, individually encodes each area portion, and matches the division boundary of the background data and the division boundary of the mask data. The encoding processing apparatus described.   6. The encoding unit according to claim 4, wherein the encoding means performs JPEG2000 encoding using tile division or precinct division on the background data, and matches the division boundary of the background data and the division boundary of the mask data. Encoding processing device.   6. The encoding means according to claim 4, wherein the background data is divided into a plurality of area portions, each area portion is individually encoded, and the division boundary of the background data is matched with the division boundary of the mask data. The encoding processing apparatus described.   6. The encoding processing apparatus according to claim 4, wherein there are a plurality of pairs of foreground data and mask data, and the division boundaries of the foreground data of all pairs and the division boundaries of the mask data are made to coincide. Background data obtained by decomposing the original image data, data acquisition means for acquiring at least one pair of foreground data and mask data, and encoding for encoding the background data, foreground data and mask data acquired by the data acquisition means, respectively Means, and code forming means for combining the codes of the background data, foreground data and mask data generated by the encoding means to form encoded data of a predetermined format,
In the encoding means, the mask data consists of a plurality of partial areas, and each of the partial areas is individually encoded,
An encoding processing apparatus, wherein the code forming means shares the code of foreground data among codes of a plurality of partial areas of mask data. Background data obtained by decomposing the original image data, a data acquisition step for acquiring at least one foreground data and mask data pair, and encoding for encoding the background data, foreground data, and mask data acquired by the data acquisition step, respectively A process,
In the encoding step, JPEG2000 encoding using tile division or precinct division is performed on mask data. Background data obtained by decomposing the original image data, a data acquisition step for acquiring at least one foreground data and mask data pair, and encoding for encoding the background data, foreground data, and mask data acquired by the data acquisition step, respectively A process,
In the encoding step, the mask data is divided into a plurality of area portions, and each area portion is individually encoded.   14. The encoding process according to claim 12 or 13, wherein in the encoding step, JPEG2000 encoding using tile division or precinct division is performed on the foreground data to match the division boundary of the foreground data and the division boundary of the mask data. Encoding method.   The foreground data is divided into a plurality of area portions in the encoding step, each area portion is individually encoded, and the foreground data division boundary and the mask data division boundary are made to coincide with each other. The encoding processing method as described.   14. The encoding process according to claim 12 or 13, wherein in the encoding step, JPEG2000 encoding using tile division or precinct division is performed on the background data to match the division boundary of the background data and the division boundary of the mask data. Encoding processing method.   14. The encoding process according to claim 12, wherein the background data is divided into a plurality of area portions, each area portion is individually encoded, and the division boundary of the background data is matched with the division boundary of the mask data. The encoding processing method as described.   16. The encoding process according to claim 14 or 15, wherein, in the encoding step, JPEG2000 encoding using tile division or precinct division is performed on the background data to match the division boundary of the background data and the division boundary of the mask data. Encoding processing method.   16. In the encoding step, the background data is divided into a plurality of area portions, each area portion is individually encoded, and the division boundary of the background data and the division boundary of the mask data are matched. The encoding processing method as described.   A program that causes a computer to function as each unit of the encoding processing device according to any one of claims 1 to 9.   10. A computer-readable information recording medium on which a program for causing a computer to function as each unit of the encoding processing apparatus according to claim 1 is recorded.

Images