JP4590986B2 - Encoding device, decoding device, encoding method, decoding method, and program thereof - Google Patents

Description

  The present invention relates to an encoding device and a decoding device to which a predictive encoding method is applied.

As a method of encoding paying attention to data autocorrelation, there are, for example, run length encoding, JPEG-LS, LZ encoding (Ziv-Lempel encoding), and the like. In particular, in the case of image data, since neighboring pixels have a high correlation, it is possible to encode image data at a high compression rate by paying attention to this point.
Further, Patent Document 1 calculates the difference image data between the frames by paying attention to the correlation between the frames constituting the moving image, the calculated difference image data, and the input image data (frame image). An image data compression apparatus that selectively compresses and encodes the image is disclosed.
Japanese Patent Laid-Open No. 11-313326

  The present invention has been made from the above-described background, and decodes encoded data encoded by an encoding device that efficiently encodes an input image using correlation between images or encoded data by this encoding device. It is an object of the present invention to provide a decoding device that can be realized.

[Encoder]
In order to achieve the above object, an encoding apparatus according to the present invention is an encoding apparatus that encodes a target image, which is a layer image constituting an input image, and is a pixel of interest included in one layer image. When encoding a value, first prediction means for calculating a first predicted value based on the pixel value of a pixel included in another layer image corresponding to the target pixel, and the same as the target pixel A second prediction unit that calculates a second predicted value based on a pixel value of another pixel included in the layer image; and another layer based on the first predicted value and the second predicted value. Reference information generating means including selection information for selecting reference information indicating a position of a pixel included in an image or reference information indicating a position of another pixel included in the same layer image, and as code data of the target pixel, The reference selected by the selection means And a code generating means for generating information of the code data, the layer images forming the input image, the position by the reference information selected by the pixel value and the selection means of the pixel of interest is other among the pixels indicated A transparent region that is a region in which pixel values of pixels included in the layer image coincide with each other, and a non-transparent region that is a region other than the transparent region, and the code generation unit includes: The pixel value of the target pixel is used as code data of the target pixel depending on whether the target pixel and other pixels included in the same layer image are in the transparent region or the non-transparent region .

  According to the encoding apparatus of the present invention, an input image can be efficiently encoded using the correlation between the input image and another reference image.

[Background and outline]
First, in order to help understanding of the present invention, its background and outline will be described.
For example, in the predictive encoding method such as the LZ encoding method, the prediction data is generated by referring to the pixel value at the predetermined reference position, and the generated prediction data matches the image data of the target pixel. The reference position of the predicted data (hereinafter referred to as reference information) is encoded as the code data of the target pixel. Therefore, a higher compression rate can be expected as the matching frequency (target ratio) of predicted data is higher. Therefore, in the predictive coding method, the compression efficiency varies greatly depending on where the reference position is set. Generally, since the correlation is high in the pixel group in the vicinity, the reference position is set to a pixel (on the same image) in the vicinity of the target pixel.
In JPEG-LS (irreversible mode) and the like, the pixel value of the succeeding pixel is replaced with the pixel value determined by the preceding pixel, so that the hit rate of the prediction data is further increased and the compression rate is increased. We are trying to improve.

Some input images to be encoded constitute a plurality of image groups that are correlated with each other. For example, a plurality of frame images constituting a moving image almost coincide with each other in a non-moving image region, and even in a moving image region, it can be said that there is a certain degree of correlation if the direction of movement and the amount of movement are taken into account.
Therefore, when encoding an input image (target image) to be encoded, the image processing apparatus according to the present embodiment generates prediction data with reference to at least another reference image (for example, another frame image). Then, predictive encoding processing using the generated prediction data is performed. That is, the image processing apparatus encodes reference information for another reference image as at least a part of code data of the target image.
In addition, when decoding the code data generated in this way, the present image processing apparatus refers to another reference image according to the code data, and generates a decoded image using the image data included in the reference image. To do.

In the method described in Patent Document 1, when the current frame to be encoded is encoded, a difference image from the previous frame (reference image) is generated.
FIG. 1 is a diagram for explaining a difference between an encoding method involving generation of a difference image and the encoding method in the present embodiment. FIG. 1A shows a difference image between a previous frame and a current frame. For example, FIG. 1B illustrates a reference position that is referred to when predictive data is generated in the present embodiment.
As illustrated in FIG. 1A, the difference image between the previous frame (reference image) and the current frame is configured with a difference value calculated by comparing pixels belonging to each frame with respect to all the pixels. Therefore, the difference value becomes 0 in the non-moving part, but the difference value exists in the moving part and can be various values. That is, the difference image has different pixel values at least between the moved part and the non-moved part. Therefore, discontinuity of pixel values occurs in the difference image, which hinders improvement of the compression rate.
On the other hand, as illustrated in FIG. 1B, the image processing apparatus according to this embodiment includes reference pixels A to D on the same image as the target pixel X, and the like, as illustrated in FIG. Reference pixel E on the image (reference image) is referred to. Then, the image processing apparatus selects any reference pixel (A to E) having a certain relationship with the target pixel X, and generates prediction data (reference information) based on the pixel value of the selected reference pixel. To do. In other words, the present image processing apparatus does not apply the pixel values of other images (previous frame) uniformly, but applies the pixel values of other images only when advantageous from the viewpoint of the compression rate, and performs high compression. Realize the rate.

Next, an example of an image group suitable for application of the present invention will be described.
FIG. 2 is a diagram illustrating image data managed in a layer structure, and illustrates a mode in which the present invention is applied to image data having a layer structure.
As shown in FIG. 2, the image data 700 includes a plurality of mask layers 710 (710a and 710b) and one image layer 720. Hereinafter, a method for managing image data with such a layer structure is referred to as a multi-mask method.
The mask layer 710 is a layer to which an image element with little gradation change (for example, 16 gradations or less) such as a character image or a simple CG (Computer Graphics) image is assigned. In this example, a binary image element (object) is assigned. Assigned. Therefore, the image elements (objects) included in each mask layer 710 are composed of a single color and expressed in two gradations.
The image layer 720 is a layer to which an image element (object) having a larger number of gradations than the mask layer 710 such as a photographic image is assigned. In this example, an image element (object) having multiple values (16 gradations or more) is assigned. It has been. For example, the image layer 720 includes a complex CG image or a continuous tone image. Here, the continuous tone image is an image in which a sufficiently continuous gradation is expressed in view of human visual characteristics, for example, an image expressed in 16 gradations or more per color.
The display image 750 is displayed or printed by overlaying the mask layer 710 on the image layer 720 in a preset order.

As described above, when a plurality of objects constituting one output image are assigned to a plurality of layers (mask layer and image layer) and managed independently according to the attribute, editing can be performed for each layer or Advantages such as being able to apply a corresponding compression method for each layer can be obtained.
The present invention can be applied to image data having such a layer structure. For example, when encoding any one layer as a target layer, the image processing apparatus according to the present invention refers to image data of another layer and encodes the image data of the target layer. Further, when decoding the code data generated in this way, the present image processing apparatus generates a display image 750 by referring to another layer according to the code data. Details will be described later.

3A is a diagram illustrating a three-dimensional cross-sectional image, and FIG. 3B is a diagram illustrating an image subjected to concealment processing by applying the present invention.
As illustrated in FIG. 3A, the three-dimensional shape has a plurality of cross-sectional images, and these cross-sectional images often have a high correlation with each other.
Therefore, the present invention can be applied to the encoding process of these cross-sectional images. For example, when an image processing apparatus according to the present invention encodes one cross-sectional image as a cross-sectional image of interest, the image data of the cross-sectional image of interest is encoded with reference to another cross-sectional image as a reference image. In addition, when decoding the code data generated in this way, the image processing apparatus refers to another cross-sectional image according to the code data, and generates image data of a cross-sectional image (a target cross-sectional image).

  In addition, as illustrated in FIG. 3B, the image processing apparatus according to the present invention performs browsing control on an input image by encoding the input image with reference to a reference image including a noise region. Can do. For example, when an input image is encoded using a reference image in which a region corresponding to a region to be concealed (hereinafter referred to as a concealment region) is configured with noise, the concealment region of the input image is encoded with reference to noise. Therefore, prediction is performed with the pixel value of the reference image randomly (non-uniformly), and code data is generated based on the prediction data. Therefore, when this code data is decoded without using this reference image, the region corresponding to the noise region (the secret region) is decoded as a scrambled image. On the other hand, an area that is not concealed (non-confidential area) is encoded with reference to an area (reference image) that is uniformly filled with a predetermined pixel value (for example, the minimum value or the maximum value). Even if it is decoded without using, it is played back in a viewable state.

  As described above, the present invention can be widely applied to image groups having a correlation with each other. Hereinafter, a case where the present invention is applied to image data having a layer structure will be described in detail as a specific example.

[Hardware configuration]
Next, a hardware configuration of the image processing apparatus 2 in the first embodiment will be described.
FIG. 4 is a diagram illustrating a hardware configuration of the image processing apparatus 2 to which the encoding method and the decoding method according to the present invention are applied, centering on the control apparatus 20.
As illustrated in FIG. 4, the image processing apparatus 2 includes a control device 21 including a CPU 212 and a memory 214, a communication device 22, a recording device 24 such as an HDD / CD device, an LCD display device or a CRT display device, and a keyboard. A user interface device (UI device) 25 including a touch panel and the like is included.
The image processing apparatus 2 is, for example, a general-purpose computer in which an encoding program 5 (described later) and a decoding program 6 (described later) according to the present invention are installed as a part of a printer driver, such as a communication device 22 or a recording device 24. The image data is acquired via, and the acquired image data is encoded or decoded and transmitted to the printer apparatus 3.

[Encoding program]
FIG. 5 is a diagram illustrating a functional configuration of the first encoding program 5 which is executed by the control device 21 (FIG. 4) and implements the encoding method according to the present invention.
As illustrated in FIG. 5, the first encoding program 5 includes a plurality of intra-layer prediction units 510 (first intra-layer prediction unit 510a, second intra-layer prediction unit 510b, third intra-layer prediction unit 510c, and layers. A fourth prediction unit 510d), an interlayer prediction unit 520, a prediction error calculation unit 530, a run counting unit 540, a selection unit 550, and a code generation unit 560. The combination of the intra-layer prediction unit 510, the interlayer prediction unit 520, the prediction error calculation unit 530, the run counting unit 540, and the selection unit 550 is an example of the reference information generation unit according to the present invention.
In the encoding program 5, image data is input via the communication device 22 or the recording device 24. The input image data is rasterized in the previous stage of the encoding program 5.

  The intra-layer prediction unit 510 refers to the pixel value of a pixel (reference pixel) that is different from the target pixel, sets this pixel value as a predicted value, and compares the predicted value with the pixel value of the target pixel as a run counting unit. Output to 540. The first intra-layer prediction unit 510a to the fourth intra-layer prediction unit 510d in this example are the pixel values of the reference pixels A to D (FIGS. 1 and 6) and the pixel of the target pixel X (FIGS. 1 and 6), respectively. When the pixel values match with each other (that is, when the prediction is correct), a prediction unit ID (described later) for identifying itself is output to the run counting unit 540, and in other cases In addition, the fact that they did not match is output to the run counter 540. The intra-layer prediction unit 510 may be one or more types, and for example, only the first intra-layer prediction unit 510a that refers to the reference position A may be provided.

  The inter-layer prediction unit 520 refers to the pixel value of another image (reference image) different from the image to be encoded (target image), sets the pixel value of the reference image as a predicted value, and uses the predicted value and the attention The comparison result with the pixel value of the pixel (pixel included in the target image) is output to the run counting unit 540. The interlayer prediction unit 520 of this example compares the pixel value of the reference pixel E (FIGS. 1 and 6) included in the reference image with the pixel value of the target pixel X (FIGS. 1 and 6), Are matched (that is, when the prediction is correct), a prediction unit ID (described later) for identifying the self is output to the run counting unit 540. Output to the counting unit 540. The relative position of the reference pixel E in the reference image corresponds to the relative position of the target pixel X in the target image. For example, when the resolution of the target image matches the resolution of the reference image, the relative position is the same. That is, the reference pixel E is a pixel that overlaps the target pixel X when the target image and the reference image are overlapped.

  The prediction error calculation unit 530 predicts the pixel value of the target pixel by a predetermined prediction method, subtracts the prediction value from the actual pixel value of the target pixel, and performs the run counting unit 540 and the selection unit 550 as prediction error values. Output for. The prediction method of the prediction error calculation unit 530 only needs to correspond to the prediction method of a decoding program (described later) that decodes code data. In this example, the prediction error calculation unit 530 uses a pixel value at the same reference position (reference pixel A) as the first intra-layer prediction unit 510a as a prediction value, and this prediction value and an actual pixel value (pixel value of the target pixel X) ) Is calculated.

  The run counting unit 540 counts the number of consecutive identical prediction unit IDs, and outputs the prediction unit ID and the continuous number thereof to the selection unit 550. The prediction unit ID and the number of consecutive parts are examples of reference information for the target image and the reference image. For example, when a prediction error value is input, the run counting unit 540 outputs the prediction unit ID counted by the internal counter and its continuous number, and then the input prediction error value is directly selected by the selection unit 550. Output for.

  The selection unit 550 selects the longest continuous prediction unit ID based on the prediction unit ID, the continuous number, and the prediction error value input from the run counting unit 540, and the prediction unit ID, the continuous number, and the prediction error value. Is output to the code generation unit 560 as prediction data.

  The code generation unit 560 encodes the prediction unit ID, the number of continuations, and the prediction error value input from the selection unit 550, and outputs them to the communication device 22 or the recording device 24.

FIG. 6 is a diagram for explaining an encoding process performed by the encoding program 5. FIG. 6A illustrates the positions of pixels referred to by the intra-layer prediction unit 510 and the inter-layer prediction unit 520. 6B illustrates a code associated with each reference pixel, and FIG. 6C illustrates code data generated by the code generation unit 560.
As illustrated in FIG. 6A, the reference positions of the intra-layer prediction unit 510 and the inter-layer prediction unit 520 are set as relative positions with respect to the target pixel X. Specifically, the reference pixel A of the first intra-layer prediction unit 510a is set upstream of the target pixel X in the main scanning direction, and the reference pixels B to B of the second intra-layer prediction unit 510b to the fourth intra-layer prediction unit 510d. D is set on the main scanning line above the target pixel X (upstream in the sub-scanning direction). Further, the reference pixel E of the interlayer prediction unit 520 is set on another image (reference image) different from the target image including the target pixel X.

Further, as illustrated in FIG. 6B, priorities are set for the respective reference pixels A to E, and when the prediction is correct with a plurality of reference pixels, the run counter 540 (FIG. 5) increases the number of consecutive prediction unit IDs according to the set priority. From the viewpoint of encoding with a layer structure, it is desirable that the priority order of the interlayer prediction unit 520 is higher than the priority order of the intra-layer prediction unit 510. The priorities set in the plurality of intra-layer prediction units 510 are set according to the predictive value hit rate (probability that the pixel value of the reference pixel matches the pixel value of the target pixel X), It may be dynamically updated by an MRU (Most Recently Used) algorithm.
In addition, as illustrated in FIG. 6B, the code generation unit 560 associates the prediction unit (reference position) and the code with each other, and the code corresponding to the reference position where the pixel of interest X matches the pixel value. Is output. Note that the code associated with each reference position is, for example, an entropy code set according to the hit rate of each reference position, and has a code length corresponding to the priority order.

  In addition, when the pixel values continuously match at the same reference position, the code generation unit 560 encodes the continuous number counted by the run counting unit 540. Thereby, the code amount is reduced. In this way, as illustrated in FIG. 6C, the encoding program 5, when the pixel values match at any reference position, the code corresponding to the reference position and the pixel at the reference position If the pixel values do not match at any reference position, the difference (prediction error value) between the pixel value at the predetermined reference position and the pixel value of the target pixel X is encoded. Encode.

FIG. 7 is a diagram exemplifying layer-structured image data encoded by the encoding program 5 (FIG. 5). FIG. 7A illustrates a case where a mask layer 710a to which a CG image is allocated is encoded. FIG. 7B illustrates a case where the mask layer 710b to which the character image is assigned is encoded.
As illustrated in FIG. 7A, the encoding program 5 can generate code data of the mask layer 710a by processing the input image using an image obtained by removing the CG image from the input image as a reference image. That is, the encoding program 5 can encode the mask layer 710a to which the CG image is assigned by referring to an image obtained by removing the CG image from the input image. Of the encoded mask layer 710a, the hatched region is a region on which the prediction of the interlayer prediction unit 520 (FIG. 5) referring to the reference image has been hit, and is a sequence of codes corresponding to the interlayer prediction unit 520. Encoded.

  Similarly, as illustrated in FIG. 7B, when the encoding program 5 processes an input image using an image obtained by removing the character image from the input image as a reference image, the encoding layer 5 of the mask layer 710 b to which the character image is assigned. Code data can be generated. That is, the encoding program 5 can encode the mask layer 710a to which the character image is assigned by referring to an image obtained by removing the character image (font image) from the input image.

This reference image may be processed by the encoding program 5 as the input image. For example, the encoding program 5 uses the reference image illustrated in FIG. 7A as the input image, and processes an image obtained by removing the character image from the reference image (that is, only the image image) as the next reference image. Thereby, the encoding program 5 can generate the code data of the mask layer 710b to which the character image is assigned from the reference image illustrated in FIG. That is, the encoding program 5 performs encoding processing in series using images obtained by removing the objects in stages from the input image as reference images, thereby encoding a plurality of mask layers 710 as illustrated in FIG. Data can be generated. In this way, the code data of each layer generated by the serial encoding process has a sequential relationship, and when decoding, a predetermined order (that is, an order reverse to the generation order by the encoding process) Needs to be decrypted with
Further, the image layer 720 (FIG. 2) may be encoded by intra-layer prediction, or may be encoded by another encoding method such as JPEG.

  Further, when decoding the code data generated as described above, the image processing apparatus 2 refers to the pixel value at the corresponding reference position or predicts the error value in accordance with the code that is sequentially input. And the pixel value of the target pixel is reproduced.

FIG. 8 is a diagram for explaining processing for a boundary region between a transparent pixel and a non-transparent pixel. Note that the transparent pixel is a rectangular area represented as “transparent” in the figure, and is an area on which prediction has been made with reference to another layer (reference image). The code corresponding to the transparent pixel is information instructing to refer to another layer. In addition, the non-transparent pixel is a rectangular area represented as “colored” in the figure, and an area in which prediction (reference pixels A to D) in the same image is hit, or inter-layer prediction and intra-layer prediction. This is a region where none of them hit. The code | symbol of this non-transparent pixel becomes the information which instruct | indicates prediction within a layer, or the information which shows a prediction error.
As illustrated in FIG. 8, prediction error processing becomes a problem in a region where transparent pixels and non-transparent pixels are adjacent to each other. For example, at the target pixel X in this figure, neither intra-layer prediction (reference pixels A to D) nor inter-layer prediction (reference pixel E) hits, so a prediction error is calculated.
In order to be able to edit each layer independently, the calculation of the prediction error is preferably closed within the layer (target image). However, since the encoding program 5 in the present embodiment calculates the difference between the immediately left (reference pixel A) and the target pixel X as a prediction error, the target pixel X in FIG. The difference from the target pixel X is calculated.
Therefore, the encoding program 5 in the present embodiment performs exceptional processing in the calculation of the prediction error in the boundary region between the transparent pixel and the non-transparent pixel. For example, the encoding program 5 encodes the pixel value itself of the target pixel X in the boundary region between the transparent pixel and the non-transparent pixel. More specifically, the encoding program 5 encodes the pixel value of the target pixel X as it is when the immediately left pixel (reference pixel A) is a transparent pixel and the target pixel X is a non-transparent pixel. If the pixel immediately to the left (reference pixel A) is a non-transparent pixel and the target pixel X is a non-transparent pixel, the difference between the pixel values of the reference pixel A and the target pixel X is encoded as a prediction error. Turn into.
Further, the encoding program 5 may fill the transparent pixels with a predetermined pixel value (default value). As a result, the encoding program 5 encodes the difference value between the immediately left pixel (reference pixel A) and the target pixel X as a prediction error regardless of whether or not the boundary region is between the transparent pixel and the non-transparent pixel. it can. In this case, when the image processing apparatus 2 decodes the code corresponding to the prediction error, when the code immediately before this code corresponds to the transparent pixel (when the code corresponds to the reference pixel E), When the pixel value of the target pixel X is decoded by adding the default value and the prediction error value, and the code immediately before this code corresponds to a non-transparent pixel, the pixel value on the left and the predicted pixel value And the pixel value of the target pixel X is decoded.
The encoding program 5 may calculate the prediction error with reference to the nearest non-transparent pixel when calculating the prediction error. In this case, there is a high possibility that the target pixel X and the predicted value take close values, which is advantageous in terms of the compression rate.

  As described above, the image processing apparatus 2 according to the present embodiment performs the predictive coding process with reference to another layer different from the layer to be coded, thereby obtaining the image data managed in the layer structure. Can be encoded.

[Modification 1]
Next, a modification of the first embodiment will be described.
FIG. 9 is a diagram illustrating a functional configuration of the second encoding program 52. It should be noted that among the components in this figure, the same reference numerals are given to the components that are substantially the same as those shown in FIG.
As illustrated in FIG. 9, the second encoding program 52 has a configuration in which the first interlayer prediction unit 520 is replaced with a second interlayer prediction unit 522 that generates prediction data based on transparent pixel values. .
The second interlayer prediction unit 522 in the present modification acquires a value set as the transparent pixel value, compares the acquired transparent pixel value with the pixel value of the target pixel X, and if they match, The prediction unit ID corresponding to the interlayer prediction unit 522 is output to the run counting unit 540. In other words, the interlayer prediction unit 522 determines that the interlayer prediction has been correct when the preset transparent pixel value matches the pixel value of the target pixel X included in the input image, and determines that the reference image is correct. Reference information for instructing reference is generated (prediction unit ID for inter-layer prediction). Also in this case, the relative position of the reference pixel in the reference image corresponds to the relative position of the target pixel X in the input image.
Thereby, the encoding program 52 in this modification can generate the code data of the mask layer 710 without referring to the reference image. Note that the image processing apparatus 2 refers to the reference image according to the code data when decoding the code data generated in this way.

  In addition, the image processing apparatus 2 may set a region where reference pixels should be referred to in accordance with a user operation, and generate code data corresponding thereto. For example, the image processing apparatus 2 accepts designation of an area to be a transparent pixel by a pointing device or the like, and code data (reference pixel E) for instructing reference to a reference image (another image) as image data in the designated area. May be replaced by the prediction unit ID corresponding to the number of consecutive predictions).

[Modification 2]
In addition, for a moving image composed of a plurality of frame images, a reference position can be set on another frame. The moving image in this case is composed of, for example, a multi-mask type frame image.
FIG. 10A illustrates a multi-mask type frame image, and FIG. 10B illustrates a reference position with respect to the frame image illustrated in FIG.
FIG. 11 is a diagram illustrating a functional configuration of the third encoding program 54 applied to encoding of moving images. It should be noted that among the components in this figure, the same reference numerals are given to the components that are substantially the same as those shown in FIG.

As illustrated in FIG. 10A, a multi-mask type frame image has a mask layer 710 and an image layer 720, respectively. In this example, a moving object (image element) is assigned to the mask layer 710, and a still object as a background is assigned to the image layer 720. Therefore, the position of the object (image element) is different between the mask layer 710 in the previous frame 700 and the mask layer 710 ′ in the current frame 700 ′.
When encoding such a current frame 700 ′, the image processing apparatus 2 performs predictive encoding by setting reference positions on other layers of the current frame 700 ′ and on the layer of the previous frame 700. . Specifically, when the image processing apparatus 2 encodes the mask layer 710 ′ of the current frame 700 ′, as illustrated in FIG. 10B, the image processing apparatus 2 includes a plurality of reference pixels A˜ on the current frame 700 ′. D is set, a reference pixel E (corresponding to interlayer prediction) is set on the image layer 720 of the current frame 700 ′, and a reference pixel F (corresponding to interframe prediction) is set on the mask layer 710 of the previous frame 700. .
Further, the image processing apparatus 2 moves the position of the reference pixel F according to the moving direction and moving amount of the object assigned to the mask layer 710. As a result, the prediction accuracy based on the reference pixel F can be improved.

As illustrated in FIG. 11, the third encoding program 54 has a configuration in which a plurality of first interlayer prediction units 520 (interlayer first prediction unit 520a and interlayer second prediction unit 520b) are arranged.
In the encoding program 54, the interlayer first prediction unit 520a refers to the pixel value of the reference pixel E illustrated in FIG. 10B, sets the pixel value of the reference image as a predicted value, and uses the predicted value and the target pixel. The comparison result with the pixel value of (pixels included in the target image) is output to the run counter 540.

  The interlayer second prediction unit 520b refers to the pixel value of the reference pixel F illustrated in FIG. 10B, sets the pixel value of the reference image as a predicted value, and uses the predicted value and the target pixel (included in the target image). The comparison result with the pixel value of pixel) is output to the run counter 540. Further, the interlayer second prediction unit 520b moves the position of the reference pixel F according to the moving direction and moving amount of the object.

  Thereby, since the 3rd encoding program 54 encodes the mask layer 710 'using the inter-layer prediction and inter-frame prediction in the same frame, it can anticipate a high compression rate.

[Second Embodiment]
Next, a second embodiment will be described. As a method of managing image data with a layer structure, there is an MRC (Mixed Raster Content) method in addition to the multi-mask method. In the MRC method, an image is composed of two or more image layers to which a multi-valued image is assigned and a selection layer for selecting an image element to be output for each image region from these image layers.

FIG. 12 is a diagram for explaining the layer structure of the MRC method.
The MRC image data is composed of a plurality of image layers to which image elements constituting an image are assigned, and a selection layer for selecting image elements to be output for each image area. As shown in FIG. 12, the image data 800 of this example includes a foreground layer 810 and an image layer 820 as image layers, and further includes a selection layer 830 for selecting an image element to be output from these layers.
The foreground layer 810 of this example is assigned a low gradation image such as a simple CG image or a character image. The foreground layer 810 has a plurality of color information and halftone information included in the CG image or character image.
The image layer 820 of this example is assigned a continuous tone image having a larger number of tones than the foreground layer 810.
The selection layer 830 is composed of binary data indicating which image element of the foreground layer 810 and the image layer 820 is output for each image region (for example, for each pixel), and a pattern image is configured by the binary data. ing. In the drawing, the black portion of the selection layer 830 is a pattern image for selecting an image element of the foreground layer 810, and the white portion is a pattern image for selecting an image element of the image layer 820.
The display image 850 is obtained by displaying or printing an image element selected from the image elements included in the foreground layer 810 and the image layer 820 according to the binary pattern included in the selection layer 830.

  When the image data 800 illustrated in FIG. 12 is encoded, the encoding program 54 illustrated in FIG. 11 can be applied. That is, the interlayer first prediction unit 520a refers to the foreground layer 810 (FIG. 12), and the interlayer second prediction unit 520b refers to the image layer 820. In this case, the encoding program 54 applies the prediction result by the interlayer first prediction unit 520a and the prediction result by the interlayer second prediction unit 520b in preference to the prediction result by the intra-layer prediction unit 510. By encoding the prediction result (prediction unit ID and its continuous number), code data of the selection layer 830 can be generated. In this case as well, the relative position of the reference pixel in the reference image corresponds to the relative position of the target pixel in the target image. For example, when the resolution of the target image matches the resolution of the reference image, the same relative position It becomes. That is, the reference pixel referred to by the interlayer first prediction unit 520a and the reference pixel referred to by the interlayer second prediction unit 520b are the pixel of interest X when the target image and these reference images are superimposed. Overlapping pixels.

As described above, the image processing apparatus 2 according to the present embodiment can generate code data of the selection layer 830 (FIG. 10) by encoding an input image with reference to a plurality of reference images. .
Here, when encoding is performed with reference to a plurality of reference images as in the present embodiment, the generated encoded data of each layer has a tree relationship, and when decoding, foreground layers are used. The decoding order of the code data of 810 and the code data of the image layer 820 is arbitrary. The selection layer 830 holds the image element assigned to the foreground layer 810 and the trimming shape of the image element assigned to the image layer 820. Therefore, the selection layer 810 and the image layer 820 do not need to have a shape, and may be filled around the assigned image element with, for example, a pattern or color that increases the compression rate.

[Modification 1]
Next, a first modification of the second embodiment will be described.
FIG. 13 is a diagram illustrating a functional configuration of the fourth encoding program 56. It should be noted that among the components in this figure, the same reference numerals are given to the components that are substantially the same as those shown in FIG.
As illustrated in FIG. 13, the fourth encoding program 56 causes the first interlayer prediction unit 520 to generate a third interlayer prediction unit that generates prediction data based on the image data of the selected layer 830 (FIG. 12). The configuration replaced with 524 is adopted.
The third interlayer prediction unit 524 in this modification example acquires the image data (bit pattern) of the selected layer 830, and based on the acquired image data, any reference image (the foreground layer 810 or the image layer in FIG. 12). 820) is output to the run counter 540. For example, when the image data 800 illustrated in FIG. 12 is input, the interlayer prediction unit 524 causes the foreground layer 810 in the black region of the selection layer 830 (the region of the binary pattern instructing selection of the foreground layer 810). It is determined that the pixel value of the reference pixel matches the pixel value of the target pixel, and the code generation unit 560 is caused to generate a code instructing reference to the foreground layer 810, and the white region of the selection layer 830 (selection of the image layer 820) In the binary pattern area instructing the image layer 820, it is determined that the pixel value of the reference pixel in the image layer 820 matches the pixel value of the target pixel, and the code generation unit 560 generates a code instructing the reference of the image layer 820. .
Thereby, the encoding program 56 in the present modification can generate code data of the selection layer 830 without referring to the reference image. Note that the image processing apparatus 2 refers to the reference image (the foreground layer 810 and the image layer 820) according to the code data when decoding the code data generated in this way.

[Modification 2]
In addition, as described above, the image data divided for each layer may be subjected to resolution conversion according to the properties of the image elements assigned to each layer. For example, a photographic image or the like has a smaller edge amount than a character image or a CG image, and tends to be inconspicuous as image quality deterioration even if the resolution is lowered. Therefore, for example, the image processing apparatus 2 can reduce the code amount by reducing the resolution of the image data of the image layer 820 in the MRC image data illustrated in FIG.

FIG. 14 is a diagram illustrating a method for decoding code data of layers having different resolutions. FIG. 14A illustrates image data having different resolutions between layers, and FIG. 14B illustrates layers. It is a figure which illustrates the function structure of the decoding program 6 which decodes the code data from which resolution differs between.
As illustrated in FIG. 14A, the image processing apparatus 2 converts image data of an image layer out of MRC image data to a resolution lower than that of other layers (foreground layer and selection layer). Can do. The image layer of this example is converted to 1/2 resolution with respect to other layers. As a result, the image layer image data has half the number of pixels in the main scanning direction and the sub-scanning direction.

As illustrated in FIG. 14B, the decoding program 6 includes a code decoding unit 610, an intra-layer extraction unit 620, an error processing unit 630, a coordinate conversion unit 640, an interpolation processing unit 650, an interlayer extraction unit 660, and a decoded image. A generation unit 670 is included.
In the decoding program 6, the code decoding unit 610 has a table for associating codes and prediction unit IDs (reference positions) with each other, similar to the example illustrated in FIG. 6B, based on the input code data. To specify the reference position. The code decoding unit 610 also decodes the number of consecutive prediction unit IDs and numerical values such as prediction errors based on the input code data.
The reference position, the number of continuations, and the prediction error thus decoded are input to the in-layer extraction unit 620, the error processing unit 630, and the coordinate conversion unit 640.

  The intra-layer extraction unit 620, when the prediction unit ID input from the code decoding unit 610 corresponds to any of intra-layer prediction (that is, corresponding to the reference pixels A to D), the pixel at the corresponding reference position The pixel value of the pixel is output to the decoded image generation unit 670 as decoded data. Further, when the continuous number is input together with the prediction unit ID, the in-layer extraction unit 620 outputs the continuous number to the decoded image generation unit 670 in association with the pixel value corresponding to the prediction unit ID.

  When a prediction error is input from the code decoding unit 610, the error processing unit 630 outputs a pixel value corresponding to the input prediction error to the decoded image generation unit 670 as decoded data. The error processing unit 630 of this example adds the input prediction error and the pixel value of the right-left pixel (position corresponding to the reference pixel A) to obtain decoded data. When image data is encoded by the MRC method, the code of the selection layer 830 is information that always refers to either the foreground layer 810 or the image layer 820. Therefore, the error processing unit 630 generates the decoded data based on the prediction error when the code data of the foreground layer 810 or the image layer 820 is decoded.

When the prediction unit ID input from the code decoding unit 610 corresponds to one of the inter-layer predictions (that is, when the code is an instruction to refer to either the foreground layer 810 or the image layer 820). In addition, coordinate conversion of the reference position is performed according to the difference between the resolution of the layer to be referred to and the resolution of the selection layer 830, and the reference position subjected to the coordinate conversion and the continuous number are output to the interpolation processing unit 650. . The coordinate conversion is performed so that the relative position of the target pixel in the selection layer 830 matches the relative position of the reference pixel in the image layer 820. For example, as illustrated in FIG. 14A, when an image layer 820 having a resolution of ½ is referred to, the coordinate conversion unit 640 halves the coordinate of the pixel of interest X, The reference position in the image layer 820 is used.
When the nearest neighbor method is applied (that is, when the interpolation processing unit 650 is not provided), the coordinate conversion unit 640 images the integer part when the coordinate of the pixel of interest X is halved. The reference position in the layer 820 is used. Further, when the resolution of the layer to be referred to and the resolution of the selection layer 830 are the same, coordinate conversion is not performed.

The interpolation processing unit 650 performs interpolation processing on the image layer 820 based on the reference position where the coordinate conversion is performed by the coordinate conversion unit 640 and the pixel values of neighboring pixels in the vicinity of the reference position. The interpolation process is, for example, a linear interpolation method or a cubic convolution method.
In the case where the interpolation processing unit 650 is not provided, the coordinate conversion unit 640 performs the coordinate conversion, thereby obtaining the same effect as the interpolation by the nearest neighbor method.

  When the prediction unit ID and the continuous number corresponding to the interlayer prediction are input from the code decoding unit 610, the interlayer extraction unit 660 refers to the pixel of the foreground layer 810 or the image layer 820 and extracts the pixel value of the pixel The extracted pixel value and the input continuous number are output to the decoded image generation unit 670. In addition, when the resolution-converted layer (image layer in this example) is referred to, the interlayer extraction unit 660 performs pixel processing on which interpolation processing has been performed according to the reference position converted by the coordinate conversion unit 640. To extract.

  The decoded image generation unit 670 generates a decoded image based on the decoded data input from the intra-layer extraction unit 620, the decoded data input from the error processing unit 630, and the decoded data input from the interlayer extraction unit 660. To do. More specifically, the decoded image generation unit 670 continuously arranges pixels having the input pixel value by the continuous number when the decoded data (pixel value and continuous number) is input from the in-layer extraction unit 620. To do. Further, when the decoded data (the sum of the prediction error and the immediately left pixel value) is input from the error processing unit 630, the decoded image generation unit 670 arranges the pixel having the sum. In addition, when the decoded data (pixel value and continuous number) is input from the interlayer extraction unit 660, the decoded image generation unit 670 continuously arranges pixels having the input pixel value by the continuous number. The pixel group arranged in this way becomes a decoded image.

  As described above, when the resolution of the decoded image and the resolution of the reference image are different, the decoding program 6 of this example generates a decoded image by the coordinate conversion process and the interpolation process, and the resolution of the decoded image and the resolution of the reference image Are substantially the same, the functions of the coordinate conversion unit 640 and the interpolation processing unit 650 are disabled, and a decoded image is generated.

Note that the image processing apparatus 2 may perform flip output, rotation output, or repeated output of image data of a specific layer (image layer 820) by applying the coordinate conversion unit 640.
FIG. 15 is a diagram illustrating a method of repeatedly outputting image data of layer 2, and FIG. 15A illustrates layer images (layer 1 and layer 2) used for repeated output, and FIG. ) Illustrates a decoded image in which the layer 2 illustrated in FIG. 15A is repeatedly output.
As illustrated in FIG. 15A, layer 1 is a foreground layer 810 to which a character image “soap bubble” is assigned, and layer 2 is an image layer 820 to which a CG image is assigned. Layer 2 has a lower resolution than layer 1, the number of pixels in the main scanning direction is W, and the number of pixels in the sub-scanning direction is H.
The coordinate conversion unit 640 illustrated in FIG. 14B uses the remainder obtained by dividing the coordinates (x, y) of the pixel of interest in the selection layer 830 (not shown) by (W, H) as a reference position. As a result, the decoding program 6 (FIG. 14B) repeatedly refers to the image elements assigned to layer 2 (FIG. 15A), and the decoding as illustrated in FIG. 15B is performed. Images can be output.

[Modification 3]
FIG. 16 is a diagram illustrating a functional configuration of the fifth encoding program 58. It should be noted that among the components in this figure, the same reference numerals are given to the components that are substantially the same as those shown in FIG.
As illustrated in FIG. 16, the fifth encoding program 58 has a configuration in which a quantization unit 570 is added before the intra-layer prediction unit 510 and the interlayer prediction unit 520.
The quantization unit 570 in the present modification includes a predicted value providing unit 572 and a pixel value change processing unit 574, generates a quantized image based on the input image data, and generates an intra-layer prediction unit 510 and an interlayer prediction unit 520. And output to the prediction error calculation unit 530.
Specifically, the predicted value providing unit 572 sends the pixel value of the reference pixel (for example, the pixel immediately above or directly above the same image, or the pixel on the reference image) to the pixel value change processing unit 574. And the pixel value change processing unit 574 compares the input pixel value of the reference pixel with the pixel value of the target pixel, and if the difference is within a predetermined range, the pixel value of the target pixel Is replaced with the pixel value of the reference pixel, and when the difference is outside the predetermined range, the pixel value of the target pixel is output as it is. That is, the quantization unit 570 fills the pixel of interest X whose pixel value is approximate with the pixel values of any of the reference pixels A to E.
Thereby, the encoding program 58 in this modification can improve the prediction accuracy by the intra-layer prediction unit 510 or the interlayer prediction unit 520, and can improve the compression rate.

It is a figure explaining the difference with the encoding system accompanying the production | generation of a difference image, and the encoding system in this embodiment, (A) illustrates the difference image of a previous frame and the present frame, (B) Shows an example of a reference position that is referred to when predictive data is generated in the present embodiment. It is a figure which illustrates the image data managed by a layer structure (multi-mask system). (A) is a figure which illustrates the cross-sectional image of a solid shape, (B) is a figure which illustrates the image to which the concealment process was performed by applying this invention. It is a figure which illustrates the hardware constitutions of the image processing apparatus 2 with which the encoding method and decoding method concerning this invention are applied centering on the control apparatus 20. FIG. It is a figure which illustrates the functional structure of the 1st encoding program 5 which is performed by the control apparatus 21 (FIG. 4), and implement | achieves the encoding method concerning this invention. It is a figure explaining the encoding process performed by the encoding program 5, (A) illustrates the position of the pixel referred by the intra-layer prediction part 510 and the interlayer prediction part 520, (B) The code | symbol matched with the reference pixel is illustrated, (C) is a figure which illustrates the code | cord | chord data produced | generated by the code generation part 560. FIG. It is a figure which illustrates the image data of the layer structure encoded by the encoding program 5 (FIG. 5), (A) illustrates the case where the mask layer 710a to which the CG image was allocated is encoded, (B ) Illustrates a case where the mask layer 710b to which the character image is assigned is encoded. It is a figure explaining the process with respect to the boundary area | region of a transparent pixel and a non-transparent pixel. 3 is a diagram illustrating a functional configuration of a second encoding program 52. FIG. (A) illustrates a frame image of a multi-mask method, and (B) illustrates a reference position with respect to the frame image illustrated in (A). It is a figure which illustrates the function structure of the 3rd encoding program 54 applied to encoding of a moving image. It is a figure explaining the layer structure of a MRC system. It is a figure which illustrates the function structure of the 4th encoding program. It is a figure explaining the method to decode the code data of a layer from which resolution differs, (A) illustrates image data from which resolution differs between layers, (B) shows code data from which resolution differs between layers. It is a figure which illustrates the function structure of the decoding program 6 to decode. It is a figure explaining the method to output repeatedly the image data of layer 2, (A) illustrates the layer image (Layer 1 and Layer 2) used for repeated output, (B) illustrates to (A) An example of a decoded image in which the layer 2 is repeatedly output is illustrated. It is a figure which illustrates the function structure of the 5th encoding program.

Explanation of symbols

2 ... Image processing device 5, 52, 54, 56, 58 ... Coding program 510 ... Intra-layer prediction unit (first intra-layer prediction unit to fourth intra-layer prediction unit)
520, 522, 524 ... Interlayer prediction unit 530 ... Prediction error calculation unit 540 ... Run counting unit 550 ... Selection unit 560 ... Code generation unit 6 ... Decoding program 610 ... Code decoding unit 620 ... In-layer extraction unit 630 ... Error processing unit 640 ... Coordinate conversion unit 650 ... Interpolation processing unit 660 ... Interlayer extraction unit

Claims (1)

An encoding device that encodes a target image that is a layer image constituting an input image,
When encoding the pixel value of the target pixel included in one layer image, the first prediction value is calculated based on the pixel value of the pixel included in another layer image corresponding to the target pixel. Prediction means,
Second prediction means for calculating a second predicted value based on pixel values of other pixels included in the same layer image as the target pixel;
Based on the first predicted value and the second predicted value, reference information indicating the position of a pixel included in another layer image or reference information indicating the position of another pixel included in the same layer image is selected. Reference information generating means including:
Code generating means for generating code data of reference information selected by the selecting means as code data of the pixel of interest;
The layer image constituting the input image is an area where the pixel value of the pixel of interest and the pixel value of the pixel included in the other layer image out of the pixels whose positions are indicated by the reference information selected by the selection unit Including a transparent region and a non-transparent region other than the transparent region,
The code generation means, depending on whether the pixel of interest and other pixels included in the same layer image are in the transparent region or the non-transparent region at the boundary between the transparent region and the non-transparent region, An encoding device that uses a pixel value of a target pixel as code data of the target pixel .

Images