JP2005159859A - Image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To propose an image encoding method capable of obtaining a still image with high resolution in an arbitrary motion picture frame, preventing the motion picture from getting deteriorated, and reducing an amount of data to be recorded compared with conventional ones. <P>SOLUTION: A still image with high resolution is recorded for each frame. Hierarchical encoding is applied to the still image with high resolution to divide image data into sub bands. Reversible compression is applied to sub bands constituting the motion picture frame, and irreversible encoding is applied to the remaining sub bands. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image encoding device that inputs image data including a mixture of a moving image and a still image having a higher resolution than the moving image, and compresses the input image within a frame, and an image code for performing discrete wavelet transform on the input image It relates to the conversion method.

  In recent years, there is an increasing demand for shooting both moving images and still images. For example, a video camera that shoots a moving image has appeared that has a still image shooting function such as a digital camera. This video camera cuts out one frame of a moving image and records the cut out still image on a recording medium different from the moving image when a release for still image is pressed during shooting of the moving image. Since such still images have a purpose of printing with a printer, they are usually recorded at a higher resolution than a moving image.

  On the other hand, it is cumbersome to manage the captured moving image data and still image data in separate data files. In other words, it is more convenient to manage moving image data and still image data in one data file. Conventionally, in order to manage moving image data and still image data as one data file, only moving image data is managed and a still image is generated from one frame of the moving image.

  Here, it is desirable that the moving images to be recorded are recorded for each frame in consideration of the subsequent cutout of still images. When compressing and encoding an image to be recorded, if compression processing is performed for each frame, the compressed data can be easily cut out and decoded for each frame.

  As an image encoding method for compressing and encoding one frame of a still image or a moving image, JPEG2000 is standardized by ISO / IEC (see ISO / IEC 154444-1: Non-Patent Document 1). Since JPEG2000 performs hierarchical encoding related to resolution, high-resolution image data and low-resolution image data can be encoded together. Furthermore, either a reversible compression or an irreversible compression can be selected for the image within a common format. FIG. 7 is a block diagram showing a JPEG 2000 encoding process procedure when lossless compression is performed. The JPEG 2000 lossless compression process will be described in detail below.

  JPEG2000 first converts a pixel value included in each component of an input image into a signed integer by performing a process called DC level shift. The shift amount is calculated from the bit accuracy of the pixel value.

  The DC level shifted image data is subjected to component conversion and converted into a luminance component and a color difference component.

  The component-converted image data is subjected to two-dimensional wavelet transform for each component. The JPEG2000 wavelet transform employs a lifting operation. In the lifting operation, filtering and sub-sampling are simultaneously performed by repeating an operation of adding the coefficient obtained by multiplying the sum of the adjacent data in the one-dimensional data to the coefficient at the center. The lifting calculation (forward conversion) of the reversible filter defined in the JPEG2000 standard is expressed by the following recurrence formula.

  Since the recurrence formula shown in Formula 1 is composed only of integer addition / subtraction, shift calculation, and floor function, decimal calculation is not performed, and the output coefficient is an integer. Therefore, the relationship between the value before calculation and the value after calculation has reversibility. As a result of the lifting operation, a low frequency component appears when the pixel coordinates are even, and a high frequency component appears when the pixel coordinates are odd. In the wavelet transform, the lifting operation shown in the asymptotic formula is repeatedly performed in a one-dimensional direction to divide the image data into a low frequency component and a high frequency component. JPEG2000 divides the image data into two-dimensional subbands by sequentially applying this calculation to the pixel lines in the horizontal direction from the left end to the right end, and further sequentially for each pixel line in the vertical direction from the upper end to the lower end. To do. FIG. 8 is a schematic diagram showing a state in which an image of 4 pixels in the vertical direction and 4 pixels in the horizontal direction is divided into subbands by wavelet transform. Hereinafter, the wavelet transform process will be described with reference to FIG.

  In FIG. 8, L and H in the figure represent frequency components included in the coefficients, L represents a low frequency component, and H represents a high frequency component. The numbers shown in parentheses represent the coordinates of the pixels in the original image. The left-hand value is the vertical coordinate from the upper end to the lower end, and the right-hand value is the horizontal coordinate from the left end to the right end. Represents. In the wavelet transform, a one-dimensional lifting operation is first performed from the left end to the right end for each vertical pixel line from the upper end to the lower end. The result of this lifting operation is shown in FIG. After the lifting calculation, rearrangement is performed to combine the calculation results for each of the L and H components. The result of rearranging the coefficients is shown in FIG. Subsequently, a one-dimensional lifting operation is performed from the upper end to the lower end for each horizontal pixel line from the left end to the right end. The result is shown in FIG. Subsequently, as in the calculation for each vertical pixel line, the calculation result is rearranged for each of the L and H components. The result is shown in FIG.

  The subband coefficients generated by the wavelet transform can be quantized by setting a quantization step for each subband. In the case of lossless compression, all subband coefficients are quantized in a quantization step of value 1. At this time, since the subband coefficients that have become integers are quantized in the quantization step of value 1, the quantized coefficients are not different from the coefficients before quantization. Therefore, the reversible transformation quantization process means that the quantization is practically not performed.

  The quantized subband coefficients are further divided into code blocks, and coefficient bit modeling is performed for each code block. First, the code block will be described with reference to FIG. In FIG. 9, code blocks included in four subbands constituting one image are described.

The code block has four subbands shown in FIG. 9 (LL subband, HL subband, LH subband, and HH subband) as the origin, and the subband coefficient is the horizontal width: 2 ^xcb The vertical width is divided into 2 ^ycb blocks. Here, xcb and ycb are integers of 2 to 10, and are arbitrary values satisfying xcb + ycb ≦ 12.

  Next, the coefficient bit modeling process will be described. Each code block including subband coefficients is processed by dividing each coefficient into a code and an absolute value, and the bits included in the absolute value are processed in a predetermined order for each bit plane.

  FIG. 10 shows a state in which a code block of 8 vertical coefficients and 8 horizontal coefficients is divided into positive and negative signs of each coefficient and 8 bit planes from MSB to LSB in the absolute value of each coefficient. In coefficient bit modeling, processing is advanced for each plane for all bit planes from the MSB to the LSB.

  In coefficient bit modeling, each bit included in a bit plane is scanned in a predetermined order using three processing passes.

  All the bits processed in the three processing passes in the coefficient bit modeling are arithmetically encoded. Since all bits are arithmetically encoded, the processing from coefficient bit modeling to arithmetic encoding has no loss of data and maintains reversibility.

  Arithmetic-coded codes are collected for each resolution component by aligning the order of the codes, collected for each small area in the image, or divided into a plurality of layers by dividing the MSB to LSB of the image into multiple layers. To be collected.

  As described above, the lossless compression of JPEG2000 maintains the reversibility of processing by limiting the coefficients of operations used for a series of processing to only integers and further virtually not performing quantization.

After JPEG2000 was standardized, a continuous image encoding method called Motion JPEG2000 (JPEG2000 Part3) was standardized by ISO / IEC (see ISO / IEC 15444-3: Non-Patent Document 2). Motion JPEG2000 is a standard for recording by adding various attribute information necessary for continuous image reproduction to individual image data compressed and encoded by the JPEG2000 Part1 method.
ISO / IEC-15444-1: JPEG2000 standard part-1 ISO / IEC-15444-3: JPEG2000 standard part-3

  Conventionally, when recording both a high-resolution still image and a low-resolution moving image, the still image and the moving image are recorded on different recording media or recording areas. In this case, even if a still image is shot together with a moving image by shooting at one place, it is not recorded as a single file on one recording medium, and is difficult to manage.

  Further, when recording both a still image and a moving image, only the frame at the timing specified at the time of shooting is recorded as the still image to be shot. For this reason, a high-resolution still image in an arbitrary moving image frame cannot be obtained when the captured moving image is played back.

  To obtain a high-resolution still image in any video frame when playing back the captured video, record a high-resolution still image for all frames, and record a high-resolution still image for a low-resolution video frame. A method of extracting from low frequency components in a still image has been taken.

  However, if high-resolution still images are recorded for all frames, the amount of data to be recorded is large. Therefore, conventionally, still images with high resolution components have been compressed and recorded. Here, when a high-resolution component still image is reversibly compressed, the image quality does not deteriorate, but the amount of data to be recorded is still large and impractical. Further, when irreversible compression is performed on a high-resolution still image, not only image quality deterioration occurs in the still image but also image quality deterioration occurs in a moving image frame extracted as a low-frequency component of the still image.

  The present invention has been made in view of the above-described problems. A high-resolution still image can be obtained in an arbitrary moving image frame, the moving image is not deteriorated, and the amount of data to be recorded is smaller than that in the past. An object of the present invention is to provide an image encoding method that can be used.

  In order to solve the above problems, the present invention records a high-resolution still image for each frame, applies hierarchical encoding to the high-resolution still image, divides the image data into subbands, and configures a moving image frame It is characterized in that lossless compression is performed on the subbands to be processed and lossy encoding is performed on the remaining subbands.

  The above configuration is newly organized and shown in the following (1) to (7).

(1) In an image encoding method for inputting image data and compressing and encoding it within a frame,
The input image data is subband divided by reversible calculation,
Of the generated subbands, apply reversible compression to subbands containing low frequency components below a certain level.
An image encoding method, characterized in that lossy compression is performed on the remaining subbands.

  (2) In the above (1), the reversible calculation uses integer multiplication coefficients, shift calculation, and rounding processing for integer data, thereby preventing an error in truncation of significant calculation digits. Method.

  (3) In the above (2), the generated subband is quantized in a quantization step of value 1, and the lossy compression process cuts off certain lower bits of the remaining subband coefficient bits. An image encoding method characterized by the above.

  (4) The image encoding method according to (3), wherein the image encoding method is a JPEG2000 system.

  (5) In the above (2), the lossless compression process quantizes the subband including the low frequency component below the predetermined value in the quantization step of value 1, and the lossy compression process includes the remaining subbands. An image encoding method, wherein a band is quantized with more than one quantization step.

  (6) The image encoding method according to (5), wherein the quantization step is recorded in a file header of a data file output by encoding image data.

  (7) The image encoding method according to (6), wherein the image encoding method is a JPEG2000 system.

  According to the present invention, when recording both a high-resolution still image and a low-resolution moving image, the high-resolution still image is recorded for each frame, and the high-resolution still image is subjected to hierarchical encoding. By dividing the data into subbands, performing lossless compression on the subbands that make up the video frame, and irreversible encoding on the remaining subbands, a high-resolution still image can be obtained in any video frame. In addition, the moving image is not deteriorated and the amount of data to be recorded can be reduced as compared with the case where the entire still image is reversibly compressed. Further, the image data recorded in the embodiment of the present invention can be decoded by all decoders compliant with the JPEG2000 system.

  Hereinafter, embodiments of the present invention will be described in detail.

  The image encoding method according to the first embodiment of the present invention divides into subbands by applying wavelet transform to an input image in accordance with the JPEG2000 system, quantizes each subband, and generates coefficient bits. Modeling is performed, and the lower bits of the subband that are not required for decoding of the moving image frame are discarded during coefficient bit modeling, and the remaining bits are arithmetically encoded.

  FIG. 1 is a block diagram for explaining a processing procedure according to the first embodiment of the present invention. In the present embodiment, description will be made on the assumption that a high-resolution still image having an image size that is four times each of the vertical and horizontal directions with respect to the image size of the moving image frame is input. The processing procedure according to the second embodiment of the present invention when high-resolution still image data is input will be described below.

  The input image data is first subjected to JPEG2000 DC level shift and component conversion.

  The component-converted image data is subjected to two-dimensional wavelet transform using a JPEG2000 5 × 3 filter, and is divided into a plurality of subbands. FIG. 2 is a schematic diagram showing a state in which image data is divided into a plurality of subbands by two-dimensional wavelet transform. As shown in FIG. 2, the input high-resolution still image is divided into a plurality of sub-bands, and the 2LL sub-band necessary for the low-resolution moving image frame and the 1HL, 1LH, 1HH, It is divided into 2HL, 2LH and 2HH subbands.

  All the generated subbands are quantized in a quantization step of value 1 in this embodiment. Note that this processing effectively means that no quantization is performed.

  The quantized subband coefficients are divided into code blocks, and JPEG2000-based coefficient bit modeling is performed on a code block basis.

  Of the coefficient bits that have been coefficient bit-modeled, the lower-order bits of the coefficient bits included in the subbands that are not necessary for the reproduction of the moving image frame are discarded according to the allowable image quality degradation level. By truncating the lower bits included in a specific code block, lossy compression can be performed on a specific subband, and the amount of data can be reduced.

  FIG. 3 is a schematic diagram illustrating a subband including a coefficient bit for performing lower bit truncation and a subband including a coefficient bit for not performing lower bit truncation among coefficient bits of the entire image. In FIG. 3, the coefficient bits included in the 2LL subbands necessary for the reproduction of the moving image frame are not truncated, and the other subbands (1HL, 1LH, 1HH, 2HL, 2LH, 2HH) This indicates that the lower order bits are cut off for the coefficient bits included in the (subband).

  The remaining bits are subjected to JPEG2000 arithmetic coding and output.

  By inputting the number of high-resolution still images for the number of frames of the moving image and performing the above-described image processing on each image, the low-resolution moving image frames that are losslessly compressed are recorded and the high-resolution still images that are irreversibly compressed are recorded. The image can be recorded, and the entire high-resolution still image can be recorded with a smaller amount of data compared to reversible compression.

  The image encoding method according to the second embodiment of the present invention performs a wavelet transform on an input image to divide it into subbands, and a quantization step with a value of 1 in the subbands necessary for decoding a moving image frame Is quantized by a quantization step of a value other than 1 in other subbands, and arithmetically encoded. FIG. 4 is a block diagram for explaining a processing procedure according to the second embodiment of the present invention. In the present embodiment, description will be made on the assumption that a high-resolution still image having an image size that is four times each of the vertical and horizontal directions with respect to the image size of the moving image frame is input. The processing procedure according to the second embodiment of the present invention when a high-resolution still image is input will be described below.

  The input image data is first subjected to JPEG2000 DC level shift and component conversion.

  The component-converted image data is subjected to two-dimensional wavelet transform using a JPEG2000 5 × 3 filter, and is divided into a plurality of subbands. The divided subbands are the same as the subband modes illustrated in FIG. 2 described in the first embodiment.

  In the present embodiment, of the generated subbands, the subbands necessary for playback of the moving image frame are quantized by a quantization step having a value of 1, and the other subbands are quantized by a quantization step having a value of 1 or more. To do. An example of the quantization step set for each subband is shown in FIG. Further, this quantization mode is shown in FIG. In FIG. 6, the coefficients included in the 2LL subbands necessary for the reproduction of the moving image frame are quantized in the quantization step of value 1, and the other subbands (1HL, 1LH, 1HH, 2HL, 2LH, 2HH) are quantized. The coefficient included in each subband of FIG. 4 indicates that quantization is performed in a quantization step having a value exceeding 1. The quantization step used for quantization is recorded in the file header.

  The quantized subband coefficients are divided into code blocks, subjected to JPEG2000 coefficient bit modeling and arithmetic coding for each code block, and output. In the present embodiment, it is arbitrary whether or not to perform lower bit truncation for subbands other than 2LL used for reproduction of moving image frames.

  By inputting the number of high-resolution still images for the number of frames of the moving image and performing the above-described image processing on each image, the low-resolution moving image frames that are losslessly compressed are recorded and the high-resolution still images that are irreversibly compressed are recorded. The image can be recorded, and the entire high-resolution still image can be recorded with a smaller amount of data compared to reversible compression.

The block diagram which shows the process sequence of the image coding method which concerns on the 1st Embodiment of this invention. The schematic diagram which shows the aspect divided | segmented into the several subband by the two-dimensional wavelet transformation which concerns on the 1st Embodiment of this invention Schematic diagram according to the first embodiment of the present invention, illustrating a subband including coefficient bits that perform truncation of lower bits and a subband that includes coefficient bits that do not perform truncation of lower bits. The block diagram which shows the process sequence of the image coding method which concerns on the 2nd Embodiment of this invention. The figure which shows an example of the quantization step set for every subband which concerns on the 2nd Embodiment of this invention with a table | surface Schematic diagram for explaining quantization of each subband according to the second embodiment of the present invention. The block diagram which showed the processing procedure of the encoding of JPEG2000 in the case of performing lossless compression Schematic diagram showing how an image of 4 pixels vertically and 4 pixels horizontally is divided into subbands by wavelet transform Schematic showing how subband coefficients are divided into code blocks Schematic diagram for explaining how the bits of the coefficients included in the code block are processed

Claims (7)

In an image encoding method for inputting image data and compressing and encoding it in a frame,
The input image data is subband divided by reversible calculation,
Of the generated subbands, apply reversible compression to subbands containing low frequency components below a certain level.
An image encoding method, characterized in that lossy compression is performed on the remaining subbands.   2. The image encoding method according to claim 1, wherein the lossless calculation uses an integer multiplication coefficient, a shift calculation, and a rounding process for integer data, thereby preventing a truncation error of a calculation effective digit.   3. The generated subband is quantized by a quantization step of value 1, and the lossy compression process is characterized by truncating certain lower bits of the remaining subband coefficient bits. An image encoding method to be performed.   4. The image encoding method according to claim 3, wherein the image encoding method is a JPEG2000 system.   3. The lossless compression process according to claim 2, wherein the lossless compression process is performed by quantizing a subband including a low frequency component equal to or less than a certain value in a quantization step of 1, and the lossy compression process is performed on the remaining subband by 1 An image encoding method characterized by performing quantization with a quantization step exceeding 1.   6. The image encoding method according to claim 5, wherein the quantization step is recorded in a file header of a data file output by encoding image data.   7. The image encoding method according to claim 6, wherein the image encoding method is a JPEG2000 system.