JP4040404B2 - Code string conversion apparatus and method, image processing apparatus, and image recording apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a code string converter,The present invention relates to an image processing apparatus, an image recording apparatus, and a code string conversion method including the same.
[0002]
[Prior art]
  Conventionally, MPEG1 / MPEG2 / MPEG4 dedicated to moving images and Motion JPEG that handles still images as continuous frames are used as image compression / decompression algorithms. Recently, a new method called Motion JPEG2000 is being standardized as an international standard for encoding the latter motion still image.
[0003]
  The difference between the MPEG system and the Motion still image system is that the latter performs only intra-frame coding, whereas the former performs correlation not only in images within the same frame but also between different frames, thereby increasing the compression rate. There is in being able to. On the other hand, in the latter method in which each frame is handled independently, editing for each frame is possible as compared with the former, and an error during communication does not reach other frames. As described above, the MPEG system and the Motion still image system have their respective features. The method is properly used for each application.
[0004]
  In the Motion JPEG2000 system, discrete wavelet transform is used as a conversion system, but there is one in which image data is compressed and encoded using this discrete wavelet transform (see, for example, Patent Document 1).
[0005]
[Patent Document 1]
JP 2001-309281 A
[0006]
[Problems to be solved by the invention]
  In the technique disclosed in Patent Document 1, not only the pixel values are subjected to discrete wavelet transform and compression coding, but also the correlation between the images between different frames, and the moving image data when there is no image movement between the frames. Therefore, the data compression rate can be further improved.
[0007]
  By the way, in such a conventional technique, there is a process for performing “quantization” optimized for still images in a frame and a process for performing “code amount reduction” based on a comparison result of wavelet coefficient values between frames. Each is performed independently, and the still image in the frame is quantized based on a unique “quantization table”.
[0008]
  However, such conventional techniques have the following problems.
[0009]
[Problem 1]
  Conventionally, the compression of Motion still images has been focused only on “quantization” optimized for still images in a frame, and “quantization” has not been performed in consideration of the visual characteristics unique to moving images. It was. Originally, although the amount of motion between frames differs for each image region, conventionally, it has been assumed that the amount of motion is uniform over the entire image region. As a result, completely new image quality degradation that could not be expected when displaying a still image often appeared when displaying a moving image.
[0010]
  FIG. 25 is an explanatory diagram showing a conventional “quantization” method. FIG. 25 shows a quantization method in which the amount of motion is ignored and the subband codes are uniformly reduced. Here, the vertical axis represents the amount of motion, and the correlation coefficient value obtained from the difference between the wavelet coefficient values for each tile and each subband between the current frame and the previous frame is used. The horizontal axis represents the unit of tile. A tile is a rectangular area created by dividing an original still image in a frame.
[0011]
  Looking at the amount of motion, for example, the tile A is large and the tile C is small. However, in the conventional method, as shown in FIG. 25, the entire image is replaced with a uniform amount of motion. In this example, the movement amount of all tiles is handled as “none”. Therefore, the only “quantization table” is referenced regardless of the amount of motion.
[0012]
[Problem 2]
  In addition, the quantization of still images within a frame uses only one “quantization table” that is uniquely defined by the compression rate or bit rate, but the code amount distribution of subbands is originally different for each image area. Although it should have been, it was not considered at all in the past. As a result, the image quality of the quantized image varies greatly depending on the properties of the original image, such as an image with many high-frequency components, an image with many low-frequency components, and an image widely distributed from high to low frequencies.
[0013]
  FIG. 26 is an explanatory diagram showing a conventional “quantization” method. FIG. 26 shows a quantization method in which the distribution of the code amount is ignored and the subband codes are uniformly reduced. Here, the vertical axis represents the relative code amount when the low-frequency subband code amount in the highest layer (maximum number of decomposition levels) is normalized to “1”. The horizontal axis represents the subband frequency band. The thin line portion of the curve indicating the code amount distribution for each tile represents the subband from which the code amount is to be deleted, while the thick line portion represents the subband in which the code amount is stored as it is. . FIG. 26A shows a case where the number of subbands targeted for code amount reduction is large, and FIG. 26B shows a case where the number of subbands targeted for code amount reduction is small and the compression rate is low. It corresponds to the case of high.
[0014]
  As shown in FIG. 26, the tile A and the tile D hold code data up to a high frequency range. In addition, the tile B has a relatively large amount of information in the middle range, and the tile C has most of the information in the low range. As described above, even when the subband code amount is different for each tile, the conventional method uniformly reduces the subband code data regardless of the code amount distribution. As a result, when the compression rate is particularly high, the image quality of tiles A, B, and D, which have a relatively large amount of data in the middle to high range, is compared with the image quality of tile C in which data is concentrated in the low range. As a result, the deterioration was remarkable.
[0015]
  Here, in order to specifically explain the above-described problem 1 and problem 2, the observation results will be described in detail below regarding the phenomenon that occurs when Motion JPEG2000 is used as the compression / decompression algorithm. FIG. 27 shows three consecutive frames extracted from the moving image content, and FIG. 28 is a table showing the motion amount and pixel frequency of a predetermined tile in each frame shown in FIG. Here, these three frames will be described as an example. The example shown in FIG. 27 represents a bicycle that runs against a landscape with a small amount of movement so that a tile with a large amount of movement and a tile with a small amount of movement can be easily distinguished. B, C, and D correspond to the tiles A, B, C, and D shown in FIGS.
[0016]
  As described above, when a motion still image encoded by conventional quantization is expanded and displayed without considering the motion amount between frames and the code amount distribution of subbands, the following phenomenon occurs. appear.
[0017]
[Tile A (bicycle spokes)]
  The bicycle is closest to the shooting point, and the code amount distribution of the high-frequency subband is large due to structural features (the number of edges increases). However, according to the conventional “quantization” method, the code amount of the high frequency sub-band is reduced as the compression rate increases, so that the spoke part of the wheel is discriminated one by one in the still image in the frame. It becomes difficult to do. However, since the determination of spokes on a moving bicycle is beyond the limit of human visual acuity, the effect on moving images is not so great.
[0018]
[Tile B (leaf)]
  There is almost no movement of the tree located in the middle of the foreground and foreground bicycles. The leaf portion has a large code amount distribution in the middle band. However, according to the conventional “quantization” method, as the compression rate increases, it becomes difficult to distinguish the leaves one by one from the still image in the frame. However, the tendency is not as strong as the bicycle spokes in tile A.
[0019]
[Tile C (background)]
  In the distant view, a skyline having a large code amount distribution in the low frequency subband is arranged. Even if the compression rate increases, it is relatively difficult to be affected by the code amount reduction. However, since there is almost no amount of motion, if even a small amount of low-frequency sub-band data is deleted, the blur will spread over the entire still image in the frame. When viewed in a moving image, “smooth distortion” occurs in the background, which is very disturbing.
[0020]
[Tile D (Tree)]
  In the image area, there are many contour portions of the tree. As a result, the code amount distribution is different from the tile C in that the data amount on the high frequency side is slightly larger. When the tile C is compared with the image, there is not much difference between the still images in the frame. However, when a moving image is displayed, “smooth distortion” appears more prominently. This seems to be because the amount of the high-frequency component that has been shaved is large.
[0021]
  An object of the present invention is to provide a code string conversion device, an image processing device, an image recording device, a program, a storage medium, and a code amount reduction capable of performing quantization while suppressing deterioration in image quality due to the amount of motion and the level of image frequency. Is to provide a method.
[0022]
  An object of the present invention is to provide a code string conversion device, an image processing device, an image recording device, a program, a storage medium, and a code amount reduction method capable of performing quantization while suppressing deterioration in image quality due to the amount of motion. It is.
[0023]
  An object of the present invention is to provide a code string conversion device capable of performing quantization while suppressing image quality deterioration associated with high and low image frequencies.Image processing apparatus, image recording apparatus, and code string conversion method including the sameIs to provide.
[0024]
[Means for Solving the Problems]
  The code string conversion device of the invention according to claim 1 is:Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. And a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame based on the analysis result of the syntax analysis means, and a correlation between the current frame and the previous frame A motion amount detecting means for detecting a motion amount and setting the motion amount to one of a plurality of predetermined motion amount index values; and a subband code in the current frame based on an analysis result of the syntax analyzing means Subband code quantity distribution detecting means for detecting a quantity distribution and setting the subband code quantity distribution to one of a plurality of predetermined subband code quantity distribution index values; A compression rate specifying means for receiving a compression rate specification and setting the compression rate to one of a plurality of predetermined compression rate index values; a plurality of motion amount index values; and a plurality of subband code amount distribution index values; A quantization table group having a plurality of quantization tables related to combinations of values with a plurality of compression rate index values in a three-dimensional matrix, the motion amount index value set by the motion amount detection means, and the subband code A predetermined quantization table is selected from the quantization table group based on the subband code amount distribution index value set by the amount distribution detection unit and the compression rate index value set by the compression rate specifying unit, and the selection is made The code of the code string data is quantized by the quantized table, and the code amount of the code string data is reduced for each rectangular area. And the issue amount reduction meansIs provided.
[0025]
  Therefore, motion amount and subband code amount distribution detected from code sequence data created by orthogonally transforming pixel values of moving image data for each rectangular area and hierarchically compressing and encoding, and compression specified by the user According to the rate, the code amount of the code string data is reduced for each rectangular area. As a result, quantization is performed in consideration of not only the compression rate but also the amount of motion and the subband code amount distribution, so that quantization is performed while suppressing deterioration in image quality due to the amount of motion and the level of image frequency. It becomes possible.By detecting the amount of motion based on the correlation coefficient value of the data amount of the packet length between the current frame and the previous frame, the amount of motion can be obtained at high speed and accurately.
[0026]
  The code string conversion apparatus according to the second aspect of the present invention is obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A syntax analysis means for analyzing the data described in the header portion of the code string data, and a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame based on the analysis result of the syntax analysis means. The motion amount detecting means for detecting the motion amount between the current frame and the previous frame and setting the motion amount to one of a plurality of predetermined motion amount index values, and specifying the compression rate A compression rate specifying means for accepting and setting the compression rate to one of a plurality of predetermined compression rate index values; and each of a plurality of motion amount index values and a plurality of compression rate index values Motion amount index set in the quantization table group having a plurality of quantization tables in a two-dimensional matrix according to the combination of the motion amount detecting means A predetermined quantization table is selected from the quantization table group based on the value and the compression ratio index value set by the compression ratio specifying means, and the code of the code string data is quantized by the selected quantization table Code amount reducing means for reducing the code amount of the code string data for each rectangular area;Is provided.
[0027]
  Therefore, according to the amount of motion detected from code sequence data created by orthogonally transforming the pixel values of moving image data for each rectangular area and hierarchically compressing and encoding, and the compression rate specified by the user, The code amount of data is reduced for each rectangular area. As a result, not only the compression rate but also the amount of motion is taken into account for quantization, so that it is possible to perform quantization while suppressing image quality deterioration associated with the amount of motion.
[0028]
  According to a third aspect of the present invention, there is provided a code string conversion device obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. Syntactic analysis means for analyzing the data described in the header portion of the code string data, and a subband code amount distribution in the current frame is detected based on the analysis result of the syntax analysis means, and the subband code amount Subband code amount distribution detecting means for setting the distribution to one of a plurality of predetermined subband code amount distribution index values, and receiving a designation of compression rate, and a plurality of predetermined compression rate indexes A plurality of compression ratio designating means set to one of the values, a plurality of subband code amount distribution index values, and a plurality of compression ratio index values, A quantization table group having a child table in a two-dimensional matrix, a subband code amount distribution index value set by the subband code amount distribution detecting unit, and a compression rate index value set by the compression rate specifying unit; Based on the above, a predetermined quantization table is selected from the quantization table group, the code of the code string data is quantized by the selected quantization table, and the code amount of the code string data is reduced for each rectangular area Code amount reduction means andIs provided.
[0029]
  Therefore, according to the subband code amount distribution detected from the code string data created by orthogonally transforming the moving image data for each rectangular area and hierarchically compressing and encoding, and the compression rate specified by the user The code amount of the code string data is reduced for each rectangular area. As a result, quantization is performed in consideration of not only the compression rate but also the subband code amount distribution. Therefore, it is possible to perform quantization while suppressing deterioration in image quality due to high and low image frequencies.
[0030]
  According to a fourth aspect of the present invention, there is provided a code string conversion device that divides still image data into a plurality of rectangular areas, orthogonally transforms pixel values for each rectangular area, and hierarchically compresses and encodes code string data obtained. A syntactic analysis unit for analyzing data described in the header portion; and a subband code amount distribution is detected based on an analysis result of the syntactic analysis unit, and the subband code amount distribution is determined by a plurality of predetermined subbands. Subband code amount distribution detecting means to be set to one of the band code amount distribution index values, and a compression rate to accept the designation of the compression rate and set the compression rate to one of a plurality of predetermined compression rate index values A two-dimensional matrix of a plurality of quantization tables related to a combination of each of a specifying means and a plurality of subband code amount distribution index values and a plurality of compression ratio index values The quantization table group, a subband code amount distribution index value set by the subband code amount distribution detecting means, and a compression rate index value set by the compression rate specifying means. A code amount reducing unit that selects a predetermined quantization table from the group, quantizes the code of the code string data by the selected quantization table, and reduces the code amount of the code string data for each rectangular area;Is provided.
[0031]
  Therefore, according to the subband code amount distribution detected from code string data created by orthogonally transforming still image data for each rectangular area and hierarchically compressing and encoding, and the compression rate specified by the user The code amount of the code string data is reduced for each rectangular area. As a result, quantization is performed in consideration of not only the compression rate but also the subband code amount distribution. Therefore, it is possible to perform quantization while suppressing deterioration in image quality due to high and low image frequencies.
[0032]
  The invention according to claim 5 is the invention according to claims 1 to 4.In any one ofIn the described code string conversion apparatus, the quantization table specifies a subband in which the code amount is reduced and a subband in which the code amount is stored. Therefore, it is possible to reliably reduce the code amount of the code string data.
[0033]
  Invention of Claim 6 is described in Claim 1 or 2Set by the motion amount detection meansAmount of movementChange index valueMeans. The invention according to claim 7 is the code string conversion device according to any one of claims 1, 3, and 4, wherein the subband code amount distribution index value set by the subband code amount distribution detecting means is set. A means for changing is provided.Therefore, it is possible to encode the original image while maintaining high image quality.
[0034]
  Invention of Claim 8 is described in any one of Claim 1 thru | or 7.The code string conversion apparatus includes a code string creating unit that arranges the code string data, the code amount of which is reduced for each rectangular area by the code amount reducing unit, into code string data in a standard format. Therefore, the newly generated code string data can be expanded by a general-purpose decoder.
[0035]
  Invention of Claim 9 is described in any one of Claim 1 thru | or 8.In the code string conversion apparatus,The processing of each means isIt is executed for each component. Therefore, by detecting the “motion amount” and “subband code amount distribution” only for the Y (luminance) component of the YCbCr component and the G (green) component of the RGB component, encoding is performed at high speed and with low power consumption. Control becomes possible.
[0036]
  Invention of Claim 10 is described in any one of Claim 1 thru | or 9.In the code string conversion apparatus,SaidThe rectangular area is a tile. Therefore, discrete wavelet transform can be used as orthogonal transform.
[0037]
  Invention of Claim 11 is described in any one of Claim 1 thru | or 9.In the code string conversion apparatus,SaidThe rectangular area is a precinct. Therefore, even when tile division is not performed (all image areas = tiles), it is possible to perform quantization that suppresses image quality degradation, in the same manner as in units of tiles. Further, the detection unit can be an image area smaller than the tile.
[0038]
  Invention of Claim 12 is described in any one of Claim 1 thru | or 9.In the code string conversion apparatus,SaidA rectangular area is a code block. Therefore, even when tile division is not performed (all image areas = tiles), it is possible to perform quantization that suppresses image quality degradation, in the same manner as in units of tiles. Further, the detection unit can be an image area smaller than the tile.
[0039]
  Claim 13The image processing apparatus according to the invention described above is an image compression apparatus that divides moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforms pixel values for each rectangular area, and hierarchically compresses and encodes the pixel values.Claims 1, 2, 3A code string conversion device.
[0040]
  Claim 145. The image processing apparatus according to claim 4, wherein the image processing apparatus divides still image data into one or a plurality of rectangular areas, orthogonally transforms pixel values for each rectangular area, and hierarchically compresses and encodes the pixel values. A code string conversion device.
[0041]
  Claim 15The image recording apparatus according to the present invention includes an image input device that captures moving image data and processes the captured moving image data.Claim 13And the image processing apparatus described.
[0042]
  Claim 16The image recording apparatus according to the present invention includes an image input device that captures still image data, and processes the captured still image data.Claim 14And the image processing apparatus described.
[0043]
  According to a seventeenth aspect of the present invention, there is provided a code string conversion method obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A syntax analysis step for analyzing data described in the header portion of the code string data, and a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame based on the analysis result of the syntax analysis step. A motion amount detecting step of detecting a motion amount between the current frame and the previous frame and setting the motion amount to one of a plurality of predetermined motion amount index values; and Based on the analysis result, a subband code amount distribution in the current frame is detected, and the subband code amount distribution is one of a plurality of predetermined subband code amount distribution index values. Subband code amount distribution detection step to be set, compression rate designation step of accepting designation of compression rate and setting the compression rate to one of a plurality of predetermined compression rate index values, and a plurality of motion amount index values The motion amount detecting step using a quantization table group having a plurality of quantization tables related to a combination of values of a plurality of subband code amount distribution index values and a plurality of compression rate index values in a three-dimensional matrix The quantization table based on the motion amount index value set in step 1, the subband code amount distribution index value set in the subband code amount distribution detection step, and the compression rate index value set in the compression rate specifying means. Selecting a predetermined quantization table from the group, quantizing the code of the code string data by the selected quantization table, Code amount reduction process to reduce the amount of code of the serial code string data for each rectangular areaincluding.
[0044]
  A code string conversion method according to an eighteenth aspect of the present invention is obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A syntax analysis step for analyzing data described in the header portion of the code string data, and a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame based on the analysis result of the syntax analysis step. A motion amount detecting step for detecting a motion amount between the current frame and the previous frame, and setting the motion amount to one of a plurality of predetermined motion amount index values, and specifying a compression rate. Accepting and setting the compression rate to one of a plurality of predetermined compression rate index values, a plurality of motion amount index values, and a plurality of compression rate index values Using a quantization table group having a plurality of quantization tables related to a combination of values in a two-dimensional matrix, the motion amount index value set in the motion amount detection step and the compression rate set in the compression rate designation step Based on the index value, a predetermined quantization table is selected from the quantization table group, the code of the code string data is quantized by the selected quantization table, and the code amount of the code string data is set for each rectangular area. Code amount reduction process to reduceincluding.
[0045]
  According to a nineteenth aspect of the present invention, there is provided a code string conversion method obtained by dividing moving image data into one or a plurality of rectangular regions for each frame, orthogonally transforming pixel values for each rectangular region, and hierarchically compressing and encoding. A syntactic analysis step for analyzing data described in the header portion of the code string data, and a subband code amount distribution in the current frame based on an analysis result of the syntax analysis step, and the subband code amount A subband code amount distribution detecting step for setting the distribution to one of a plurality of predetermined subband code amount distribution index values, and receiving a designation of a compression rate, and a plurality of predetermined compression rate indexes for the compression rate A plurality of compression ratio designating steps set to one of the values, and a plurality of combinations of values of a plurality of subband code amount distribution index values and a plurality of compression ratio index values Quantization table group having a quantization table in a two-dimensional matrixUsingA predetermined quantization table from the quantization table group based on the subband code amount distribution index value set in the subband code amount distribution detection step and the compression rate index value set in the compression rate designation step. And a code amount reduction step of quantizing the code of the code string data by the selected quantization table and reducing the code amount of the code string data for each rectangular area.
[0046]
  According to a 20th aspect of the present invention, there is provided a code string conversion method that divides still image data into a plurality of rectangular areas, orthogonally transforms pixel values for each rectangular area, and hierarchically compresses and encodes code string data obtained. A syntax analysis step for analyzing data described in the header portion, and a subband code amount distribution is detected based on the analysis result of the syntax analysis step, and the subband code amount distribution is determined by a plurality of predetermined subbands. Sub-band code amount distribution detection step to be set to one of the band code amount distribution index values, and a compression rate to accept the designation of the compression rate and set the compression rate to one of a plurality of predetermined compression rate index values A two-dimensional matrix of a plurality of quantization tables relating to a combination of each value of a designation step and a plurality of subband code amount distribution index values and a plurality of compression ratio index values Quantization table group having a focalUsingA predetermined quantization table from the quantization table group based on the subband code amount distribution index value set in the subband code amount distribution detection step and the compression rate index value set in the compression rate designation step. And a code amount reduction step of quantizing the code of the code string data by the selected quantization table and reducing the code amount of the code string data for each rectangular area.
[0047]
  According to the code string conversion method of the inventions of the seventeenth to twentieth aspects, the same operation as the first to fourth aspects is achieved.
[0048]
DETAILED DESCRIPTION OF THE INVENTION
  A first embodiment of the present invention will be described with reference to FIGS.
[0049]
  First, an outline of the “hierarchical encoding algorithm” and the “JPEG2000 algorithm” which are the premise of the present invention will be described.
[0050]
  FIG. 1 is a functional block diagram of a system that realizes a hierarchical encoding algorithm that is the basis of the JPEG2000 system. This system includes color space transform / inverse transform unit 101, two-dimensional wavelet transform / inverse transform unit 102, quantization / inverse quantization unit 103, entropy encoding / decoding unit 104, and tag processing unit 105. It is configured.
[0051]
  One of the biggest differences between this system and the conventional JPEG algorithm is the conversion method. While JPEG uses discrete cosine transform (DCT), this hierarchical encoding algorithm uses discrete wavelet transform (DWT: Discrete Wavelet Transform) in the two-dimensional wavelet transform / inverse transform unit. ing. DWT has the advantage that the image quality in the high compression region is better than DCT, and this is one of the major reasons why DWT is adopted in JPEG2000, which is a successor algorithm of JPEG.
[0052]
  Another major difference is that in this hierarchical encoding algorithm, a functional block of the tag processing unit 105 is added in order to perform code formation at the final stage of the system. The tag processing unit 105 generates compressed data as code string data during an image compression operation, and interprets code string data necessary for decompression during the decompression operation. With the code string data, JPEG2000 can realize various convenient functions. For example, the compression / decompression operation of a still image can be freely stopped at an arbitrary hierarchy (decomposition level) corresponding to octave division in block-based DWT (see FIG. 3 described later).
[0053]
  In many cases, color space conversion / inverse conversion 101 is connected to the input / output portion of the original image. For example, the RGB color system composed of R (red) / G (green) / B (blue) components of the primary color system and the Y (yellow) / M (magenta) / C (cyan) components of the complementary color system This corresponds to the part that performs conversion or reverse conversion from the YMC color system consisting of the above to the YUV or YCbCr color system.
[0054]
  Next, the JPEG2000 algorithm will be described.
[0055]
  As shown in FIG. 2, in a color image, each component 111 (RGB primary color system here) of an original image is generally divided by a rectangular area. This divided rectangular area is generally called a block or a tile. In JPEG2000, it is generally called a tile. Therefore, such a divided rectangular area is hereinafter referred to as a tile. (In the example of FIG. 2, each component 111 is divided into a total of 16 rectangular tiles 112, 4 × 4 in length and breadth). When such individual tiles 112 (R00, R01,..., R15 / G00, G01,..., G15 / B00, B01,..., B15 in the example of FIG. 2) execute the image data compression / decompression process. Is the basic unit. Therefore, the compression / decompression operation of the image data is performed independently for each component and for each tile 112.
[0056]
  At the time of encoding image data, the data of each tile 112 of each component 111 is input to the color space conversion / inverse conversion unit 101 in FIG. A dimensional wavelet transform (forward transform) is applied to divide the space into frequency bands.
[0057]
  FIG. 3 shows subbands at each decomposition level when the number of decomposition levels is three. In other words, the tile original image (0LL) (decomposition level 0) obtained by tile division of the original image is subjected to two-dimensional wavelet transform, and the subbands (1LL, 1HL, 1LH shown in the decomposition level 1) , 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transformation to separate the subbands (2LL, 2HL, 2LH, 2HH) indicated by the decomposition level 2. Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate subbands (3LL, 3HL, 3LH, 3HH) shown in the decomposition level 3. In FIG. 3, the subbands to be encoded at each decomposition level are indicated by shading. For example, when the number of decomposition levels is 3, subbands (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) indicated by shading are to be encoded, and the 3LL subband is encoded. It is not converted.
[0058]
  Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits by the quantization / inverse quantization unit 103 shown in FIG.
[0059]
  The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each subband. This was introduced to use memory efficiently in implementation. As shown in FIG. 4, one precinct consists of three rectangular regions that are spatially coincident. Further, each precinct is divided into non-overlapping rectangular “code blocks”. This is the basic unit for entropy coding.
[0060]
  The coefficient values after the wavelet transform can be quantized and encoded as they are, but in JPEG2000, in order to increase the encoding efficiency, the coefficient values are decomposed into “bit plane” units, and each pixel or code block is divided. Ranking can be performed on “bitplanes”.
[0061]
  Here, FIG. 5 is an explanatory diagram showing an example of a procedure for ranking the bit planes. As shown in FIG. 5, this example is a case where the original image (32 × 32 pixels) is divided into four 16 × 16 pixel tiles, and the size of the precinct and code block at the composition level 1 is Each is 8 × 8 pixels and 4 × 4 pixels. The numbers of the precinct and the code block are assigned in raster order. In this example, the number of assigns is assigned from numbers 0 to 3, and the code block is assigned from numbers 0 to 3. A mirroring method is used for pixel expansion outside the tile boundary, wavelet transform is performed with a reversible (5, 3) filter, and a wavelet coefficient value of decomposition level 1 is obtained.
[0062]
  An explanatory diagram showing an example of the concept of a typical “layer” configuration for tile 0 / precinct 3 / code block 3 is also shown in FIG. The converted code block is divided into subbands (1LL, 1HL, 1LH, 1HH), and wavelet coefficient values are assigned to the subbands.
[0063]
  The layer structure is easy to understand when the wavelet coefficient values are viewed from the horizontal direction (bit plane direction). One layer is composed of an arbitrary number of bit planes. In this example, layers 0, 1, 2, and 3 are made up of bit planes of 1, 3, 1, and 3, respectively. A layer including a bit plane closer to the LSB (Least Significant Bit) is first subjected to quantization, and conversely, a layer close to the MSB (Most Significant Bit: most significant bit) is quantized to the end. It will remain without being. A method of discarding from a layer close to the LSB is called truncation, and the quantization rate can be finely controlled.
[0064]
  The entropy encoding / decoding unit 104 illustrated in FIG. 1 performs encoding on the tile 112 of each component 111 by probability estimation from the context and the target bit. In this way, encoding processing is performed in units of tiles 112 for all components 111 of the original image. Finally, the tag processing unit 105 performs a process of combining all the encoded data from the entropy encoding / decoding unit 104 into one code string data and adding a tag thereto.
[0065]
  FIG. 6 shows a schematic configuration for one frame of the code string data. Tag information called a header (a main header (tile header) that is tile header position information) is provided at the head of the code string data and the head of the code data (bit stream) of each tile. Appended, followed by the encoded data for each tile. Note that the main header (Main header) describes coding parameters and quantization parameters. A tag (end of codestream) is placed again at the end of the code string data.
[0066]
  On the other hand, when the encoded data is decoded, the image data is generated from the code string data of each tile 112 of each component 111, contrary to the case of encoding the image data. In this case, the tag processing unit 105 interprets tag information added to the code string data input from the outside, decomposes the code string data into code string data of each tile 112 of each component 111, and Decoding processing (decompression processing) is performed for each code string data of each tile 112. At this time, the position of the bit to be decoded is determined in the order based on the tag information in the code string data, and the quantization / inverse quantization unit 103 determines the peripheral bits (that have already been decoded) of the target bit position. Context is generated from the sequence of The entropy encoding / decoding unit 104 performs decoding by probability estimation from the context and code string data, generates a target bit, and writes it in the position of the target bit. Since the data decoded in this way is spatially divided for each frequency band, the two-dimensional wavelet transform / inverse transform unit 102 performs two-dimensional wavelet inverse transform on each of the components of the image data. The tile is restored. The restored data is converted to original color system image data by the color space conversion / inverse conversion unit 101.
[0067]
  The above is an outline of the “JPEG2000 algorithm”. A “motion JPEG2000 algorithm” is an extension of a still image, that is, a method for a single frame to a plurality of frames. That is, “Motion JPEG2000”, as shown in FIG. 7, is a moving image by continuously displaying a JPEG2000 image of one frame at a predetermined frame rate (the number of frames reproduced per unit time). .
[0068]
  Hereinafter, a first embodiment of the present invention will be described. Here, an example relating to a moving image compression / decompression technique typified by Motion JPEG 2000 will be described, but it goes without saying that the present invention is not limited to the contents of the following description.
[0069]
  FIG. 8 is a block diagram showing a schematic configuration of a movie camera system 1 to which the present invention is applied. As shown in FIG. 8, a movie camera system 1 to which the moving image display system of the present invention is applied includes an image recording device 1a that is a movie camera and a moving image playback device 1b that is a personal computer over a network 1c that is the Internet. It is connected via.
[0070]
  In the following, an image recording apparatus 1a that exhibits the characteristic functions of the present invention will be described. The moving image playback apparatus 1b may be a standard system capable of decompressing code string data compressed by the Motion JPEG2000 method, and thus detailed description thereof is omitted.
[0071]
  As shown in FIG. 8, the image recording apparatus 1a is separated from an image input apparatus 2 that captures a moving image, an image compression apparatus 3 that compresses and encodes the captured image data, and a compression-encoded code string. And a code string conversion device 4 for compressing an image. The image processing apparatus of the present invention in which the image compression apparatus 3 and the code string conversion apparatus 4 compress moving image data is implemented.
[0072]
  FIG. 9 is a block diagram illustrating an example of a hardware configuration of the image recording apparatus 1a. As shown in FIG. 9, the image recording apparatus 1 a includes a CPU (Central Processing Unit) 11 that is a main part of a computer and controls each part centrally. The CPU 11 includes various ROMs (Read Only). A memory 12 that is a storage medium including a memory (RAM) and a RAM (Random Access Memory), a predetermined communication interface 13 that communicates with the network 1c, and an operation panel 18 that receives various operations from the user via the bus 14. It is connected.
[0073]
  In the image recording apparatus 1 a, in addition to the image input apparatus 2 and the image compression apparatus 3 described above, a logic circuit 19 is connected to the CPU 11 via the bus 14.
[0074]
  Control programs such as a moving image processing program for processing moving images are stored in the memory 12 (ROM) of the image recording apparatus 1a having such a configuration. This moving image processing program implements the program of the present invention. And the function of the code sequence converter 4 is implement | achieved by the process which CPU11 performs based on this moving image processing program.
[0075]
  As the memory 12, various types of media such as various optical disks such as CD and DVD, various magnetic disks such as various magneto-optical disks and flexible disks, and semiconductor memories can be used. Alternatively, the program may be downloaded from the network 1 c and installed in the memory 12. In this case, the storage device storing the program in the server on the transmission side is also a storage medium of the present invention. Note that the program may operate on a predetermined OS (Operating System), and in that case, the OS may take over the execution of some of the various processes described below, It may be included as a part of a group of program files constituting the application software or OS.
[0076]
  Here, the operation of each part of the image recording apparatus 1a will be briefly described. First, the image input device 2 of the image recording device 1a captures a moving image in units of frames using a photoelectric conversion device such as a CCD or a MOS image sensor, and outputs a digital pixel value signal of the moving image to the image compression device 3. . The image compression apparatus 3 compresses and encodes the digital pixel value signal of the moving image according to the “Motion JPEG2000 algorithm”. By the processing in the image compression device 3, the moving image data of the R, G, and B components of the original moving image is divided into one or a plurality of (usually a plurality of) tiles for each frame, and each tile is layered. Thus, the encoded data is compressed and encoded.
[0077]
  In the code string conversion device 4, the code amount of the code string data output from the image compression device 3 is reduced. The code data for each tile is reduced and converted into code string data with a smaller data amount. As shown in FIG. 10, the code string conversion apparatus 4 includes a code string input unit 41 that receives input of a code string, a syntax analysis unit 42, and a code amount reduction unit 43 that reduces the code amount of the code string (quantization table selection). Means 43a, packet switch 43b), code string creating means 44, code string output means 45 for outputting the code string, motion amount detecting means 46, subband code amount distribution detecting means 47, and input of the compression rate by the user. It comprises various means of the compression rate designation means 48 to be received, a quantization table group T, and the like. These various means are realized by processing performed by the CPU 11 in accordance with the above-described moving image processing program. In addition, when real-time property is regarded as important, it is necessary to speed up the processing. For this purpose, it is desirable to realize the functions of various means by the operation of the logic circuit 15.
[0078]
  Next, each part which comprises the code sequence converter 4 is demonstrated.
[0079]
  The syntax analysis means 42 decodes the syntax of the code string data input from the image compression apparatus 3, that is, the data described in the header part.
[0080]
  The code string creating unit 44 prepares the code string whose code amount has been reduced by the code amount reducing unit 43 into a standard format code string. By doing so, it is possible to decompress a newly generated code string with a general-purpose decoder.
[0081]
  The motion amount detection means 46 detects the motion amount between the current frame and the previous frame. For the amount of motion between the current frame and the previous frame detected by the motion amount detection means 46, a correlation coefficient value obtained from the difference in packet length between the current frame and the previous frame is used. A large correlation coefficient value means that images between frames are similar, and therefore corresponds to a small amount of motion.
[0082]
  Here, the detection of the motion amount by the motion amount detection means 46 will be described. Here, a method for detecting the amount of motion based on the data amount of the payload portion, that is, the “packet length” will be described. As shown in FIG. 11, since the data amount of the payload portion, that is, “packet length” is described in the header portion, the information is read by the syntax analysis means 42 to detect the packet length of each frame ( Packet length detection means), and stores the detected packet length of each frame (packet length storage means). Since the amount of motion appears in the amount of change in code amount, that is, the amount of change in packet length, the amount of motion is calculated by obtaining the difference in packet length between the current frame and the previous frame (difference detecting means). Can be detected. According to such a method, although there is a slight error in the amount of motion, the configuration of the image processing apparatus is simple and it has a great merit that it can be processed at high speed. . If the change in the code amount is smaller than a preset threshold value, it may be determined that there is no motion between frames.
[0083]
  Note that the amount of motion detection is not limited to that obtained from the difference in packet length between the current frame and the previous frame. For example, it can be obtained from the difference of wavelet coefficient values between the current frame and the previous frame. In this case, the payload data is decoded for each subband (decoding means), further inversely quantized (inverse quantization means), the wavelet coefficient value is directly obtained, and then the wavelet coefficient of each detected frame is determined. Numerical values are stored (wavelet coefficient value storage means). Thereafter, by using the wavelet coefficient values obtained in this way and comparing corresponding bits between frames (difference detecting means), the amount of motion can be detected. According to such a method, the configuration of the code string conversion apparatus becomes complicated and the processing time becomes long, but the amount of motion can be obtained accurately.
[0084]
  Here, FIG. 12 is an explanatory diagram for explaining “quantization” optimized in accordance with the “motion amount” of a moving image. The vertical axis represents the amount of motion, and the correlation coefficient value obtained from the difference between the wavelet coefficient values for each tile and each subband between the current frame and the previous frame is used. The horizontal axis represents the unit of tile. A tile is a rectangular area created by dividing an original still image in a frame. The tiles A, B, C, and D correspond to the tiles A, B, C, and D shown in FIG. FIG. 12 shows an example in which the user has selected k = p as the compression rate index value k via the compression rate specifying means 48. This compression rate index value is used when the code amount is roughly determined semi-quantitatively. The user first specifies this compression ratio index value. Note that k = 0 represents lossless encoding (lossless compression).
[0085]
  As shown in FIG. 12, in order to reflect different motion amounts for each image area, a plurality of threshold values are provided for the correlation coefficient value, and corresponding “motion amount index values” are assigned between the threshold values. Yes. A “quantization table” is prepared for each “motion amount index value”. Here, FIG. 13 is an explanatory diagram showing a specific example of the “quantization table” corresponding to the “motion amount index value”. As shown in FIG. 13, in this “quantization table”, a subband in which encoded data is reduced and a subband in which encoded data is stored are designated. In other words, in the quantization of each rectangular image area, according to the setting of the “code amount distribution index value”, the encoded data is matched to the target moving image processing such as the amount of motion to be noticed and the amount of motion to be ignored. Can be determined and subbands to be reduced.
[0086]
  That is, as shown in FIG. 12, a tile A with a large amount of motion is given i = 9 as a motion amount index value, and a tile C with a small amount of motion is given i = 1 as a motion amount index value. Quantization is performed using a “quantization table” corresponding to each index value.
[0087]
  By reflecting the “motion amount” of the moving image in the quantization in this way, it is possible to significantly reduce the “distortion caused by the haze” that appears in the background, which is an obstacle in the conventional method.
[0088]
  The subband code amount distribution detecting means 47 detects the subband code amount distribution in the current frame. Here, detection of the subband code amount distribution by the subband code amount distribution detecting means 47 will be described. Here, a method of detecting the subband code amount distribution based on the subband code amount of the payload portion, that is, the “packet length” will be described. As described above, since the subband code amount of the payload portion, that is, “packet length” is described in the header portion, the information is read by the parsing means 42 to detect the packet length of the frame (packet length). Length detection means), the subband code amount distribution can be detected based on the packet length of the detected frame.
[0089]
  Here, FIG. 14 is an explanatory diagram for explaining “quantization” optimized according to “subband code amount distribution”. The vertical axis represents the relative code amount when the low-frequency subband code amount in the highest layer (the maximum number of decomposition levels) is normalized to “1”. The horizontal axis represents the subband frequency band. The thin line portion of the curve indicating the code amount distribution for each tile represents the subband from which the code amount is to be deleted, while the thick line portion represents the subband in which the code amount is stored as it is. . The tiles A, B, C, and D correspond to the tiles A, B, C, and D shown in FIG. 14A shows an example in which the user has selected k = p, which is a low compression rate, as the compression rate index value k via the compression rate designating unit 48, and FIG. In this example, k = q, which is a high compression ratio, is selected as the compression ratio index value k via the ratio specifying means 48. As shown in FIG. 14, the tile A and the tile D hold code data up to a high frequency range. Further, the tile B has a relatively large amount of information in the middle region, and the tile C has a large amount of information in the low region. As described above, since the distribution of the encoded data amount of the subband is different for each tile, three types of “code amount distribution index values” are assigned to the subband code amount distribution here in the quantization process. . A corresponding “quantization table” is prepared for each “code amount distribution index value”. Here, FIG. 15 is an explanatory diagram showing a specific example of the “quantization table” corresponding to the “code amount distribution index value”. As shown in FIG. 15, in the “quantization table”, subbands (parts indicated by hatching) in which the code amount is reduced are set corresponding to the “code amount distribution index value”.
[0090]
  That is, as shown in FIG. 14, in this “quantization table”, j = H is used as the code amount distribution index value for tile A and tile D with a large distribution of code data amount in the high region, and distribution in the middle region. J = M as the code amount distribution index value for many tiles B, and as the code amount distribution index value for tiles C with many distributions in the low bandj = LRespectively.
[0091]
  As a result, the subband code data can be reduced according to the code amount distribution index value. Therefore, as shown in FIG. 14B, the image quality degradation of the tiles A, B, and D when the compression rate is high. Thus, the image quality equivalent to that of the tile C can be maintained.
[0092]
  FIG. 16 is another explanatory diagram for explaining “quantization” optimized according to “subband code amount distribution”. FIG. 16A shows an example in which the user selects k = p, which is a low compression rate, as the compression rate index value k via the compression rate designating unit 48, and FIG. This is an example in which k = q, which is a high compression ratio, is selected as the compression ratio index value k via the ratio specifying means 48. Here, the vertical axis represents the sum of code amounts of all subbands, and the horizontal axis represents tile #. A tile in which image information is gathered in the low frequency subband has a small code amount sum for all subbands. Conversely, a tile in which image information is concentrated in the high frequency region has a large code amount sum. The code amount of each tile is reduced to the minimum code amount sum provided in the classified category.
[0093]
  In this way, since the rate of reducing the code amount is determined according to the code amount before quantization, when the code data after quantization is decoded and displayed as an image, all of the high range to low range are displayed. The degradation of image quality is not so conspicuous over the bandwidth. This is because, unlike the prior art, the code amount before quantization is not taken into consideration, and the code amount is not reduced uniformly.
[0094]
  That is, as shown in FIG. 17, the quantization table group T according to the present embodiment includes a “motion amount index value (i = 0, 1,..., 9)” and a “code amount distribution index value (j = L, M, H) ”and“ compression ratio index values (k = 0, 1,..., N) ”have quantization tables in a three-dimensional matrix (i, j, k). is there.
[0095]
  The quantization table group T will be described in more detail. FIG. 18 shows the relationship between the “motion amount index value” and the “compression rate index value” of the quantization table group T. As described above, the “motion amount index value” is classified according to the motion amount and a preset threshold value, and the “compression rate index value” is designated by the user. In this example, the “motion amount index value” is classified into nine index values, and the “compression rate index value” is classified into index values from lossless (k = 0) to the highest compression rate (k = n). ing. As shown in FIG. 18, when k = p is selected as the value of the “compression ratio index value”, the quantization table for tile A is (9, p), and the quantization table for tile B is (2, p )
[0096]
  FIG. 19 shows the relationship between the “code amount distribution index value” and the “compression rate index value” of the quantization table group T. As described above, the “code amount distribution index value” is classified into several types of patterns according to the characteristics of each rectangular image area (tile), and the “compression rate index value” is designated by the user. In this example, the “code amount distribution index value” is classified into three index values, and the “compression rate index value” is classified from the lossless (k = 0) to the highest compression rate (k = n) index value. Has been. As shown in FIG. 19, when k = p is selected as the value of “compression ratio index value”, the quantization index value of tile A is (H, p), and the quantization index value of tile B is (M , P).
[0097]
  That is, the quantization table selection unit 43a of the code amount reduction unit 43 is obtained by classifying the features of each rectangular image region by using the motion amount detection unit 46, the subband code amount distribution detection unit 47, and the compression rate designation unit 48. The quantization table group T is referred to according to the “index value” (i, j, k), and the optimum quantization table is selected. For example, as shown in FIG. 17, when k = p is selected as the “compression ratio index value”, the quantization table for tile A is (9, H, p), and the quantization table for tile B is (2 , M, p).
[0098]
  When the optimum quantization table is selected by the quantization table selection unit 43a, the packet switch 43b is turned off, and the code amount of the code string is reduced by the code amount reduction unit 43. At the same time, the code string creating unit 44 rewrites the header data of the code stream of the code string whose code amount has been reduced by the code amount reducing unit 43 to prepare a standard format code string. Thus, the generated code string is output to the network 1c by the code string output means 45.
[0099]
  On the other hand, when the optimum quantization table is not selected by the quantization table selection unit 43a, the packet switch 43b remains on and the code sequence data of the current frame is not deleted. The code string output means 45 outputs the result to the network 1c.
[0100]
  In the present embodiment, “motion amount index value (i = 0, 1,..., 9)”, “code amount distribution index value (j = L, M, H)”, and “compression rate index”. The quantization table group T having the quantization table relating to the value (k = 0, 1,..., N) ”in the form of a three-dimensional matrix (i, j, k) is used. It is not a thing. For example, the quantization table relating to “motion amount index value (i = 0, 1,..., 9)” and “compression ratio index value (k = 0, 1,..., N)” is two-dimensionally represented. A quantization table group having a matrix (i, k) shape may be used. In this case, as shown in FIG. 20, the subband code amount distribution detecting means 47 is not necessary in the code string conversion device 4. In such a case, the image quality is slightly lowered as compared with the case where the quantization table group T which is a three-dimensional matrix (i, j, k) is used, but the processing speed can be increased. .
[0101]
  In addition, a quantization table related to “code amount distribution index value (j = L, M, H)” and “compression rate index value (k = 0, 1,..., N)” is represented as a two-dimensional matrix (j , K) may be used. In this case, as shown in FIG. 21, the motion amount detection means 46 is not required in the code string conversion device 4. In such a case, the image quality is slightly lowered as compared with the case where the quantization table group T which is a three-dimensional matrix (i, j, k) is used, but the processing speed can be increased. . In addition, according to such a code string conversion device 4, since it is not necessary to detect the amount of motion, it is possible to apply not only to moving images but also to still images.
[0102]
  Here, the motion amount and subband code amount distribution detected from code sequence data created by orthogonally transforming the pixel values of moving image data for each rectangular area and hierarchically compressing and coding, and specified by the user In accordance with the compression rate, the code amount of the code string data is reduced for each rectangular area. As a result, quantization is performed in consideration of not only the compression rate but also the amount of motion and the subband code amount distribution, so that quantization is performed while suppressing deterioration in image quality due to the amount of motion and the level of image frequency. It becomes possible.
[0103]
  When the input original image is composed of a plurality of components, it is possible to further increase the speed of the entire system. That is, for example, when the input code string is a YCbCr signal, the packet length may be detected using only the Y component. This is because the quantization table for the Cb and Cr components can be easily selected from the quantization table group T based on the quantization table obtained for the Y component.
[0104]
  In this embodiment, the user designates the compression rate via the compression rate designation means 48, but the present invention is not limited to this. For example, a bit rate may be designated instead of the compression rate, and the compression rate index value k may be set based on the designated bit rate.
[0105]
  Next, a second embodiment of the present invention will be described with reference to FIGS. In addition, the same part as 1st Embodiment mentioned above is shown with the same code | symbol, and abbreviate | omits description.
[0106]
  As described in the first embodiment, “code amount distribution index value (j = L, M, H)” and “compression ratio index value (k = 0, 1,..., N)” When a quantization table group having such a quantization table in a two-dimensional matrix (j, k) is used, the motion amount detection means 46 is not necessary in the code string conversion device 4, so that not only a moving image, It can also be applied to still images.
[0107]
  Here, FIG. 22 is a block diagram showing a configuration of the code string conversion device 4 of the present embodiment. As shown in FIG. 22, the code string conversion device 4 of the present exemplary embodiment includes an index value changing unit 50 in addition to the configuration shown in FIG. 21. The index value changing means 50 changes the index value (j, k) determined by the subband code amount distribution detecting means 47 and the compression rate specifying means 48 to (j *, k *).
[0108]
  Here, the index value changing unit 50 will be described with a specific example. FIG. 23 is an image example for describing encoding of a still image in consideration of the subband code amount distribution. As shown in FIG. 23, the tile U has a blue sky, the tile W has a dense front side of a high-rise building window frame, and the tile V has both a high-rise building and the sky as main images. As for the subband code amount distribution, the image information is gathered in the tile U for the low frequency, the tile W for the high frequency, and the tile V in the entire range from the low frequency to the high frequency. Here, since the image information of the tile V spreads over the entire range from the low frequency to the high frequency, when the tile V is quantized, the front side of the high frequency building (tile W) and the low frequency ( It is necessary to make sure that the image quality balance with both of the tiles U) is not lost. However, this is very difficult and has been a cause of image quality degradation. Moreover, as the compression ratio increases, finding an optimal solution tends to become increasingly difficult.
[0109]
  Therefore, in the present embodiment, by using the index value changing means 50, an index value having a smaller compression rate than a predetermined compression rate index value is applied to the rectangular area corresponding to the boundary between the sky and the building. I try to reassign. Specifically, as shown in FIG. 24, lossless compression (k = 0) is performed at a low compression rate (compression rate index value k = p), and lower compression is performed at a medium compression rate (compression rate index value k = q). The index value (k = pm) on the rate side is used, and the index value k = rn on the lower compression rate side is used for the high compression rate (compression rate index value k = r). Thus, by changing the index value, it is possible to encode the original image while maintaining high image quality.
[0110]
  In each of the embodiments, the case where the tile division process is performed on the original image has been described. Even when tile division is not performed on the original image, if the precinct or code block is used as a rectangular area, the “motion amount” and “subband code amount distribution” can be finely divided as in the case of tile division. It is possible to detect and quantize with reduced image quality degradation.
[0111]
  In the above description, the image recording apparatus 1a of the present invention is applied to a movie camera. However, the image recording apparatus 1a is applied to an information terminal apparatus such as a personal digital assistant (PDA) or a mobile phone. You can also.
[0112]
【The invention's effect】
  According to the code string conversion apparatus, the image processing apparatus including the code string conversion apparatus, the image recording layer, and the code string conversion method of the present invention, the following effects can be obtained.
(1)The motion amount and subband code amount distribution detected from code sequence data created by orthogonally transforming moving image data for each rectangular area and hierarchically compressing and encoding, and the compression rate specified by the user Accordingly, by reducing the code amount of the code string data for each rectangular area, quantization considering not only the compression rate but also the motion amount and the subband code amount distribution is performed. It is possible to perform quantization while suppressing image quality deterioration associated with high and low frequencies.
[0113]
(2)Quantization including fine-grained image quality control by selecting the optimum quantization table from the quantization table group and reducing the code amount of code string data for each rectangular area based on the selected quantization table Can be executed.
[0114]
(3)Depending on the amount of motion detected from the code string data created by orthogonally transforming the pixel values of the moving image data for each rectangular area and hierarchically compressing and coding, and the compression rate specified by the user, the code string data By reducing the code amount for each rectangular area, quantization is performed in consideration of not only the compression rate but also the amount of motion, so that it is possible to perform quantization while suppressing deterioration in image quality due to the amount of motion.
[0115]
(4)According to the subband code amount distribution detected from the code string data created by orthogonally transforming the pixel values of the moving image data for each rectangular area and hierarchically compressing and encoding, and the compression rate specified by the user By reducing the code amount of the column data for each rectangular area, quantization that takes into account not only the compression rate but also the subband code amount distribution is performed, so quantization that suppresses image quality degradation due to high and low image frequencies It can be performed.
[0116]
(5)Depending on the subband code amount distribution detected from the code string data created by orthogonally transforming the pixel values of the still image data for each rectangular area and hierarchically compressing and coding, and the compression rate specified by the user By reducing the code amount of the column data for each rectangular area, quantization that takes into account not only the compression rate but also the subband code amount distribution is performed, so quantization that suppresses image quality degradation due to high and low image frequencies It can be performed.
[0117]
(6)The quantization table can reliably reduce the code amount of the code string data by designating the subband in which the code amount is reduced and the subband in which the code amount is stored.
[0118]
(7)By providing index value changing means for changing various set index values, it is possible to encode the original image while maintaining high image quality.
[0119]
(8)By preparing the code string data in which the code amount is reduced for each rectangular area into the code string data in the standard format, the newly generated code string data can be expanded by a general-purpose decoder.
[0120]
(9)Since only the Y (luminance) component of the YCbCr component and the G (green) component of the RGB component can be detected, the “motion amount” and the “subband code amount distribution” can be detected, so that encoding is performed at high speed and with low power consumption. Control can be performed.
[0121]
(10)Since the rectangular area is a tile, discrete wavelet transform can be used as orthogonal transform.
[0122]
(11)Since the rectangular area is a precinct, even when tile division is not performed (all image areas = tiles), it is possible to perform quantization that suppresses image quality degradation in the same manner as in tile units. Further, the detection unit can be an image area smaller than the tile.
[0123]
(12)Since the rectangular area is a code block, even when tile division is not performed (all image areas = tiles), quantization that suppresses image quality degradation can be performed in the same manner as in tile units. Further, the detection unit can be an image area smaller than the tile.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a system that realizes a hierarchical encoding algorithm that is the basis of a JPEG2000 system that is a premise of the present invention.
FIG. 2 is an explanatory diagram showing a divided rectangular area of each component of the original image.
FIG. 3 is an explanatory diagram showing subbands at each decomposition level when the number of decomposition levels is 3. FIG.
FIG. 4 is an explanatory diagram showing a precinct.
FIG. 5 is an explanatory diagram showing an example of a procedure for ranking bit planes;
FIG. 6 is an explanatory diagram illustrating a schematic configuration of one frame of code string data.
FIG. 7 is an explanatory diagram showing the concept of Motion JPEG2000.
FIG. 8 is a block diagram showing a schematic configuration of the movie camera system according to the first embodiment of the present invention.
FIG. 9 is a block diagram illustrating an example of a hardware configuration of an image recording apparatus.
FIG. 10 is a block diagram illustrating a configuration of a code string conversion apparatus.
FIG. 11 is an explanatory diagram showing detection of a packet length by a syntax analysis unit.
FIG. 12 is an explanatory diagram for explaining “quantization” optimized according to “motion amount” of a moving image;
FIG. 13 is an explanatory diagram of a specific example of a “quantization table” corresponding to a “motion amount index value”.
FIG. 14 is an explanatory diagram for explaining “quantization” optimized according to “subband code amount distribution”;
FIG. 15 is an explanatory diagram illustrating a specific example of a “quantization table” corresponding to a “code amount distribution index value”;
FIG. 16 is another explanatory diagram illustrating “quantization” optimized according to “subband code amount distribution”;
FIG. 17 is an explanatory diagram showing a quantization table group considering both motion amount and subband code amount distribution;
FIG. 18 is an explanatory diagram showing a relationship between a “motion amount index value” and a “compression ratio index value” in a quantization table group.
FIG. 19 is an explanatory diagram illustrating a relationship between a “code amount distribution index value” and a “compression rate index value” in a quantization table group;
FIG. 20 is a block diagram illustrating a configuration of another code string converter.
FIG. 21 is a block diagram illustrating a configuration of another code string converter.
FIG. 22 is a block diagram showing a configuration of a code string conversion apparatus according to a second embodiment of the present invention.
FIG. 23 is an explanatory diagram illustrating an image example for describing encoding of a still image in consideration of a subband code amount distribution;
FIG. 24 is an explanatory diagram of an example of changing an index value.
FIG. 25 is an explanatory diagram showing a conventional “quantization” method;
FIG. 26 is an explanatory diagram showing a conventional “quantization” method;
FIG. 27 is an explanatory diagram showing three consecutive frames extracted from moving image content.
FIG. 28 is an explanatory diagram showing a table showing the motion amount and pixel frequency of a predetermined tile in each frame shown in FIG. 27;
[Explanation of symbols]
1a Image recording device
2 Image input device
3 Image compression device
4 Code string converter
12 storage media
42 Syntax analysis means
43 Code amount reduction means
43a Quantization table selection means
44 Code string creation means
46 Movement amount detection means
47 Subband code amount distribution detecting means
48 Compression rate designation means
50 Index value changing means
T quantization table group

Claims (20)

Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing means for parsing
Based on the analysis result of the parsing means, a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame is obtained, and a motion amount between the current frame and the previous frame is detected, and the motion A motion amount detection means for setting the amount to one of a plurality of predetermined motion amount index values;
Based on the analysis result of the parsing means, a subband code amount distribution in the current frame is detected, and the subband code amount distribution is set to one of a plurality of predetermined subband code amount distribution index values. Subband code amount distribution detecting means;
A compression rate designating unit that accepts designation of a compression rate and sets the compression rate to any one of a plurality of predetermined compression rate index values;
A quantization table group having a plurality of quantization tables according to combinations of values of a plurality of motion amount index values, a plurality of subband code amount distribution index values, and a plurality of compression rate index values in a three-dimensional matrix;
Based on the motion amount index value set by the motion amount detection unit, the subband code amount distribution index value set by the subband code amount distribution detection unit, and the compression rate index value set by the compression rate designating unit. A code amount for selecting a predetermined quantization table from the quantization table group, quantizing the code of the code string data by the selected quantization table, and reducing the code amount of the code string data for each rectangular area Reduction measures,
A code string conversion device comprising: Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing means for parsing
Based on the analysis result of the parsing means, a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame is obtained, and a motion amount between the current frame and the previous frame is detected, and the motion A motion amount detection means for setting the amount to one of a plurality of predetermined motion amount index values;
A compression rate designating unit that accepts designation of a compression rate and sets the compression rate to any one of a plurality of predetermined compression rate index values;
A quantization table group having a plurality of quantization tables according to combinations of values of a plurality of motion amount index values and a plurality of compression ratio index values in a two-dimensional matrix;
Based on the motion amount index value set by the motion amount detection means and the compression rate index value set by the compression rate specifying means, a predetermined quantization table is selected from the quantization table group, and the selected quantum table is selected. A code amount reduction unit that quantizes the code of the code string data using a conversion table and reduces the code amount of the code string data for each rectangular area;
A code string conversion device comprising: Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing means for parsing
Based on the analysis result of the parsing means, a subband code amount distribution in the current frame is detected, and the subband code amount distribution is set to one of a plurality of predetermined subband code amount distribution index values. Subband code amount distribution detecting means;
A compression rate designating unit that accepts designation of a compression rate and sets the compression rate to any one of a plurality of predetermined compression rate index values;
A quantization table group having a plurality of quantization tables according to combinations of values of a plurality of subband code amount distribution index values and a plurality of compression ratio index values in a two-dimensional matrix;
A predetermined quantization table is selected from the quantization table group based on the subband code amount distribution index value set by the subband code amount distribution detection unit and the compression rate index value set by the compression rate designation unit Code amount reduction means for quantizing the code of the code string data by the selected quantization table and reducing the code amount of the code string data for each rectangular area;
A code string conversion device comprising: Parsing still image data into a plurality of rectangular areas, parsing the pixel values for each rectangular area and analyzing the data described in the header of the code string data obtained by hierarchical compression coding Means,
A subband code amount that detects a subband code amount distribution based on an analysis result of the parsing means and sets the subband code amount distribution to one of a plurality of predetermined subband code amount distribution index values Distribution detection means;
A compression rate designating unit that accepts designation of a compression rate and sets the compression rate to any one of a plurality of predetermined compression rate index values;
A quantization table group having a plurality of quantization tables according to combinations of values of a plurality of subband code amount distribution index values and a plurality of compression ratio index values in a two-dimensional matrix;
A predetermined quantization table is selected from the quantization table group based on the subband code amount distribution index value set by the subband code amount distribution detection unit and the compression rate index value set by the compression rate designation unit Code amount reduction means for quantizing the code of the code string data by the selected quantization table and reducing the code amount of the code string data for each rectangular area;
A code string conversion device comprising: The quantization table, a subband coding amount is reduced, the code sequence conversion apparatus according to claims 1, characterized in that specifying the subband any one of 4 the code amount is stored .   3. The code string conversion device according to claim 1, further comprising index value changing means for changing a motion amount index value set by the motion amount detecting means.   5. The code string according to claim 1, further comprising: an index value changing unit that changes a subband code amount distribution index value set by the subband code amount distribution detecting unit. Conversion device.   The code sequence creation means for adjusting the code sequence data, the code amount of which has been reduced for each rectangular area by the code amount reduction means, into standard format code sequence data. The code string conversion device described in 1.   The code string conversion device according to any one of claims 1 to 8, wherein the processing of each means is executed for each component.   The code string conversion device according to claim 1, wherein the rectangular area is a tile.   The code string conversion device according to claim 1, wherein the rectangular area is a precinct. The rectangular region, the code sequence conversion apparatus according to any one of claims 1 to 9, characterized in that a code block. An image compression device that divides moving image data into one or a plurality of rectangular regions for each frame, orthogonally transforms pixel values for each rectangular region, and hierarchically compresses and encodes the pixel values;
The code string conversion device according to any one of claims 1 to 3,
An image processing apparatus comprising: An image compression apparatus that divides still image data into one or a plurality of rectangular areas, orthogonally transforms pixel values for each rectangular area, and hierarchically compresses and encodes the pixel values;
A code string conversion device according to claim 4;
An image processing apparatus comprising: An image input device for capturing moving image data;
The image processing apparatus according to claim 13, which processes the captured moving image data;
An image recording apparatus comprising: An image input device for capturing still image data;
The image processing device according to claim 14, which processes the captured still image data;
An image recording apparatus comprising: Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing process for parsing
Based on the analysis result of the parsing step, a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame is obtained, and a motion amount between the current frame and the previous frame is detected, and the motion A motion amount detection step of setting the amount to one of a plurality of predetermined motion amount index values;
Based on the analysis result of the parsing process, a subband code amount distribution in the current frame is detected, and the subband code amount distribution is set to one of a plurality of predetermined subband code amount distribution index values. A subband code amount distribution detection step;
A compression ratio designating step of accepting designation of a compression ratio and setting the compression ratio to any one of a plurality of predetermined compression ratio index values;
By using a quantization table group having a plurality of quantization tables related to combinations of values of a plurality of motion amount index values, a plurality of subband code amount distribution index values, and a plurality of compression rate index values in a three-dimensional matrix form The motion amount index value set in the motion amount detection step, the subband code amount distribution index value set in the subband code amount distribution detection step, and the compression rate index value set in the compression rate specifying means A code that selects a predetermined quantization table from the quantization table group, quantizes the code of the code string data by the selected quantization table, and reduces the code amount of the code string data for each rectangular area Volume reduction process,
A code string conversion method comprising: Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing process for parsing
Based on the analysis result of the parsing step, a correlation coefficient value of the data amount of the packet length between the current frame and the previous frame is obtained, and a motion amount between the current frame and the previous frame is detected, and the motion A motion amount detection step of setting the amount to one of a plurality of predetermined motion amount index values;
A compression ratio designating step of accepting designation of a compression ratio and setting the compression ratio to any one of a plurality of predetermined compression ratio index values;
The quantization table group having a plurality of quantization tables related to a combination of each of a plurality of motion amount index values and a plurality of compression rate index values in a two-dimensional matrix form is set in the motion amount detection step. A predetermined quantization table is selected from the quantization table group based on the motion amount index value and the compression rate index value set in the compression rate designating step, and the code sequence data code is selected by the selected quantization table. And a code amount reduction step of reducing the code amount of the code string data for each rectangular area;
A code string conversion method comprising: Data described in the header portion of code string data obtained by dividing moving image data into one or a plurality of rectangular areas for each frame, orthogonally transforming pixel values for each rectangular area, and hierarchically compressing and encoding. A parsing process for parsing
Based on the analysis result of the parsing process, a subband code amount distribution in the current frame is detected, and the subband code amount distribution is set to one of a plurality of predetermined subband code amount distribution index values. A subband code amount distribution detection step;
A compression ratio designating step of accepting designation of a compression ratio and setting the compression ratio to any one of a plurality of predetermined compression ratio index values;
Using the quantization table group having a plurality of quantization tables related to a combination of values of a plurality of subband code amount distribution index values and a plurality of compression ratio index values in a two-dimensional matrix , the subband code amount distribution Based on the subband code amount distribution index value set in the detection step and the compression rate index value set in the compression rate designation step, a predetermined quantization table is selected from the quantization table group, and the selected quantum table A code amount reduction step of quantizing the code of the code string data by a conversion table and reducing the code amount of the code string data for each rectangular region;
A code string conversion method comprising: Parsing still image data into a plurality of rectangular areas, parsing the pixel values for each rectangular area and analyzing the data described in the header of the code string data obtained by hierarchical compression coding Process,
A subband code amount that detects a subband code amount distribution based on the analysis result of the syntax analysis step and sets the subband code amount distribution to one of a plurality of predetermined subband code amount distribution index values A distribution detection step;
A compression ratio designating step of accepting designation of a compression ratio and setting the compression ratio to any one of a plurality of predetermined compression ratio index values;
Using the quantization table group having a plurality of quantization tables related to a combination of values of a plurality of subband code amount distribution index values and a plurality of compression ratio index values in a two-dimensional matrix , the subband code amount distribution Based on the subband code amount distribution index value set in the detection step and the compression rate index value set in the compression rate designation step, a predetermined quantization table is selected from the quantization table group, and the selected quantum table is selected. A code amount reduction step of quantizing the code of the code string data by a conversion table and reducing the code amount of the code string data for each rectangular region;
A code string conversion method comprising:

Images