JP7259280B2 - Encoding device, encoding method and encoding program - Google Patents

Description

The present invention relates to an encoding device, an encoding method and an encoding program.

2. Description of the Related Art In recent years, digital cameras, MFPs (Multi-Function Peripherals), and the like have come to perform various types of image processing. These image processes are functions realized by software processing using hardware such as a CPU and a semiconductor memory. Data to be processed in image processing is image data that tends to have a large data size, and the processing content is also complicated and large in volume.

Therefore, since a large amount of access processing to the semiconductor memory occurs during execution of image processing, reducing the amount of access leads to improvement in efficiency of image processing. Therefore, as a method for minimizing the size of data used for memory access processing in order to reduce the amount of access, a method of compressing image data to reduce the amount of data is known. As one example, a known data processing technique uses discrete cosine transform (DCT) to transform image data, which is a discrete signal, into the frequency domain and encodes the data into fixed-length bits.

For example, assume image processing in a digital camera. In this case, RGB image data based on the subject image obtained through the optical system is converted into YUV image data, and moving image compression processing is performed, and image processing for recording the moving image data is required. Among them, for example, when executing distortion correction processing using geometric transformation, a large amount of memory access occurs. leads to

When performing fixed-length coding of image data, the vertical, horizontal, and diagonal orientations of edges in the image are recognized from the DCT coefficients output by frequency transform processing, and limiter processing is performed by switching the DCT coefficient limiters to perform fixed-length encoding. A coding technique has been disclosed (for example, Patent Document 1).

When converting YUV image data from RGB image data into fixed-length code, the fixed-length code includes bit strings corresponding to a luminance component (Y component) and two color difference components (U component and V component). included. Therefore, among the fixed-length codes, the bit length of the DCT component of the luminance (Y) associated with the portion of the original image with weak edges (the portion where the scalar amount of the YUV component is low or medium) is set to a certain length. Otherwise, the edges of the decoded image will be dull.

On the other hand, in order to suppress the deterioration of the image quality related to the decoded image data, if the bit length of the DCT component of the brightness of the part related to the low and medium scalar amount is set to a certain extent, the (number of bits) of the DCT components of the U component and the V component must be shortened. Reducing the bit length of the DCT components of the U component and the V component (reducing the amount of data) will reduce the color reproducibility of the decoded image.

Further, in the technique disclosed in Japanese Patent Laid-Open No. 2002-100000, edges are recognized by roughly classifying them into three types: vertical, horizontal, and oblique, based on the DCT coefficients. With this technique, it is difficult to fully grasp the features of each edge. Further, in the technique disclosed in Patent Document 1, the limiting value (limiter) for limiting the DCT coefficient is uniformly set regardless of the scalar amount of the DCT coefficient, so limiter processing is uniformly performed on the DCT coefficient. will be This technique has a problem in the reproducibility of edges included in the decoded image.

That is, even if the conventional technology represented by Patent Document 1 is used, there is still a problem in suppressing the impact on image quality while achieving a high compression rate in order to reduce the amount of memory access during image processing. be.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an encoding apparatus capable of enhancing reproducibility of edges in an original image even when fixed-length codes are used in order to reduce the amount of memory access during image processing.

In order to solve the above technical problem, one aspect of the present invention is an encoding device that converts an image block obtained by dividing an image including a plurality of pixels into a predetermined number of pixels into a fixed-length code, the encoding device comprising: frequency transforming means for performing frequency transform processing on the image block and outputting frequency transform coefficients corresponding to the image block; and the frequency corresponding to each pixel included in the image block. condition-satisfactory frequency transform coefficient specifying means for specifying a frequency transform coefficient whose absolute value satisfies a predetermined condition as a condition-satisfactory frequency transform coefficient; and said frequency transform corresponding to said pixel based on said condition-satisfactory frequency transform coefficient. limit value selection means for selecting a limit value group including limit values for individually limiting coefficient values; limiting processing means for executing limiting processing for limiting values of frequency transform coefficients; and encoding means for encoding the frequency transform coefficients after the limiting processing has been executed as fixed-length codes, wherein the limiting processing means, in limiting processing for the condition-satisfactory frequency transform coefficients included in the frequency transform coefficients, limit the condition-satisfactory frequency transform coefficients by using a value smaller than a limit value included in the selected limit value group; It is characterized by limiting the value.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an encoding apparatus capable of improving reproducibility of edges in an original image even when fixed-length codes are used in order to reduce the amount of memory access during image processing. Even if fixed-length coding is performed in order to reduce the amount of memory access during image processing, reproducibility of the original image can be ensured.

1 is a configuration diagram showing an embodiment of an image forming system to which an encoding device and a decoding device according to the present invention can be applied; FIG. FIG. 2 is a functional block diagram of the image forming system; 4A and 4B are diagrams showing examples of image data (a) and fixed-length codes (b) to be processed in the present embodiment; The figure which shows the functional block of the encoding apparatus which concerns on this invention. FIG. 4 is a diagram showing an image of DCT coefficients according to the present embodiment; FIG. 4 is a diagram showing an image of DCT coefficients according to the present embodiment, in which specific coefficients are distinguished; FIG. 4 is a diagram showing an image of a limit value group used by the encoding device according to the embodiment; FIG. 4 is a diagram showing an example of the data structure of limit values used by the encoding device according to the embodiment; FIG. 4 is a diagram showing an example of a fixed-length code format generated by the encoding device according to the present embodiment; FIG. 4 is a diagram showing an image of fixed-length code generation in the encoding device according to the present embodiment; The figure which shows the functional block of the decoding apparatus which concerns on this invention. FIG. 4 is a diagram showing an image of conversion from a fixed-length code format to DCT coefficients in decoding according to the present embodiment; FIG. 2 is a functional configuration diagram of an image processing block included in the encoding device according to the embodiment; FIG. 2 is a functional configuration diagram of a variable-length video compression block included in the encoding device according to the embodiment; 1 is a flow chart showing an embodiment of an image processing method including an encoding method and a decoding method according to the present invention; 4 is a flow chart illustrating an embodiment of an encoding method according to the present invention; 4 is a flowchart illustrating a detailed embodiment of an encoding method according to the invention; 4 is a flowchart illustrating a detailed embodiment of an encoding method according to the invention; FIG. 4 is a diagram showing an image of limiter processing included in the encoding method according to the present invention; 4 is a flow chart illustrating an embodiment of a decoding method according to the present invention;

[Summary of the present invention]
First, the gist of the present invention will be outlined. An encoding apparatus according to the present invention is an apparatus that applies frequency transform processing using a discrete cosine transform (DCT) to encode image data into fixed-length data. A decoding device according to the present invention corresponds to the encoding device and is a device for decoding fixed-length data into image data. The discrete cosine transform is hereinafter referred to as "DCT". Coefficients obtained by frequency transform processing using DCT are referred to as "DCT coefficients".

The apparatus according to the present invention encodes image data into a fixed-length bit string of 144 bits. By performing image processing using this, the amount of data related to memory access processing is reduced, thereby reducing the memory access cost. In addition, by applying it to an ASIC (Application Specific Integrated Circuit) that performs image processing, it is possible to reduce the current consumption of the ASIC, simplify the cooling mechanism of the ASIC, and reduce the manufacturing cost of the image processing device. .

The encoding method and decoding method executed by the encoding device and decoding device according to the present invention divide an image into image blocks each having a predetermined pixel size, and then divide each image block into fixed-length bit strings (fixed long code). In the present invention, the Y (brightness) DCT coefficient of the high scalar (large scalar amount) image block included in the fixed-length code is suppressed to the shortest possible bit length within the range in which the image quality after decoding can be compensated. to

Similar to the encoding method and decoding method according to the present invention, in the prior art using frequency transform processing using DCT, for example, image data is divided into 8-pixel by 8-pixel units, and the frequency using DCT is divided into image blocks. A transform process is performed to obtain the DCT coefficients. Characteristic determination is performed on the DCT coefficients, and before performing fixed-length coding after quantization of the DCT coefficients, a "limiter processing TYPE" for adjusting the length of the bit string is specified, and the DCT coefficients are quantized. become After quantization, fixed-length encoding processing such as limiter processing is executed based on the TYPE of limiter processing. In such a conventional technique, features are extracted in the vertical/horizontal/diagonal directions of the DCT coefficients, and limiter processing is performed while varying limit values used in the limiter processing for limiting the DCT coefficients.

However, in the prior art, since the scalar amount of the DCT coefficients is not considered in the limiter process, the difference (undulation) in the scalar amount of the DCT coefficients is flattened by the limiter process. As a result, the feature of the image block subjected to the limiter processing is lost, and especially the edge of the image having high frequencies is degraded.

In this respect, according to the encoding apparatus of the present invention, YUV image data is subjected to frequency transform processing using DCT to calculate DCT coefficients, which are converted into fixed-length codes and stored in memory for data access processing. Access volume can be reduced. Further, according to the decoding device according to the present invention, deterioration of image quality can be suppressed by decoding the fixed-length code generated by the encoding device. That is, by using the encoding device and the decoding device according to the present invention, the amount of memory access when executing processing that requires a large amount of memory access (for example, geometric transformation processing such as distortion correction processing for image data) can be reduced. It is possible to secure the reproducibility of the image after decoding while reducing it. These features and possible embodiments are described below.

[Embodiment of device provided with encoding device/decoding device according to the present invention]
INDUSTRIAL APPLICABILITY The encoding device and decoding device according to the present invention are applicable, for example, to an image processing system that captures and processes images. FIG. 1 shows a configuration example of an image processing system for describing embodiments of an encoding device and a decoding device according to the present invention. In the following description, the image processing system according to this embodiment is assumed to be a digital camera 1 having a function of capturing and recording a subject image.

The digital camera 1 according to this embodiment includes an optical system 20, an image sensor 30, an A/D converter 40, a processor 10 that performs image processing, a memory controller 50, a fixed length memory 60, and a moving image memory .

The optical system 20 includes an optical lens and the like, and forms a subject image on the imaging surface of the imaging device 30 . The optical lens included in the optical system 20 is, for example, a wide-angle lens. By providing two optical lenses, it is possible to converge the reflected light of the subject within a range of 180 degrees.

The imaging element 30 is a photoelectric conversion element that converts the light condensed by the optical system 20 into an electrical signal and appropriately outputs the image signal at predetermined time intervals. The imaging device 30 can use, for example, a solid-state imaging device such as a CCD sensor or a CMOS sensor.

The A/D converter 40 converts the image signal output from the imaging element 30 into digital data (image data) and outputs the digital data to the processor 10 as appropriate. This image data is subjected to image processing by the processor 10 in the subsequent stage.

A processor 10 constituting a so-called computer is, for example, an ASIC (Application Specific Integrated Circuit), and executes image processing including geometric transformation processing such as distortion correction processing. Further, in performing image processing, the processor 10 executes encoding processing for converting image data into fixed-length codes and decoding processing for decoding fixed-length codes into image data.

The memory controller 50 performs a data storage operation when temporarily storing the fixed-length code generated by the processing of the processor 10 in the fixed-length memory 60, and an operation of reading the fixed-length code stored in the fixed-length memory 60. to control.

The fixed-length memory 60 is a non-volatile memory that stores/reads fixed-length codes used in image processing executed by the processor 10 under the control of the memory controller 50 . For the fixed-length memory 60, for example, a DDR-SDRAM (Double-Data-Rate Synchronous Dynamic Random Access Memory) can be used.

The moving image memory 70 is a non-volatile memory that stores moving images generated by image processing performed by the processor 10 . A flash memory, for example, can be used for the moving image memory 70 .

[Functional Blocks of Processor 10]
Next, the functional blocks of processor 10 will be described in detail. As shown in FIG. 2, the image processing block 100 configured by the processor 10 includes an image processing unit 110, a fixed length encoding unit 120, a distortion correction unit 130, a fixed length decoding unit 140, and a variable length moving image. and a compression unit 150 .

The image processing unit 110 performs image processing to convert image data output from a data input unit 200 composed of the optical system 20, the imaging device 30, and the A/D converter 40 into RGB image data. The image processing unit 110 generates RGB image data and outputs it to the fixed length encoding unit 120 . Details of the image processing unit 110 will be described later.

The fixed-length encoding unit 120 performs fixed-length encoding processing on the RGB image data generated by the image processing unit 110 . Therefore, the fixed-length encoding unit 120 converts the RGB image data into fixed-length codes. The fixed-length encoding unit 120 also performs fixed-length encoding processing on the corrected image data (RGB image data after correction) generated by the distortion correction unit 130 . The fixed-length encoding unit 120 transfers the fixed-length code converted from the corrected RGB image data to the temporary storage unit 300 . At this time, the distortion correction unit 130 instructs to write the corrected fixed-length code to the specified address. Details of the fixed-length encoding unit 120 will be described later. Fixed-length coding section 120 is an embodiment of the coding device according to the present invention.

The distortion correction unit 130 reads the image data decoded by the fixed length decoding unit 140 from the pre-correction fixed length code stored in the temporary storage unit 300, and executes distortion correction processing. The distortion correction section 130 transfers the corrected image data generated by the distortion correction processing to the fixed length encoding section 120 .

Fixed-length decoding section 140 reads the fixed-length code specified by distortion correction section 130 from temporary storage section 300 , decodes it, and transfers it to distortion correction section 130 . Also, the fixed-length decoding unit 140 sequentially reads and decodes the fixed-length codes after correction, and transfers them to the variable-length video compression unit 150 . Details of the fixed-length decoding unit 140 will be described later. Fixed-length decoding section 140 constitutes an encoding device.

The variable length video compression unit 150 performs, for example, MPEG (Moving Picture Experts Group) video compression processing on the corrected RGB image data transferred from the fixed length decoding unit 140 . The variable-length moving image compression unit 150 transfers the moving image data generated by the moving image compression process to the image data storage unit 400 configured by the moving image memory 70 . Details of the variable-length video compression unit 150 will be described later.

[Example of original image data and fixed length]
Here, the relationship between the image data to be processed in the image processing block 100 according to the present embodiment and the fixed-length code related to the image data will be described with reference to the schematic diagram of FIG. FIG. 3A schematically shows original image data 601 to be subjected to image processing. The original image 601 data is composed of a plurality of pixels. FIG. 3B schematically shows image blocks obtained by dividing the original image data 601 into image blocks each having a predetermined number of pixels. Fixed-length coding processing is executed for each image block.

Original image data 601 of an object photographed by the digital camera 1 is divided into image blocks of 4×4 pixels, and the image blocks shown in FIG. . Therefore, the fixed-length code according to this embodiment is based on a value generated for each image block composed of 4×4 pixels of the original image data 601 .

[Embodiment of encoding device]
Next, the functional blocks of the fixed-length coding unit 120, which corresponds to an embodiment of the coding device according to the present invention, will be described in more detail. As shown in FIG. 4, the fixed-length encoding unit 120 according to this embodiment includes a blocking unit 121, a YUV transform unit 122, a DCT transform unit 123, a 3MAX position number generation unit 124, and a quantization unit 125. , a quantization table 126 , a limiter section 127 , a limiter table 128 , and an encoding format generation section 129 .

The blocking unit 121 divides the RGB image data received from the image processing unit 110 into image blocks of 4×4 pixels. The image of the image block is as described with reference to FIG. The blocking unit 121 corresponds to image dividing means and constitutes an image dividing unit.

The YUV conversion unit 122 performs YUV conversion processing on each of the image blocks (image blocks consisting of 4 pixels×4 pixels) passed from the blocking unit 121 . Therefore, the YUV conversion unit 122 forms a YUV image block consisting of 4 pixels×4 pixels. This YUV image block is passed from the YUV conversion unit 122 to the DCT conversion unit 123 .

The DCT transform unit 123 performs frequency transform processing using DCT on the YUV image block to generate DCT coefficients for the YUV image block. The DCT transformation unit 123 passes the generated DCT coefficients to the 3MAX position number generation unit 124 and the quantization unit 125 . The DCT transforming section 123 corresponds to frequency transforming means and constitutes a frequency transforming section.

Here, examples of DCT coefficients generated in the DCT transform section 123 are shown schematically. Note that the DCT coefficient is a value composed of frequencies of two-dimensional horizontal and vertical cosine curves. The value differs depending on how much of the frequency indicated by the DCT coefficient is included in the spatial frequency of each pixel. FIG. 5 is an example in which certain YUV image blocks according to this embodiment are distinguished by DCT coefficient values. Let this be a DCT coefficient block 1000 . The DCT coefficient block 1000 expresses each pixel region of 4×4 pixels by distinguishing them by the values of luminance DCT coefficients.

In this embodiment, four levels are used according to the value of the DCT coefficient. These are referred to as a first level DCT coefficient 1001, a second level DCT coefficient 1002, a third level DCT coefficient 1003, and a fourth level DCT coefficient 1004.

A first level DCT coefficient 1001 is a DCT coefficient in a region corresponding to a direct current component (DC). In the example of FIG. 5, the DCT coefficients of the area corresponding to the first pixel are the first level DCT coefficients 1001 .

A second-level DCT coefficient 1002 is a DCT coefficient in a region containing a pixel with a low alternating current component (AC). In the example of FIG. 5, the DCT coefficients of the regions corresponding to the second and fifth pixels are second level DCT coefficients 1002 .

A third-level DCT coefficient 1003 is a DCT coefficient in a region in which the alternating current component (AC) contained in a pixel is moderate. In the example of FIG. 5, the third, fourth, sixth to tenth, thirteenth and fourteenth pixels are the third level DCT coefficients 1003 .

A DCT coefficient in a high-frequency region of AC components contained in a pixel is assumed to be a fourth level DCT coefficient 1004 . In the example of FIG. 5, the 11th, 12th, 15th and 16th pixels are the fourth level DCT coefficients 1004 . Note that the fourth level DCT coefficient 1004 is not included in the fixed length code.

As described above, the DCT coefficient block 1000 containing DCT coefficients distinguished for each pixel has a plurality of different patterns depending on the pattern (arrangement pattern) of the DCT coefficients at the level corresponding to each pixel constituting the image block. become a thing. An image block having such a specific arrangement pattern of DCT coefficients is passed from the DCT transformation section 123 to the 3MAX position number generation section 124 and the quantization section 125 .

Return to FIG. The 3MAX position number generator 124 selects three DCT coefficient blocks 1000 received from the DCT transform unit 123, which correspond to the top three absolute values of the third level DCT coefficients 1003 when arranged in descending order. The third level DCT coefficients 1003 are distinguished. This "up to the top three" corresponds to the predetermined condition. The 3MAX position number generator 124 does not process any DCT coefficients other than the third level DCT coefficients 1003 whose absolute values are not less than the predetermined upper limit and less than the predetermined lower limit. That is, the 3MAX position number generation unit 124 excludes processing targets under a predetermined condition, and performs processing while excluding absolute values that do not correspond to the third level DCT coefficients 1003 . The differentiated third level DCT coefficients 1003 are referred to as fifth level DCT coefficients 1005 . A DCT coefficient block 1000 including this fifth level DCT coefficient 1005 is called a 3MAX DCT coefficient block 1100 .

For example, if the top three of the third level DCT coefficients 1003 included in the DCT coefficient block 1000 shown in FIG. 5 are DCT coefficients in regions corresponding to the third, fourth and seventh pixels, We obtain a 3MAX DCT coefficient block 1100 such as Note that the 3MAX DCT coefficient block 1100 is classified into 84 patterns.

FIG. 7 illustrates the full pattern of 3MAX DCT coefficient block 1100 . Note that FIG. 7 is also an example of limiter values for each classification of the 3MAX DCT coefficient block 1100 . Limiter values will be described later.

The 3MAX DCT coefficient block 1100 corresponds to a 3MAX position number of "1" to "84" depending on the arrangement pattern of the fifth level DCT coefficients 1005 among the third level DCT coefficients 1003. FIG. The 3MAX position number generator 124 identifies which of the 84 patterns included in FIG. 7 the arrangement pattern of the fifth level DCT coefficients 1005 included in the 3MAX DCT coefficient block 1100 corresponds to. Then, the specified number is transferred to the limiter section 127 and the encoding format generation section 129 as the "3MAX position number". The 3MAX position number generation unit 124 corresponds to condition-suitable frequency transform coefficient specifying means and constitutes a condition-suitable frequency transform coefficient specifying unit.

Return to FIG. The quantization unit 125 receives the DCT coefficients from the DCT transform unit 123, reads the quantization values from the quantization table 126, and quantizes the 4×4 DCT coefficients. Quantization section 125 transfers the quantized DCT coefficients to limiter section 127 .

The quantization table 126 stores quantization values. The quantization values stored in the quantization table 126 are read by the quantization section 125 .

Based on the 3MAX position number received from the 3MAX position number generator 124, the limiter unit 127 refers to the limiter table 128 and reads out the limiter value corresponding to the 3MAX position number. The limiter values read here are a limit value group corresponding to a 4×4 coefficient group generated corresponding to 4 pixels×4 pixels. Also, the limiter section 127 performs limiter processing using a limiter value on the quantized DCT coefficients received from the quantization section 125 . The limiter unit 127 transfers the quantized DCT coefficients subjected to limiter processing to the encoding format generation unit 129 . The limiter unit 127 corresponds to limit value selection means for selecting a limit value group, and constitutes a limit value selection unit. Further, the limiter section 127 corresponds to restriction processing means and constitutes a restriction processing section.

The limiter table 128 stores 3MAX position numbers and limiter values linked thereto. Examples of limiter table data 2000 stored in the limiter table 128 are illustrated in FIGS. 7 and 8. FIG. The limiter value table uses the 3MAX position number as an index and stores the corresponding limiter value. The limiter values correspond to a 4×4 3MAX DCT coefficient block 1100 . The limiter values are stored in the limiter table 128 as discriminated values corresponding to the above-described discriminated DCT coefficients (first to fifth).

Here, limiter values included in limiter table data 2000 will be described with reference to FIG. In FIG. 7, the coefficients included in the 3MAXDCT coefficient block 1100 are not labeled, but the same patterns (horizontal lines, grids, oblique lines, solid lines) shown in FIG. It is written. Therefore, the horizontal line pattern coefficients in FIG. 7 correspond to the first level DCT coefficients 1001 . The hatched pattern coefficients correspond to the second level DCT coefficients 1002 . The grid pattern coefficients correspond to the third level DCT coefficients 1003 . The filled coefficients correspond to the fifth level DCT coefficients 1005 .

The limiter values are individually set based on each DCT coefficient contained in the 3MAX DCT coefficient block 1100 as follows. Since the first level DCT coefficient 1001 is a direct current component (DC), it is set to have the highest limiter level and not to shorten the bit length. Since the second level DCT coefficient 1002 has the lowest waveform in the alternating current component (AC), the limiter level is set to the second highest and the bit length is set so as not to be too short. Since the third level DCT coefficient 1003 has a waveform with medium AC components, the scalar amount of the DCT coefficient is not large. Therefore, the third level DCT coefficient 1003 is set to have the highest limiter level and the shortest bit length. The fifth level DCT coefficient 1005 has a waveform with a medium AC component, and the scalar amount of the DCT coefficient is not large. Therefore, the fifth level DCT coefficient 1005 is set to have a moderate limiter level and a slightly shorter bit length. It should be noted that the fourth level DCT coefficient 1004 is not included in the fixed length code since it is a high frequency. Therefore, no limiter value is set.

The encoding format generation unit 129 performs format conversion processing for converting the DCT coefficients after the limiter processing, which are passed from the limiter unit 127, into a format as illustrated in FIG. The coding format generation unit 129 stores the fixed length code generated by the format conversion processing in the fixed length memory 60 forming the temporary storage unit 300 . The encoding format generator 129 corresponds to encoding means.

FIG. 9 illustrates a fixed-length code format (fixed-length coding format) according to this embodiment. As shown in FIG. 9, bits (7 bits) indicating the 3MAX position number, bits (97 bits) corresponding to the DCT coefficient of luminance, bits (20 bits) corresponding to the DCT coefficient of the U component, and the DCT coefficient of the V component. is composed of bits (20 bits) corresponding to . The fixed-length code according to this embodiment consists of 144 bits.

FIG. 10 shows an example in which the encoding format generator 129 converts the DCT coefficients passed from the limiter 127 into fixed-length codes based on the format shown in FIG. FIGS. 10(a), 10(b), and 10(c) respectively illustrate fixed-length codes based on coefficient blocks after limiter processing for 3MAX DCT coefficient blocks 1100 corresponding to different 3MAX position numbers. The coding format generator 129 rearranges the DCT coefficients of each block as shown in FIG. 10, determines the coding format based on the 3MAX position numbers, and generates the fixed length format shown in FIG. It should be noted that the numbers in the 3MAX DCT coefficient block 1100 shown in FIG. 10 are numbers written for convenience in order to illustrate the correspondence with the positions of the bit strings in the fixed-length code.

FIG. 10(a) exemplifies the 3MAX DCT coefficient block 1100 shown in FIG. The effective bit length differs depending on the DCT coefficient level. For example, the effective bit length is 11 bits for the first level DCT coefficient 1001, 10 bits for the second level DCT coefficient 1002, 8 bits for the fifth level DCT coefficient 1005, and 7 bits for the third level DCT coefficient 1003.

The encoding format of FIG. 10(a) is the same as that shown in FIG. 9, and each DCT coefficient block is stored for each level. As described above, the encoding format generation unit 129 performs encoding processing on DCT coefficients of pixel blocks subjected to limiter processing corresponding to 3MAX position numbers.

[Embodiment of decoding device]
Next, detailed functional blocks of the fixed-length decoding unit 140 corresponding to the embodiment of the decoding device according to the present invention will be explained using FIG. As shown in FIG. 11, the fixed-length decoding unit 140 according to this embodiment includes a code analysis unit 141, a DCT coefficient sorting unit 142, an inverse quantization unit 143, an inverse quantization table 144, and an inverse DCT transform. It has a section 145 , an RGB conversion section 146 , and a deblocking section 147 .

The code analysis unit 141 reads fixed-length codes having the format shown in FIG. , to the DCT coefficient sorting unit 142 . The code analysis unit 141 corresponds to code analysis means.

The DCT coefficient sorting unit 142 converts the luminance DCT coefficients from the 3MAX position numbers, and transfers the luminance DCT coefficients to the inverse quantization unit 143 . The DCT coefficient sorting unit 142 generates frequency transform coefficients having an array corresponding to the image block based on the arrangement pattern of the third-level DCT coefficients 1003 specified by the code analysis unit 141 and each analyzed frequency transform coefficient. do. The DCT coefficient sorting unit 142 corresponds to frequency transform coefficient generating means and constitutes a frequency transform coefficient generating unit.

Processing in the DCT coefficient sorting section 142 will be described with reference to FIG. 12 illustrates how 3MAX position numbers are extracted from fixed-length format bit strings and converted from fixed-length codes to 3MAX DCT coefficient blocks 1100, contrary to the diagram shown in FIG. DCT coefficient sorting section 142 performs conversion from the fixed length format to 3MAX DCT coefficient block 1100 as shown in FIG.

The inverse quantization unit 143 uses inverse quantization values read from the inverse quantization table 144 to inversely quantize the luminance DCT coefficients, the U component DCT coefficients, and the V component DCT coefficients included in the 3MAX DCT coefficient block 1100 . Perform quantization processing. The inverse quantization unit 143 transfers the result of the inverse quantization process to the inverse DCT transform unit 145 . The inverse quantization section 143 corresponds to inverse quantization means.

The inverse quantization table 144 stores quantization values for quantizing the DCT coefficients.

The inverse DCT transform unit 145 executes inverse DCT transform processing based on “the DCT coefficients of the luminance, U component, and V component obtained as a result of the inverse quantization processing” passed from the inverse quantization unit 143 . YUV image data is thus obtained, and this YUV image data is transferred to the RGB conversion unit 146 . The inverse DCT transform unit 145 corresponds to inverse frequency transform processing means and constitutes an inverse frequency transform processing unit.

The RGB conversion unit 146 executes conversion processing of the transferred YUV image data into RGB image data. The RGB conversion unit 146 outputs RGB image data consisting of a pixel block of 4 pixels×4 pixels.

The deblocking unit 147 develops the pixel block passed from the RGB conversion unit 146 into two-dimensional image data to generate RGB image data. The deblocking unit 147 transfers the generated RGB image data to the temporary storage unit 300 . The deblocking unit 147 corresponds to image block integration means and constitutes an image block integration unit.

[Detailed Functional Blocks of Image Processing Unit 110]
Here, detailed functional blocks of the image processing unit 110 according to this embodiment will be described with reference to FIG. 13 . As shown in FIG. 13 , the image processor 110 has a Byaer correction processor 111 and a filter processor 112 .

The Byaer correction processing unit 111 receives image data from the data input unit 200, reads and corrects the Bayer type image, and generates normal RGB values.

The filtering unit 112 performs filtering on the RGB image data generated by the Byaer correction processing unit 111 . The RGB image data after filtering is transferred to the fixed length encoding unit 120 .

[Detailed functional blocks of variable-length video compression unit 150]
Next, detailed functional blocks of the variable-length video compression unit 150 according to this embodiment will be described with reference to FIG. 14 . As shown in FIG. 14, the variable-length video compression unit 150 includes a motion search unit 151, a DCT unit 152, a quantization processing unit 153, a variable-length coding unit 154, an inverse quantization processing unit 155, and an inverse It has a DCT section 156 and a motion compensation section 157 .

The motion search unit 151 calculates difference data between the image data received from the fixed length decoding unit 140 and the block of the previous frame with the highest correlation from the motion compensation unit 157 .

The DCT unit 152 converts the difference data calculated by the motion search unit 151 into spatial frequency, and transfers the spatial frequency to the quantization processing unit 153 .

The quantization processing unit 153 quantizes the frequency-transformed DCT data and transfers the quantized data to the variable-length coding unit 154 .

The variable-length coding unit 154 variable-length-codes the quantized data generated by the quantization processing unit 153 to generate compression codes. The variable-length coding unit 154 stores the generated compression code in the image data storage unit 400. FIG.

The inverse quantization processing unit 155 performs inverse quantization processing on the “quantized block” generated by the quantization processing unit 153 . The inverse quantization processing section 155 transfers the processing result to the inverse DCT section 156 .

The inverse DCT unit 156 performs inverse frequency transform processing on the inverse quantized data passed from the inverse quantization processing unit 155 . The inverse DCT unit 156 transfers the processing result of the inverse frequency transform processing to the motion compensation unit 157 .

The motion compensation unit 157 adds the difference data passed from the inverse DCT unit 156 to the motion-compensated macroblocks of the previous frame to create a reference frame. The motion compensator 157 transfers the reference frame to the motion searcher 151 .

[Embodiment of Encoding Method and Decoding Method]
Next, embodiments of the encoding method and decoding method according to the present invention will be described. FIG. 15 illustrates the flow of operations of the digital camera 1 described above. The encoding method and decoding method according to this embodiment are performed in a series of image processing executed in an image processing apparatus such as the digital camera 1 .

First, reflected light from an object is focused on the imaging surface of the imaging element 30 via the optical system 20, and an image signal output from the imaging element 30 is captured (S1501). Subsequently, the image signal is converted into image data by the A/D converter 40 and input to the processor 10 (S1502). Subsequently, image data processing is executed in the processor 10 (S1503). Details of the image data processing will be described later. Moving image data is generated and output by image data processing in the processor 10, and stored in the moving image memory 70 (S1504).

[Details of image data processing]
The encoding method and decoding method according to the present invention are applied to the image data processing in S1503. Therefore, the image processing program for executing the image data processing includes the encoding program and the decoding program according to the present invention. The details of the image data processing (S1503) will be described below, and the encoding method, decoding method, and execution of the encoding program and decoding program will also be referred to.

FIG. 16 is a flowchart showing the detailed flow of image data processing in S1503. First, RGB image data is coded into a fixed length code in the fixed length coding unit 120 (S1601). The generated fixed-length code is temporarily stored in the temporary storage unit 300 (S1602). Subsequently, the fixed-length decoding unit 140 reads out the temporarily stored fixed-length code based on an instruction from the distortion correction unit 130 (S1603).

Subsequently, fixed-length decoding section 140 decodes the fixed-length code (S1604). Using the YUV image data decoded from the fixed-length code, the distortion correction unit 130 executes distortion correction processing (S1605).

Fixed-length encoding processing is performed again on the YUV image data that has been subjected to distortion correction processing (S1606). The generated fixed-length code is temporarily stored in the temporary storage unit 300 (S1607). Subsequently, the fixed-length decoding unit 140 reads out the temporarily stored fixed-length code based on an instruction from the variable-length video compression unit 150 (S1608).

Subsequently, fixed-length decoding section 140 decodes the fixed-length code (S1609). Using the YUV image data decoded from the fixed-length code, the variable-length moving image compression unit 150 generates moving image data (S1610). Subsequently, the generated moving image data is stored in the image data storage unit 400 (S1611).

Next, a detailed processing flow in fixed-length encoding processing (S1601, S1606) will be described using the flowchart of FIG.

First, the RGB image data generated by the image processing unit 110 is read from the memory, the blocking unit 121 generates image blocks of 4 pixels×4 pixels, and sequentially outputs them to the YUV conversion unit 122 (S1701). .

Subsequently, the YUV conversion unit 122 executes YUV conversion processing for converting from the RGB format to the YUV format for each image block, and transfers the generated YUV image data to the DCT conversion unit 123 (S1702).

Subsequently, the DCT transform unit 123 performs frequency transform processing using discrete cosine transform (DCT) on the YUV image block of 4 pixels×4 pixels converted into the YUV format in S1702 (S1703). The DCT transforming section 123 transfers the DCT coefficients generated by the frequency transforming process to the 3MAX position number generating section 124 and the quantizing section 125 .

Subsequently, DCT formation restriction processing is performed on the generated frequency transform coefficient (DCT) coefficients by the 3MAX position number generator 124, quantizer 125, and limiter 127 (S1704). Details of the DCT restriction processing will be described later.

Finally, the encoding format generation unit 129 executes fixed-length code generation processing (S1705). The details of the fixed-length code generation processing are based on the operation of the coding format generation unit 129 described with reference to FIGS. 9 and 10. FIG.

Next, details of the DCT coefficient limiting process (S1704) will be described with reference to the flowchart of FIG. First, it is assumed that the DCT coefficient block 1000 illustrated in FIG. 5 is generated in the DCT transform processing (S1703). On the other hand, based on the value of the DCT coefficient of each block, it is determined which level of DCT coefficient each block corresponds to (S1801).

Subsequently, the 3MAX position number generator 124 arranges the DCT coefficients corresponding to the third level DCT coefficients 1003 in descending order of absolute value, and identifies the third level DCT coefficients 1003 corresponding to the top three (S1802). ).

Next, the third level DCT coefficients 1003 corresponding to the top three are set as the fifth level DCT coefficients 1005 (see FIG. 6). Then, it is determined which pattern shown in FIG. 7 the arrangement pattern of the fifth level DCT coefficients 1005 matches, and the "3MAX position number" of the matched pattern is specified (S1803). The specified 3MAX position number is transferred to the limiter section 127 and the encoding format generation section 129 .

Also, in the DCT transform processing (S1703), the quantization unit 125 performs quantization processing on the DCT coefficients included in the generated DCT coefficient block 1000 (S1804). In S1804, the quantization section 125 refers to the quantization table 126 to specify a quantization value, and quantizes the DCT coefficients included in the DCT coefficient block 1000. FIG. The quantized DCT coefficients are transferred to limiter section 127 .

Subsequently, limiter processing for limiting the DCT coefficients is executed in the limiter section 127 (S1805). Here, the effect of limiter processing (S1805) will be described in detail. FIG. 19 is an image diagram for explaining the effect of the limit processing (S1805) executed in the limiter section 127. FIG. In FIG. 19, numbers written at positions corresponding to respective pixels are examples of DCT coefficients.

FIGS. 19(a) to 19(d) show comparative examples assuming a case where the conventional technique is used to limit the DCT coefficients. FIGS. 19(e) to 19(h) are examples of restriction processing in the encoding processing according to the present invention.

FIGS. 19(a) and 19(e) respectively illustrate DCT coefficients before quantization. As shown in FIG. 19(e), part of the third level DCT coefficients 1003 are used as fifth level DCT coefficients 1005. FIG.

FIG. 19(b) and FIG. 19(f) show the quantization processing executed on these. FIG. 19(c) and FIG. 19(g) are the result of performing limiter processing on these. Finally, FIG. 19(d) and FIG. 19(h) show the inverse quantization process performed on these. Referring to the values of the DCT coefficients before quantization and the values of the DCT coefficients after quantization shown in each figure, in the comparative example, the value of the third level DCT coefficient 1003 changes significantly, whereas in this example The variation is small in the invention. That is, with the method according to the present invention, even when compressed into fixed-length codes, the reproducibility of edges and colors of an image is high. In other words, by using fixed-length codes for data to be processed when performing image processing, it is possible to reduce the amount of access to the memory, improve the processing efficiency, and maintain the quality of the processed image at a high level.

In the quantization section 125 according to this embodiment, the third level DCT coefficient 1003 is shifted by 3 bits, the first level DCT coefficient 1001 is not shifted, and the second level DCT coefficient 1002 is shifted by 1 bit. , the fifth level DCT coefficient 1005 is shifted by 2 bits, and the value of the fourth level DCT coefficient 1004 is set to zero. All bit shift processing in the quantization unit 125 is right shift. In the inverse quantization processing executed in the inverse quantization section 143, left shift processing is similarly performed on the DCT coefficients of each level.

Further, the limiter unit 127 according to the present embodiment limits the value of the DCT coefficient to "63" when the value of the fifth level DCT coefficient 1005 is "63" or more. Also, the second level DCT coefficient 1002 and the first level DCT coefficient 1001 are not restricted regardless of the values of the DCT coefficients. As for the fifth level DCT coefficient 1005, the value is limited to "127" when the value of the DCT coefficient is "127" or more.

As described above, in the encoding method according to the present embodiment, particularly in the DCT coefficient limiting process (S1704), among the third-level DCT coefficients 1003 before quantization, the scalar amount is the highest three. A large limiter value is set for the five-level DCT coefficient 1005 . As a result, even after the inverse quantization process, the difference between the values before quantization is small, and the characteristics of the DCT coefficients, which are particularly large in scalar, are maintained. On the other hand, in the comparative example, the portion of the third level DCT coefficient 1003 is flattened and the features are lost.

Next, the detailed processing flow in the fixed-length decoding processing (S1604, S1609) will be described using the flowchart of FIG.

First, the code analysis unit 141 reads out the corresponding fixed-length code from the temporary storage unit 300, analyzes it, distinguishes the 3MAX position number and the bit string of each DCT coefficient, and transfers it to the DCT coefficient sorting unit 142 (S2001). .

Subsequently, the DCT coefficient sorting unit 142 converts luminance into DCT coefficients as illustrated in FIG. 12 based on the 3MAX position number, and transfers the DCT coefficients to the inverse quantization unit 143 (S2002).

Subsequently, the inverse quantization unit 143 refers to the inverse quantization table 144 and performs the inverse quantization on the DCT coefficients of the quantized luminance, U component, and Y component transferred from the DCT coefficient sorting unit 142 . Quantization processing is executed, and the result is transferred to the inverse DCT transform unit 145 (S2003).

Subsequently, the inverse DCT transform unit 145 performs inverse DCT transform based on the transferred DCT coefficients to generate YUV image data. The generated YUV image data is transferred to the RGB conversion unit 146 (S2004).

Subsequently, the RGB conversion unit 146 converts the transferred YUV image data into RGB image data (S2005). Finally, RGB image data (image block of 4×4 pixels) consisting of image blocks is developed into two-dimensional data to generate RGB image data and transferred to the temporary storage unit 300 (S2006).

As described above, based on the scalar amount of the DCT coefficients, the process of limiting the DCT coefficients corresponding to a predetermined image block is controlled, converted into fixed-length code, and decoded as described above to obtain an image. It is possible to reduce the amount of data with a high compression rate while suppressing the deterioration of the quality of the data.

The present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the technical scope of the invention. All matters are covered by the present invention. Although the above embodiment shows a preferred example, a person skilled in the art can realize various modifications from the disclosed contents. Such modifications are also included in the technical scope described in the claims.

1: Digital camera 10: Processor 20: Optical system 30: Image sensor 40: A/D converter 50: Memory controller 60: Fixed length memory 70: Moving image memory 100: Image processing block 110: Image processing unit 111: Byaer correction Processing unit 112 : Filter processing unit 120 : Fixed length encoding unit 121 : Blocking unit 122 : YUV conversion unit 123 : DCT conversion unit 124 : 3MAX position number generation unit 125 : Quantization unit 126 : Quantization table 127 : Limiter unit 128: Limiter table 129: Encoding format generator 130: Distortion corrector 140: Fixed length decoder 141: Code analyzer 142: DCT coefficient sorter 143: Inverse quantizer 144: Inverse quantizer table 145: Inverse DCT Transformation unit 146 : RGB transformation unit 147 : Deblocking unit 150 : Variable length video compression unit 151 : Motion search unit 152 : DCT unit 153 : Quantization processing unit 154 : Variable length coding unit 155 : Inverse quantization processing unit 156 : Inverse DCT unit 157 : Motion compensation unit 200 : Data input unit 300 : Temporary storage unit 400 : Image data storage unit 601 : Original image data

Japanese Patent Application Laid-Open No. 2004-112345

Claims (7)

An encoding device that converts an image block obtained by dividing an image including a plurality of pixels into a predetermined number of pixels into a fixed-length code,
image segmentation means for generating the image blocks;
frequency transform means for performing frequency transform processing on the image block and outputting frequency transform coefficients corresponding to the image block;
a condition-suitable frequency transform coefficient specifying means for specifying, as a condition-suitable frequency transform coefficient, a frequency transform coefficient whose absolute value of the frequency transform coefficient corresponding to each pixel included in the image block meets a predetermined condition;
Limit value selection means for selecting a limit value group including limit values for individually limiting values of the frequency transform coefficients corresponding to the pixels based on the condition-satisfactory frequency transform coefficients;
limiting processing means for executing limiting processing for limiting the values of the frequency conversion coefficients output by the frequency conversion means, using each limit value included in the selected limit value group;
encoding means for encoding the frequency transform coefficient after the limiting process has been performed as a fixed-length code;
The limit processing means uses a value smaller than a limit value included in the selected limit value group in the limit processing for the condition-suitable frequency transform coefficients included in the frequency transform coefficients. An encoding device, characterized in that it limits the values of transform coefficients. The condition-satisfactory frequency transform coefficient identifying means excludes absolute values of the frequency transform coefficients that are equal to or greater than a predetermined upper limit and less than a predetermined lower limit from the objects to be rearranged, and treats the frequency transform coefficients that are subject to condition-satisfactory frequency transform coefficients.
2. Encoding apparatus according to claim 1. The condition-suitable frequency transform coefficient specifying means specifies, as the predetermined condition, three frequency transform coefficients in descending order of absolute values as condition-suitable frequency transform coefficients.
3. Encoding device according to claim 1 or 2. The encoding means determines a fixed-length encoding format of the frequency transform coefficients subjected to the limiting process based on the pattern of the condition-satisfactory frequency transform coefficients included in the frequency transform coefficients, and performs the fixed-length encoding. fixed-length code the frequency transform coefficients based on a format;
4. Encoding apparatus according to any one of claims 1 to 3. the pattern of the conditional frequency transform coefficients used by the encoding to determine a fixed-length encoding format is the arrangement of the pixels corresponding to the conditional frequency transform coefficients;
5. Encoding device according to claim 4. an image block obtained by dividing an image containing a plurality of pixels into a predetermined number of pixels;
performing frequency transform processing on the image block;
outputting frequency transform coefficients corresponding to the image block;
identifying a frequency transform coefficient whose absolute value corresponding to each pixel included in the image block satisfies a predetermined condition as a condition-satisfactory frequency transform coefficient;
selecting a limit value group including limit values for individually limiting values of the frequency transform coefficients corresponding to the pixels based on the conditional frequency transform coefficients;
In executing the limiting process for limiting the values of the frequency transform coefficients using each limit value included in the selected limit value group, in the limiting process for the condition-satisfactory frequency transform coefficients included in the frequency transform coefficients, , limiting the value of the conditional frequency transform coefficient using a value smaller than the limit value included in the selected limit value group;
An encoding method, characterized in that the frequency transform coefficient after the limiting process is performed is encoded as a fixed-length code. An encoding program for causing a computer to execute a process of converting an image block obtained by dividing an image including a plurality of pixels into a predetermined number of pixels into a fixed-length code,
an image dividing unit that generates the image blocks;
a frequency transform unit that performs frequency transform processing on the image block and outputs a frequency transform coefficient corresponding to the image block;
a condition-suitable frequency transform coefficient specifying unit that specifies, as a condition-suitable frequency transform coefficient, a frequency transform coefficient whose absolute value of the frequency transform coefficient corresponding to each pixel included in the image block meets a predetermined condition;
a limit value selection unit that selects a limit value group including limit values for individually limiting values of the frequency transform coefficients corresponding to the pixels, based on the condition-satisfactory frequency transform coefficients;
a limit processing unit that uses each limit value included in the selected limit value group to execute a limit process for limiting the value of the frequency transform coefficient output by the frequency transform unit;
an encoding unit that encodes the frequency transform coefficient after the limiting process has been performed as a fixed-length code,
In the limiting process for the condition-satisfactory frequency transform coefficients included in the frequency transform coefficients, the limit processing unit uses a value smaller than a limit value included in the selected limit value group to limit the condition-satisfactory frequencies. An encoding program characterized by causing a process to be performed that limits the values of transform coefficients.

Images