JP2009004878A - Image processor, image processing method and image processing program, and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To change quantization values in units of image processing, and to facilitate changing processing for quantization values. <P>SOLUTION: An image processing unit 101 performs image enlargement/reduction processing etc., to a digital image signal. Image data after the image processing are JPEG-encoded by a DCT unit 102, a quantization unit 103, and a variable-length encoding/decoding unit 104. An arrangement of data of a code 105 is prescribed through processing of the image processing unit 101. A quantization table adjustment unit 108 which performs code rearrangement performs the rearrangement and outputs a code 109. The code 109 is decoded in a regular processing direction prescribed by an encoding format, for example, JPEG, so as to obtain a desired image. For adjustment of quantization values, conversion between variable-length codes differing in quantization value is performed by replacement of codes using a variable-length code replacement table. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, an image processing program, and an imaging apparatus that can be applied to compression of an image signal.

  Image processing such as processing for correcting image distortion, processing for correcting chromatic aberration, and processing for enlarging or reducing the size of an image is often performed in an imaging apparatus such as a digital still camera. In addition, image data generated by image processing is usually compressed by compression encoding such as JPEG (Joint Photographic Experts Group) or MPEG (Moving Picture Experts Group Phase). In such compression encoding, encoding is performed in units of images called macroblocks.

  In this way, when the unit of image processing is an integer number of macroblocks, that is, encoded blocks, and the arrangement of the encoded blocks is different from the order defined by the encoding, the data to be encoded later The minutes need to be stored in memory. As a result, there is a problem that a memory for temporary storage is required.

  As described above, when image processing such as image size conversion is performed, when image data is cut out and encoded in a coding unit of compression coding, it is necessary to read from memory at high speed or A problem arises that requires a large amount of memory.

  An encoding apparatus capable of solving such a problem that requires high-speed reading processing and a large-capacity memory has been proposed in Patent Document 1 below.

JP 2006-238407 A

  FIG. 18 shows a flow of processing of the encoding device described in Patent Document 1. The image processing unit 101 performs image rotation processing, image enlargement / reduction processing by resolution conversion called resizing, trimming processing, and the like on a digital image signal formed based on an imaging signal from an imaging device. . The imaging results recorded in the storage medium are sequentially cut out and executed in accordance with the image processing unit and the image processing order with a block size suitable for image processing.

  Image data from the image processing unit 101 is converted from a spatial domain to a frequency domain in a DCT (Discrete Cosine Transform) unit 102 in units of (8 × 8) pixel blocks. The coefficient data from the DCT unit 102 is quantized in the quantization unit 103. The output of the quantization unit 103 is supplied to the variable length coding unit 104, and the amount of information is compressed by entropy coding. The DCT unit 102, the quantization unit 103, and the variable length encoding unit 104 constitute a JPEG encoding apparatus.

  The arrangement of the code 105 data obtained at the output of the variable length coding unit 104 is defined by the processing of the image processing unit 101. On the other hand, in the JPEG encoding process, the format stipulates that encoding processing units of macroblocks are sequentially encoded in raster scanning order. Therefore, when the code 105 having the order of the image processing is decoded, an image different from the desired image is obtained.

  In order to solve this problem, rearrangement is performed by the code rearrangement unit 106 without using a large-capacity buffer memory, and the rearranged code 107 is output. Reference numeral 107 is recorded in, for example, a storage medium. When the code 107 is decoded in a normal processing direction defined by an encoding format such as JPEG, a desired image can be obtained.

  In the encoding method described above, when the quantization method is a predetermined one, there is a problem that the code amount of the code 107 cannot be controlled during the encoding. For example, in the case of JPEG, it is defined that the quantization table cannot be changed. Further, although the code amount changes due to code rearrangement, the quantization table is not defined in consideration of the change. Further, when the code 107 is decoded and encoded again in order to control the code amount, there are problems that the processing time becomes long and the memory amount necessary for the process increases.

  Therefore, an object of the present invention is to finely adjust the code amount using a plurality of quantized values, even when a quantized value including a quantization table and a quantizing scale is defined as one, and decoding and decoding To provide an image processing device, an image processing method, an image processing program, and an imaging device capable of adjusting the code amount without re-encoding and further adjusting the code amount simultaneously with the rearrangement process. It is in.

In order to solve the above-described problems, the present invention provides an image processing method in which image data processed by an image processing unit is encoded in a predetermined encoding unit by an encoding unit.
An encoding step of performing conversion in the order of data output from the image processing unit, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient;
A rearrangement / quantization value change step for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to a normal processing direction defined by encoding. An image processing method.
The present invention is also an image processing method for encoding image data in a predetermined encoding unit by an encoding unit,
In an image processing method having an encoding step for performing conversion, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient, and a quantization value changing step,
The quantization value changing step is an image processing method for changing a quantization value by directly replacing at least one code from a variable length code and a quantization value to a variable length code corresponding to a new quantization value. .

The present invention relates to a program for causing a computer to execute an image processing method for encoding image data processed by an image processing unit in a predetermined encoding unit in an encoding unit.
An encoding step of performing conversion in the order of data output from the image processing unit, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient;
A rearrangement / quantization value change step for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to a normal processing direction defined by encoding. A program for an image processing method.

The present invention relates to an image processing apparatus for encoding image data processed by an image processing unit in a predetermined encoding unit by an encoding unit.
An encoding unit that performs conversion in the order of data output from the image processing unit, quantizes coefficient data generated by the conversion, and variable-length-encodes the quantized coefficient;
A reordering / quantization value changing unit that adjusts the quantization value for each processing unit area of the image processing unit and performs reordering from the processing direction of the image processing unit to a normal processing direction defined by encoding. An image processing apparatus.

The present invention includes an image processing unit that performs image processing on image data captured by an image sensor, and an encoding unit that encodes image data processed by the image processing unit in a predetermined encoding unit by an encoding unit. In an imaging apparatus having
An encoding unit that performs conversion in the order of data output from the image processing unit, quantizes coefficient data generated by the conversion, and variable-length-encodes the quantized coefficient;
A reordering / quantization value changing unit that adjusts the quantization value for each processing unit area of the image processing unit and performs reordering from the processing direction of the image processing unit to a normal processing direction defined by encoding. It is an imaging device having.

  In the present invention, for example, by adjusting the quantization value in units of image processing, it is possible to control the code amount in fine units. Since the quantization value is changed after encoding such as JPEG, even if the encoding method is specified to use a single quantization value, the code amount is finely adjusted without violating the specification. Can do. In addition, for code amount control, decoding and re-encoding processing can be avoided by directly replacing a variable-length code and a quantized value with a variable-length code corresponding to a new quantized value. Therefore, it is possible to avoid an increase in processing time and an increase in the amount of memory used. Further, when the code amount is adjusted simultaneously with the rearrangement, the accuracy of the adjustment of the code amount can be increased by effectively using the information obtained for the rearrangement.

  An embodiment of the present invention will be described. FIG. 1 shows a configuration of an embodiment of an imaging apparatus according to the present invention, for example, a digital still camera. In FIG. 1, reference numeral 1 indicates a lens device, and reference numeral 2 indicates an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). For example, a three-primary color filter having a Bayer array is provided for the image sensor 2, and a three-primary color signal is obtained as an output of the image sensor 2. The three primary color signals are converted into digital signals by the A / D converter 3.

  A digital image pickup signal from the A / D converter 3 is input to a digital processing camera signal processing unit 4 having an LSI (Large Scale Integrated Circuit) configuration. The camera signal processing unit 4 performs white balance correction processing, interpolation processing, filtering processing, matrix calculation processing, gamma correction processing, luminance signal (Y) generation processing, color difference signal (Cr, Cb) generation processing, and the like. The image signal generated by the camera signal processing unit 4 is supplied to the display 5 and a captured image is displayed. Further, the image data from the camera signal processing unit 4 is subjected to size conversion, compression processing, etc., and stored in the internal or external storage 9.

  In the white balance correction process, an unbalance between colors due to a difference in color temperature environment of the subject and a difference in sensitivity due to the color filter is corrected. The interpolation process is a process for interpolating color signals that do not exist. The filtering process is a high-frequency correction process and is a process for contour enhancement. The matrix calculation process is a process for converting the imaging signal into sRGB. By performing reverse correction of the non-linear characteristic of the display device in advance by the gamma correction process, a linear characteristic is finally realized.

  In the luminance signal generation process, a luminance signal is generated by synthesizing the gamma-corrected RGB signal at a predetermined synthesis ratio. The color difference signal generation processing block generates a color difference signal by synthesizing the gamma-corrected RGB signal at a predetermined synthesis ratio. The generated color difference signal is subjected to band limitation processing, and color difference signals Cb and Cr are generated.

  The output (Y, Cr, Cb) of the camera signal processing unit 4 is input to an image processing unit 6 that performs image processing such as a size change, and for example, the image size is enlarged, enlarged, or reduced. The output data of the image processing unit 6 is input to the JPEG encoding / decoding unit 7, for example. The encoder / decoder 7 includes a DCT, a quantization inverse quantizer, and a variable length encoder / decoder. Note that the image processing unit 6 is not limited to performing image processing directly on the imaging signal, and the image data recorded in the storage 9 is decoded with respect to the image data decoded by the encoding / decoding unit 7. In some cases, image processing may be performed.

  In DCT, a DCT operation is performed for each block of (8 × 8) pixels, and a DCT coefficient component including a DC (Direct Current) coefficient component and an AC (Alternate Current) coefficient component is obtained. The DC coefficient component and the AC coefficient component are quantized by the quantization table. The quantized DC coefficient component is encoded with a difference value from the DC coefficient component of the immediately preceding block. However, immediately after a delimiter code that is a code of a predetermined value indicating a delimiter is inserted, the value of the DC coefficient component itself is encoded. Further, since the length of the delimiter code does not change even if the quantization table is changed, the change in the code amount other than the delimiter code becomes the entire code amount change. In the presence or absence of a delimiter code, it is normal that the DC coefficient value before quantization is smaller when there is no delimiter code. For this reason, the amount of code is often small even when compared excluding the delimiter code. . In the case of the quantized AC coefficient component, it is subjected to variable length coding after being rearranged by zigzag scanning within the block. Even when the quantum table is changed, the DC coefficient component and the AC coefficient component of the same DCT block are quantized with the same quantization table.

  Note that the DC coefficient component and the AC coefficient component of the same DCT block may be quantized using different quantization tables. However, if there is a certain number of DCT blocks in the image, it is changed within the DCT. The code amount can be sufficiently controlled even if it is not necessary, and the management becomes complicated, so the same quantization table is used here.

  The compressed image data compressed by the encoding / decoding unit 7 is supplied to the rearrangement / quantization value adjustment (change) unit 8. The rearrangement / quantization value adjustment unit 8 rearranges the compressed image data in the normal processing direction defined by JPEG, and changes the quantization value to make the code amount constant. Here, the normal processing direction means an encoding order in which an original image is obtained when decoding with JPEG. In addition, the quantization value means a quantization table in the case of JPEG. When quantization is performed by combining a quantization table and a quantization scale, the quantization table and the quantization scale are quantization values. The encoded image data is stored in the storage 9. When the encoding / decoding unit 7 performs a decoding process, the read data of the storage 9 is inversely quantized and the inversely quantized output is inversely DCTed.

  A CPU (Central Processing Unit) 11 is provided to control the signal processing and the like of the digital still camera described above. The camera signal processing unit 4, the image processing unit 6, the encoding / decoding unit 7, the rearrangement circuit 8, and the storage 9 are connected to the CPU 11 via the CPU bus 12 and can be controlled by the CPU 11. The CPU 11 receives an output of a key 13 such as a switch operated by a user and a GUI (Graphical User Interface). Further, a ROM 14 storing programs and the like and a RAM 15 as a working memory for the CPU 11 are connected to the CPU bus 12. Image data can be stored in the RAM 15 under the control of the CPU 11. In the configuration of FIG. 1, for simplicity, the configuration for decoding the compressed image data read from the storage 9 and displaying it on the display 5 is omitted.

  In the configuration of the above-described digital still camera, the present invention performs processing in the image processing unit 6, the encoding / decoding unit 7, and the rearrangement / quantization value (quantization table as described above in JPEG) adjustment unit 8. High-speed processing and large-capacity memory are unnecessary. Note that the JPEG in this embodiment has a macroblock size of (16 × 8) pixels as a coding unit, two macroblock (8 × 8) luminance (Y) blocks, Are composed of (8 × 8) pixel color difference (Cr, Cb) blocks. The interval between the pixels of the color difference data in the horizontal direction is twice that of the luminance data. Therefore, the average number of bits per pixel is 16 bits (Y = 8 bits, Cr = 4 bits, Cb = 4 bits). The luminance data and the two color difference data are separately processed. Since the processing of the color difference data is the same as that of the luminance data, the luminance data processing will be described in the following description.

  First, the outline of the processing of the present invention will be described with reference to FIG. The image processing unit 101 enlarges / reduces an image by converting a resolution called resizing a digital image signal formed based on an imaging signal from an imaging device or a digital image signal read from a storage medium. Image processing such as processing, image rotation processing, chromatic aberration correction processing, image distortion correction processing, camera signal processing, black-and-white inversion processing, color space conversion processing, and filtering processing is performed. The imaging results recorded in the storage medium are sequentially cut out in accordance with the image processing unit and the image processing order with a block size suitable for image processing.

  The image data from the image processing unit 101 is converted from the spatial domain to the frequency domain in the DCT unit 102 in units of (8 × 8) pixel blocks. The (8 × 8) coefficient data from the DCT unit 102 is output to the quantization unit 103 in the zigzag scan order, and is quantized by the quantization unit 103. Quantization is a process of dividing coefficient data by the quantization value in the quantization table and rounding off the division result. The quantization value is not the same for (8 × 8) coefficient data, and an appropriate quantization value is used for each coefficient data. However, in this specification, in order to facilitate understanding of the invention, it is assumed that a quantization table including a single value of a quantization value is used.

  The output of the quantization unit 103 is supplied to the variable length coding / decoding unit 104, and the information amount is compressed by entropy coding. For example, variable length codes corresponding to zero runs and levels are assigned. The DCT unit 102, the quantization unit 103, and the variable length encoding / decoding unit 104 constitute a JPEG encoder.

  The arrangement of the code 105 data obtained at the output of the variable length encoding / decoding unit 104 is defined by the processing of the image processing unit 101. On the other hand, in encoding processing such as JPEG, the format specifies that encoding processing units of macroblocks are sequentially encoded in raster scanning order. Therefore, when the code 105 having the order of the image processing is decoded, an image different from the desired image is obtained.

  In order to solve this problem, rearrangement is performed by the code rearrangement / quantization value adjustment unit 108 without using a large-capacity buffer memory, and a code 109 is output. The code | symbol 109 is recorded on a storage medium, for example. When the code 109 is decoded in a normal processing direction defined by an encoding format such as JPEG, a desired image can be obtained.

  The encoding process will be described more specifically with reference to FIG. For simplicity, it is assumed that the DCT block consists of (4 × 4) pixels. The DCT unit 102 obtains (4 × 4) coefficient data. There is a DC component value (for example, 8) in the upper left corner.

  Each coefficient data is divided by a quantization value (for example, a value of 3) by the quantization unit 103, and the division result is rounded to obtain a quantized coefficient. The quantized coefficients are sequentially output in the order of, for example, zigzag scanning, and the number of zeros is counted. As a result, a combination of the level and the number of zeros (zero run) is obtained.

  Data of a combination of the level and the number of zeros is supplied to the variable length coding unit 104. A variable length code (Huffman code) is obtained by referring to the Huffman table provided in the variable length coding unit 104.

  A series of processes shown in FIG. 3 is JPEG encoding. One method for adjusting the code amount is to perform decoding as shown in FIG. The variable length decoding unit 111 decodes the variable length code obtained by the encoding described above. As a result of decoding, data of a combination of the level and the number of zeros (zero run) is obtained. The inverse quantization unit 112 multiplies each coefficient data by (3), which is the reverse process of quantization. As a result, (4 × 4) DCT coefficient components are obtained.

  In order to adjust the code amount, the quantization unit 103 divides each coefficient data by the quantized values of different values, for example, the quantized value of (4), and rounds the division result to obtain the quantized coefficients. Since the quantization value is increased from (3) to (4), the quantization coefficient becomes smaller. The quantized coefficients are sequentially output in the order of, for example, zigzag scanning, and the number of zeros is counted. As a result, a combination of level and number of zeros is obtained.

  Data of a combination of the level and the number of zeros is supplied to the variable length coding unit 104. A Huffman code is obtained by referring to the Huffman table provided in the variable length coding unit 104. For example, the quantization table is increased (or decreased) in order, and decoding and re-encoding are repeated so that the code amount of the generated variable-length code is less than or equal to the assigned code amount, and an optimal quantization table is obtained. It is done.

  As described above, the decoding and re-encoding using the DCT coefficient data, for example, a method in which the code amount of one screen is substantially allocated code amount, requires a long processing time and requires a memory for decoding. There is a problem.

  As a method for adjusting the quantization table without performing decoding and re-encoding, it is preferable to use a variable-length code replacement table 121 as shown in FIG. Conversion of variable length codes having different quantization values can be performed by code replacement by the variable length code replacement table 121. This code replacement process is performed in the rearrangement / quantization value adjustment unit 8.

  The example shown in FIG. 5 is an example in which the variable length code with the quantization value (3) described with reference to FIGS. 3 and 4 is converted into the variable length code with the quantization value (4). In this case, it is not necessary to decode and re-encode the variable length code, and the processing time can be shortened. However, the AC coefficient component can be used for the variable length code replacement table 121, and the quantized value using the variable length code replacement table 121 cannot be changed for the DC coefficient component. The encoding process for the AC coefficient component and the DC coefficient component is performed in parallel. The encoding process for the DC coefficient component will be described later.

  A method for creating the variable-length code replacement table 121 will be described with reference to FIGS. The example of FIG. 7 is a case where a plurality of quantized values (3) and (4) are converted into quantized values (5). In the first step S101, a new quantization value is determined from the code amount or the like. For example, as will be described later, an initial quantized value is set using code length information obtained by pre-encoding required for rearrangement.

  In step S102, all the Huffman codes of all blocks before the quantization value change are listed. FIG. 7 shows an example of the original Huffman codes listed corresponding to step S102. The meaning of each Huffman code is shown in the table TBL.

  In step S103, it is determined whether a block having the same quantization value and the same Huffman table has been processed before. If the determination result is affirmative, the process moves to step S104, and the next block is set as a processing target.

  If the determination result in step S103 is negative, the quantization value of the block is read in step S105, and a value corresponding to the pre-change Huffman code before the quantization value change is created in a new column. FIG. 7 shows values (1, 2, 3, 4, 5, 6) (see table TBL) corresponding to the generated pre-change Huffman code before changing the quantized value, corresponding to step S105. ing.

  In step S106, the value corresponding to the pre-change Huffman code is inversely quantized with the pre-change quantized value (3). FIG. 7 shows values (3, 6, 9, 12, 15, 18) of the result of inverse quantization corresponding to step S106.

  In step S107, the value corresponding to the pre-change Huffman code is quantized with the post-change quantization value (5). FIG. 7 shows the quantization coefficients (1, 1, 2, 2, 3, 4) corresponding to step S107.

  In step S108, the value quantized with the changed quantization value is encoded with the changed Huffman code. FIG. 7 shows the result of encoding the quantized coefficients (1, 1, 2, 2, 3, 4) with the changed Huffman code corresponding to step S108.

  In step S109, it is determined whether or not processing has been completed for all blocks. When it is determined that the process is finished, the variable-length code replacement table creation process is finished. If not, the process proceeds to the next block in step S104.

  The process described above is for the case where the quantized value is (3), but FIG. 7 shows an example of the process when the quantized value is (4). That is, in step S105, a value corresponding to the pre-change Huffman code (original code) before the quantization value change is created.

  In step S106, the value corresponding to the pre-change Huffman code is inversely quantized with the quantized value (4) before the change. In step S107, the value corresponding to the pre-change Huffman code is quantized with the post-change quantization value (5). In step S108, the value quantized with the changed quantization value is encoded with the changed Huffman code.

  The meaning of each column of the variable length code replacement table created as described above is shown in FIG. 8A, and the reference method of the variable length code replacement table is shown in FIG. 8B. The original code string means the code before change. The quantized value 3 and the quantized value 4 are quantized values before being changed. The remaining codes are the codes after the change.

  As shown in FIG. 8B, for example, when the quantized value before the change is the quantized value 4 and the original code is (100), the new code after the change is (011).

  JPEG is an encoding method in which coefficient data is quantized using a single quantization table. However, after JPEG encoding, as shown in FIG. 9, when one image is divided into, for example, three areas A, B, and C, the quantization table for each area can be individually set. . Each area is a unit of image processing, and data in each area is data to be rearranged. Finally, each of the areas A, B, and C is processed as one block in FIG. 6 and the processing in FIG. 6 is performed to create the conversion table in FIG. By performing the processing of FIG. 13 described later, a plurality of quantization tables used in each of the regions A, B, and C are changed to one, and encoded data that conforms to the encoding standard is formed.

  An example of an encoding method for making the code amount generated in the entire image substantially constant will be described. For example, an image consisting of regions A, B, and C is encoded with JPEG, and the generated variable length code is changed to one corresponding to the quantization value set using the variable length code replacement table created as described above. To do. For example, as a result of encoding the region A with the quantization value 4, there is no more code amount, so the region B is encoded with the quantization value 5. As a result of encoding regions A and B, there is a margin in the code amount, so region C is encoded with quantized value 3. Finally, each of the regions A, B, and C is processed as one block in FIG. 6, and a plurality of quantization values used in each of the regions A, B, and C are changed to one quantization value 5.

  Processing for changing the quantized value in this way will be described with reference to the flowchart of FIG. In the first step S201, a part of the image is encoded with an initial quantized value. A part of the image is an area smaller than a unit area of each image processing. In step S202, it is determined whether or not the generated code amount is larger than the code amount assigned to a part of the image.

  If it is determined that the code amount is large, the quantization value is increased in step S203. On the other hand, if it is determined that the code amount is not large, the quantization value is decreased in step S204. In this way, an appropriate quantization value is determined based on the code amount as a result of encoding a part of the image.

  In step S205, a certain region, for example, region A is encoded with the quantized value determined in step S203 or S204.

  In step S206, it is determined whether or not all regions A, B, and C have been encoded. If it is determined that all regions have been encoded, the process ends. If it is determined that not all regions are encoded, the process returns to step S202.

  After each process is completed, each area is processed as one block in FIG. 6 and the process in FIG. 6 is performed to create the conversion table in FIG. 8, and the conversion in the table in FIG. 8 and the process in FIG. As a result, the plurality of quantization tables used in each of the regions A, B, and C are changed to one, and encoded data that conforms to the encoding standard is formed.

  In the encoding method for the AC coefficient component described above, the quantization value is changed in units of regions corresponding to the image processing units. Instead of changing the quantization value, a Huffman table for Huffman coding may be changed. Further, both the quantization value and the Huffman table may be changed. That is, as shown in FIG. 11, when the entire image is composed of six regions, each region is encoded with a combination of a quantization value (for example, 3 or 4) and a Huffman table (table A or B). .

  When encoding using multiple quantized values, the quantized coefficient distribution after quantization can be predicted by obtaining statistical information (such as a histogram) on what transform coefficient distribution to take, so the Huffman code The overall code amount can be reduced by assigning short Huffman codes to frequently generated quantization coefficients, which is a general method of code assignment.

  An example of a variable-length code replacement table that can be used when changing the quantization value and the Huffman table is shown in FIG. Similar to the variable-length code replacement table (FIG. 8) used when only the quantized value is changed, the original code string means the code before change. The quantized value 3 / huffmanA means the quantized value 3 and the Huffman table A before the change. The quantized value 4 / huffmanA means the quantized value 4 and the Huffman table A before the change. The quantized value 4 / huffman B means the quantized value 4 and the Huffman table B before the change. The remaining codes are the codes after the change.

  In addition, the code portion that does not occur is unsigned. Further, when the same code exists in both of the two Huffman tables A and B, values are entered in both the Huffman tables A and B. For example, when the quantized value and the Huffman table before the change are the quantized value 4 / huffmanA and the original code is (110), the new code is (101).

  The encoding process described above is for AC coefficient components. The difference between the DC coefficient component and the DC coefficient component of the immediately preceding block is Huffman encoded. The encoding for the DC coefficient component will be described below.

  As for the AC coefficient described above, a variable-length code after quantization value conversion can be obtained by the variable-length code replacement table, but such a code replacement cannot be applied to the DC coefficient. As shown in FIG. 13, for example, a Huffman code (100) is decoded by variable length decoding (either by processing or a circuit) 131, and, for example, a decoding result of 3 is obtained. The difference value 12 is obtained by the inverse quantization 132. A pre-decoded value before changing the quantized value, for example, 1 is added 133, and the DC coefficient value 13 is decoded.

  Next, the pre-decoded value after changing the quantized value is subtracted from the DC coefficient value. For example, when the pre-decoded value changes from 1 to 2 due to the change of the quantized value, subtraction of (13-2 = 11) is performed in the subtraction 134, and the difference value 11 is obtained. The difference value is quantized by the quantization 135, and a quantized output (2) is obtained. The quantized output is variable-length encoded by the variable-length encoding 136 to obtain a Huffman code (011).

  In the case of a DC coefficient value, the previous value decoded value may change as a result of the change of the quantized value. Therefore, a process of decoding up to the DC coefficient value and re-encoding is necessary. Further, processing of DC coefficient components related to the rearrangement processing will be described.

  In one embodiment of the present invention, after repeating decoding, image processing, and encoding in an image processing unit and image processing order for image processing, the encoded data for one screen is rearranged and recorded. The buffer memory that temporarily holds the image processing result can be reduced in capacity. As shown in FIG. 14A, after the first image processing unit AR1 with nine macroblocks 1-3, 7-9, 13-15 is generated by image processing, the remaining macroblocks 4-6, 10-12, It is assumed that the second image processing unit AR2 by 16 to 18 is generated by image processing. In this case, encoding is performed by sequentially encoding the first image processing unit AR1 and the second image processing unit AR2 by, for example, JPEG, and as indicated by arrows, the image processing units AR1 and AR2 are encoded. Encoded data is generated by encoding each macroblock in the order of raster scanning.

  On the other hand, as shown in FIG. 14B, when one image by these image processing units AR1 and AR2 is encoded by JPEG, the macroblock of this one image is rasterized according to the numerical order indicating each macroblock. The encoded data is generated in the scanning order. Therefore, the image obtained as a result of decoding the data in the order shown in FIG. 14A is different from the original image.

  In one embodiment of the present invention, encoded data obtained in the order shown in FIG. 14A is rearranged in the order shown in FIG. 14B by the rearrangement / quantization value adjusting unit 8 and output. In JPEG encoding, the DC coefficient value of the immediately preceding macroblock is held in advance, and in the subsequent macroblock, the difference value from the DC coefficient value held in front is variable-length encoded. Therefore, in the example shown in FIG. 14A, a difference value between each DC coefficient value of the macroblocks 7 and 13 and each DC coefficient value of the macroblocks 3 and 9 is obtained, and each difference value is variable-length encoded. The Similarly, the difference value between the DC coefficient values of the macroblocks 4, 10, and 16 and the DC coefficient values of the macroblocks 15, 6, and 12 is variable-length encoded.

  In the JPEG standard, it is defined that a delimiter code is inserted as a delimiter for each block of JPEG data. Stuffing bits are added so that the variable length encoded JPEG data for one block has a length that is an integral multiple of bytes, and then a delimiter code (16 bits) is added. A JPEG MCU (Minimum Coded Unit) as a minimum coding unit is data generated corresponding to a macroblock. Since Huffman coding is performed, the JPEG MCU has a variable bit length. The delimiter code indicates the delimiter of the replacement unit. This division is called ECS (Entropy Coded Segment). Therefore, the rearrangement is performed in units of ECS. The DC (direct current) coefficient of the JPEG data is transmitted as a difference data from the DC coefficient of the previous block, and the data of the DC coefficient next to the delimiter code is not the difference but the DC coefficient value itself. Has been.

  In one embodiment of the present invention, an image processing result is encoded for each image processing unit without inserting a stuffing bit and a delimiter code. However, the present invention is not limited to this, and a stuffing bit and / or a delimiter code may be inserted, and the stuffing bit and the delimiter code may be deleted when rearranging in the rearrangement unit. When a delimiter code is provided, the DC coefficient value is directly encoded into the DC coefficient in the macroblock immediately after. As a result, in this case, DC coefficient processing is separately required. For example, after a dummy macroblock is created from DC coefficient values recorded and held, and encoded data is generated continuously with the dummy macroblock. This can be executed by deleting the encoded data of the dummy macroblock. Incidentally, the dummy macroblock may be applied with the actual image data of the macroblock immediately before the macroblock to be processed in the rearranged encoded data.

  On the other hand, when the image processing units AR1 and AR2 are encoded as one image, the macroblocks 7 and 13 are the DCs of the macroblocks 6 and 12 at the right end (toward the drawing) that is the scanning end in the horizontal direction. The DC coefficient value is encoded by the difference value from the coefficient value. In the macroblocks 4, 10, and 16 on the scanning start end side in the horizontal direction of the second image processing unit AR2, the horizontal value of the first image processing unit AR1 is set. The DC coefficient value is encoded by the difference value from the DC coefficient values of the macroblocks 3, 9, 15 on the scanning end end side in the direction.

  Accordingly, it is difficult to correctly decode the encoded data in units of macroblocks obtained by the order of image processing, by rearranging the encoded data in one screen and further resetting the control code. .

  Therefore, when encoding a macroblock of an image processing unit in the raster scan order, the detected DC coefficient value is acquired and recorded in the macroblock at the end of each horizontal scan in the raster scan, and is adjacent to each other. At the time of encoding in the macroblock to be used, the recorded DC coefficient value is used instead of the DC coefficient value held in advance. An adjacent macroblock is an adjacent macroblock after rearrangement.

  In the example of FIG. 14, when encoding the first image processing unit AR <b> 1 by the macroblocks 1 to 3, 7 to 9, and 13 to 15, the DC of the macroblocks 6, 12, and 18 at the scanning end in the horizontal direction. Record and retain coefficient values. When the subsequent second image processing unit AR2 is encoded, the DC coefficients of the recorded macroblocks 3, 9, and 15 are used to record the macroblocks 4, 10, and 16 at the scanning start end in the horizontal direction. Encoding is performed.

  FIG. 15 shows a code output method suitable for rearrangement. The encoding unit 201 performs JPEG encoding. For example, macroblocks 7, 8, and 9 of the processing unit AR1 are input to the encoding unit 201 as image data 211 from the image processing unit. A DC coefficient value 212 of a macroblock necessary for forming a difference from the DC coefficient value of the leftmost macroblock is input. For example, the DC coefficient value of the macro block 6 is input. In the encoding unit 201, the difference between the DC coefficient value of the macroblock 7 and the DC coefficient value of the macroblock 6 instead of the DC coefficient value of the macroblock 3 is calculated, and the difference is variable-length encoded.

  In order to form a difference from the DC coefficient value of the leftmost macroblock 1 other than the macroblock 7, the DC coefficient value of the macroblock at the horizontal scanning end of the previous image is required, and the DC of the macroblock 13 is obtained. In order to form a difference with the coefficient value, the DC coefficient value of the macroblock 12 is required. In the case of the first macroblock 1 of the image, for example, the value itself is transmitted without forming a difference. Therefore, encoding of the macroblocks 6, 12 and 18 precedes encoding of the processing units AR1 and AR2.

  The encoding unit 201 outputs a code 213 before rearrangement, a code length 214 before rearrangement, and a DC coefficient value 215 at the right end of the image. The code length 214 is the code length of three macroblocks, for example, macroblocks 7, 8, and 9. The DC coefficient value 215 at the right end of the image is, for example, the DC coefficient value of the macro block 9. The DC coefficient value at the right end of the image is used for encoding the DC coefficient values of the macroblocks 4, 10, and 16 located at the left end of the next image processing unit AR2.

  As described above, before starting encoding in units of image processing, the rightmost macroblocks 6 and 12 are encoded, and the DC coefficient values of the macroblocks 6 and 12 are acquired and held.

  FIG. 16 shows the process of obtaining DC coefficient values required at the time of code rearrangement and encoding. In the example of FIG. 16, the size of the image processing unit is (2 × 3) macroblocks, and one image is constituted by three image processing units. A data series 221 in the order of A1, A2, A3, A4,... Is input from the image processing unit. The encoding unit performs JPEG encoding, and the data sequence 222 in the normal code order defined by JPEG is formed by rearranging the codes before rearrangement.

  In the data series 222, in order to predictively encode the respective DC coefficient values of the macro blocks B1, C1, A4, B4, and C4 circled, the macro blocks A3, B3, C3, and A6 circled in advance are enclosed. , B6 DC coefficient values need to be acquired. In this example, since the DC coefficient value of the macroblock C3 is required for encoding the leftmost macroblock A4, it is necessary to encode the macroblock C3 located at the rightmost end in advance.

  In this way, a macroblock as a coding unit located at the end (right end) of horizontal scanning of an image is coded in advance (DCT, quantization, variable length coding). Quantization and variable length are not required for predictive coding of DC coefficient values. However, if a plurality of macroblocks located at the right end are encoded in advance to obtain a code length, it is possible to estimate a generated code amount of an image to which the macroblock belongs from the code length. Based on this estimation result, an initial value of the quantized value can be set.

  FIG. 17 is a flowchart showing a DC coefficient value processing procedure performed under the control of the CPU 11. When the processing procedure is started, the process proceeds from step S301 to step S302, and the macroblock (MCU) in which the DC coefficient value cannot be detected by the repetitive encoding process by the image processing unit is controlled by the storage 9 and the encoding / decoding unit 7. The encoded data is decoded to generate image data, and the image processing unit 8 performs image processing on the image data. Also, the image data based on the processing result is encoded by the encoding / decoding unit 7, thereby detecting in advance a DC coefficient value of a macroblock for which a DC coefficient value cannot be prepared by repeated encoding processing in units of image processing.

  Subsequently, the CPU 11 sequentially performs decoding processing and image processing in units of image processing, and stores the image data in the buffer memory. In step S303, image data for one macroblock is input to the encoding / decoding unit 7 from the buffer memory that stores the image processing result in the order of raster scanning. In step S304, DCT processing is performed, and in step S305. The coefficient data obtained by DCT is quantized.

  In step S306, it is determined whether or not the macro block being processed is positioned at the scanning start end in the horizontal direction of the image processing unit. If the determination result of step S306 is affirmative, the process moves from step S306 to step S307. In step S307, after the DC coefficient value by pre-holding, which is the calculation target of the DC difference value in step S310, is replaced with the recorded and held DC coefficient value, the process moves to step S308. If the determination result of step S306 is negative, the process proceeds from step S306 to step S308.

  In step S308, it is determined whether or not the currently processed macroblock is the end of scanning in the horizontal direction of the image processing unit. If the determination result of step S308 is affirmative, the DC coefficient value detected in step S305 is recorded in step S309, and the process proceeds to step S310. If the determination result of step S308 is negative, the process proceeds from step S308 to step S310.

  In step S310, the previous DC coefficient value is subtracted from the DC coefficient value quantized in step S305 to generate a difference value. In step S311, the difference value calculated in step S310 and the coefficient data relating to the AC value quantized in step S305 are Huffman encoded. In step S312, it is determined whether processing has been completed for all macroblocks. If the determination result of step S312 is negative, the process returns from step S312 to step S303. If the determination result of step S312 is affirmative, the processing procedure ends (step S313).

Although the embodiment of the present invention has been specifically described above, the present invention is not limited to the above-described embodiment, and various modifications based on the technical idea of the present invention are possible. For example, as an image process in a digital still camera, a process of rotating an image decoded at the time of rearrangement by 180 degrees may be performed. In the case of rotation processing, the macroblock that holds the DC component coefficient value is not limited to the scanning end macroblock in each horizontal scan, but may be a macroblock on the horizontal scanning start end side.

  Further, the present invention is not limited to a digital still camera, but is applied to an image recording apparatus, a personal computer, a PDA (Personal Digital Assistants), an image transmission apparatus, a portable terminal, a cellular phone, and the like provided with a JPEG or MPEG encoding apparatus. Can do.

1 is a block diagram illustrating a configuration of an imaging apparatus to which the present invention is applied. It is a basic diagram for demonstrating the flow of the process of an encoding and rearrangement by this invention. It is a basic diagram for demonstrating a JPEG encoding process. It is a basic diagram for demonstrating an example of the process which changes a quantization value by decoding and re-encoding. It is a basic diagram for demonstrating an example of the process which changes a quantization value with a variable-length code replacement table. It is a flowchart for demonstrating an example of the preparation method of a variable-length code replacement table. It is a basic diagram for demonstrating an example of the production method of a variable-length code replacement table. It is a basic diagram used for description of a variable-length code replacement table. It is a basic diagram for demonstrating the process which changes a quantization value for every area | region. It is a flowchart for demonstrating the process which changes a quantization value for every area | region. It is a basic diagram for demonstrating the process which changes a quantization value and a Huffman table for every area | region. It is a basic diagram used for description of the variable-length code replacement table in the case of changing a quantization value and a Huffman table. It is a basic diagram for demonstrating the process which changes a quantization value regarding DC coefficient component. It is a basic diagram for demonstrating the process of rearrangement. It is a basic diagram for demonstrating the output method of the code | symbol suitable for rearrangement. It is a basic diagram for demonstrating the rearrangement process and the process of encoding of a DC coefficient component. It is a flowchart for demonstrating the encoding process of a DC coefficient component. It is a basic diagram for demonstrating the flow of the process of an encoding and rearrangement proposed previously.

Explanation of symbols

2 Image Sensor 4 Camera Signal Processing Unit 6 Image Processing Unit 7 Encoding / Decoding Unit 8, 108 Rearrangement / Quantization Value Adjustment Unit 11 CPU

Claims (9)

In an image processing method for encoding image data processed by an image processing unit in a predetermined encoding unit in an encoding unit,
An encoding step of performing conversion in the order of data output from the image processing unit, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient;
Rearrangement / quantization value change for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to the normal processing direction defined by the encoding And an image processing method.   In the rearrangement / quantization value changing step, the quantization value is changed by directly replacing at least one code from a variable length code and a quantization value to a variable length code corresponding to a new quantization value. Item 8. The image processing method according to Item 1. The coefficient data consists of a DC coefficient component and an AC coefficient component.
The image processing method according to claim 1, wherein the encoding step encodes a difference from the previous value in the normal processing direction when the DC coefficient component is encoded as a previous value difference.   The DC coefficient component value is obtained from the variable length code only for the code of the DC coefficient component, and the difference from the decoded value of the code by the new quantization value of the difference target coding unit is quantized with the new quantization value, the variable length code. The image processing method according to claim 1.   2. The image processing method according to claim 1, wherein the previous value in the normal processing direction is obtained by pre-encoding, the code length is obtained by pre-encoding, and an initial value of the quantized value is set based on the code length. In a program for causing a computer to execute an image processing method for encoding image data processed by an image processing unit in a predetermined encoding unit in an encoding unit,
An encoding step of performing conversion in the order of data output from the image processing unit, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient;
Rearrangement / quantization value change for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to the normal processing direction defined by the encoding An image processing method program comprising steps. In an image processing apparatus that encodes image data processed by an image processing unit in a predetermined encoding unit in an encoding unit,
An encoding unit that performs conversion in the order of data output from the image processing unit, quantizes coefficient data generated by the conversion, and variable-length-encodes the quantized coefficient;
Rearrangement / quantization value change for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to the normal processing direction defined by the encoding An image processing apparatus. In an imaging apparatus having an image processing unit that performs image processing on image data captured by an imaging device, and an encoding unit that encodes image data processed by the image processing unit in a predetermined encoding unit by an encoding unit ,
An encoding unit that performs conversion in the order of data output from the image processing unit, quantizes coefficient data generated by the conversion, and variable-length-encodes the quantized coefficient;
Rearrangement / quantization value change for adjusting the quantization value for each processing unit area of the image processing unit and rearranging from the processing direction of the image processing unit to the normal processing direction defined by the encoding An imaging device. An image processing method for encoding image data in a predetermined encoding unit by an encoding unit,
In an image processing method having an encoding step for performing conversion, quantizing coefficient data generated by the conversion, and variable-length encoding the quantized coefficient, and a quantization value changing step,
The image processing method in which the quantization value changing step changes the quantization value by directly replacing at least one code from a variable length code and a quantization value to a variable length code corresponding to a new quantization value.

Images