JP2014123865A - Image processing apparatus and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve image quality while suppressing an increase in code amount.SOLUTION: Predetermined data (d) from a memory 130 is considered as data obtained by subjecting difference image data between an input image and a prediction image to orthogonal transform and quantization. An image processing apparatus subjects the predetermined data to inverse quantization and inverse orthogonal transform to add prediction image data (d) to the resulting data (d) to reconstruct an input image. At this time, the apparatus variously changes combinations of parameters of the inverse quantization and the inverse orthogonal transform to generate n kinds of reconstruction images (d{i}-{n}) to calculate degrees of difference (DIFF{i}-{n}) between the input images and the reconstruction images for the respective combinations. The apparatus generates encoded data (d) by using the parameters and the predetermined data (d) in which an evaluation value based on the degree of difference becomes minimum or the degree of difference becomes minimum.

Description

  The present invention relates to an image processing apparatus and an imaging apparatus.

  MPEG-4 AVC / H. In a moving image compression method such as H.264, transform data obtained by orthogonal transform of a difference between an input image and a predicted image is quantized using a specified quantization scale, and the result is encoded. At this time, the quantization scale can be adjusted according to the target code amount and the feature amount of the input image (see, for example, Patent Document 1 below). If the difference information is restored by inverse quantization and inverse orthogonal transformation of the quantization result data, the input image can be reconstructed (restored) using the predicted image. However, a shift due to quantization error occurs between the reconstructed image obtained through quantization and the input image.

JP 2011-172137 A

  If this shift is suppressed, the image quality of the compressed image can be improved. Needless to say, improving image quality is beneficial. In conventional encoders, reducing the quantization scale, which is a parameter for quantization and dequantization, reduces the quantization error and usually reduces the deviation between the input image and the reconstructed image. The amount increases. Control of the code amount is also an important issue.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus and an imaging apparatus that contribute to an improvement in image quality while suppressing an increase in code amount.

  An image processing apparatus according to the present invention includes an encoder that generates encoded data corresponding to an input image and a predicted image of the input image, wherein the encoder is between the input image and the predicted image. As data corresponding to data obtained by orthogonally transforming and quantizing the difference image data, a memory that holds predetermined data, a restored difference image data obtained by inverse quantization and inverse orthogonal transform of the held data, and image data of the predicted image From the sum, a reconstructed image generating unit that generates a reconstructed image of the input image, a difference degree deriving unit that derives a difference degree between the input image and the reconstructed image, and an encoding that generates the encoded data And using a plurality of combinations as a combination of the inverse quantization parameter and the inverse orthogonal transform parameter. Generating and deriving a plurality of reconstructed images and a plurality of dissimilarities corresponding to an error, and the encoder further includes a combination selection unit that selects a specific combination from the plurality of combinations based on the plurality of dissimilarities. The encoding unit generates the encoded data based on a parameter corresponding to the specific combination and the retained data.

  If a specific combination (for example, a specific combination that reduces the difference) is selected from a plurality of combinations based on a plurality of differences, and encoded data is generated using parameters corresponding to the specific combinations. The shift between the input image and the image restored from the encoded data (reconstructed image) is suppressed, and the image quality of the encoded data can be improved. Regardless of which combination of parameters is used to generate the encoded data, the encoded data is generated from the predetermined retained data, so that there is little or no increase in the code amount associated with reducing the deviation. Also, the orthogonal transform and quantization processes can be omitted.

  Specifically, for example, the combination selection unit obtains a plurality of evaluation values by deriving an evaluation value based on the degree of difference and the code amount for each combination, and determines the specific combination based on the plurality of evaluation values. You may choose. Here, for example, the plurality of combinations include first to n-th combinations, and the code amount corresponding to the i-th combination includes data corresponding to the i-th combination in data obtained by encoding the retained data. This is the code amount of data with additional information added.

  Alternatively, for example, the combination selection unit may select a combination corresponding to the minimum difference among the plurality of differences as the specific combination.

  Also, for example, an orthogonal transform / quantization unit that generates quantized transform data by orthogonal transform and quantization of the difference image data, and a second obtained by inverse quantization and inverse orthogonal transform of the quantized transform data A second reconstructed image generation unit that generates a second reconstructed image of the input image from the sum of the restored difference image data and the image data of the predicted image, a second difference between the input image and the second reconstructed image A second encoder having a second degree-of-difference deriving unit for deriving a degree, a second encoder for generating second encoded data from the quantized transform data, and the identification among the plurality of evaluation values Based on the first evaluation value corresponding to the combination and the second evaluation value based on the second difference and the second code amount, the encoded data by the encoder or the second encoded data by the second encoder. And an output selection unit that selects a data as output encoded data, and the imaging apparatus may further include an output selection unit. This is a code amount of data to which additional information including the orthogonal transform and quantization parameters in the encoder is added.

  This makes it possible to complement the advantages / disadvantages of the two encoders, and it is possible to select and output encoded data that is determined to be better in terms of the degree of difference and the amount of code.

  Alternatively, for example, an orthogonal transform / quantization unit that generates quantized transform data by performing orthogonal transform and quantization on the difference image data, and a second obtained by performing inverse quantization and inverse orthogonal transform on the quantized transform data. A second reconstructed image generation unit that generates a second reconstructed image of the input image from the sum of the restored difference image data and the image data of the predicted image, a second difference between the input image and the second reconstructed image A second encoder having a second degree-of-difference deriving unit for deriving a degree, a second encoding unit for generating second encoded data from the quantized transform data, and the identification among the plurality of degrees of difference Based on the first dissimilarity corresponding to the combination and the second dissimilarity, the encoded data by the encoder or the second encoded data by the second encoder is selected as output encoded data. An output selector which may be further provided to the imaging apparatus.

  This also makes it possible to complement the advantages / disadvantages of the two encoders and to select and output encoded data that is determined to be better from the viewpoint of the degree of difference.

  An imaging apparatus according to the present invention is an imaging apparatus that acquires an input image by imaging, and includes the image processing apparatus that receives supply of image data of the input image.

  According to the present invention, it is possible to provide an image processing apparatus and an imaging apparatus that contribute to image quality improvement while suppressing an increase in code amount.

It is an internal block diagram of the basic encoder which concerns on 1st Embodiment of this invention. It is a figure which shows the structure of a moving image. 1 is an external perspective view of an imaging apparatus according to a first embodiment of the present invention. It is an internal block diagram of the special encoder which concerns on 1st Embodiment of this invention. It is a figure for comparing the data of a basic encoder and a special encoder. It is a block diagram around an encoder according to a second embodiment of the present invention. It is an internal block diagram of the imaging device which concerns on 3rd Embodiment of this invention.

  Hereinafter, an example of an embodiment of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle. In this specification, for simplification of description, a symbol or reference that refers to information, signal, physical quantity, state quantity, member, or the like is written to indicate information, signal, physical quantity, state quantity or Names of members and the like may be omitted or abbreviated.

<< First Embodiment >>
A first embodiment of the present invention will be described. The second and third embodiments to be described later are embodiments based on the first embodiment. Regarding matters not specifically described in the second and third embodiments, unless otherwise specified and there is no contradiction, The description of the first embodiment also applies to the second and third embodiments.

[About basic encoder]
Prior to the description of the special encoder according to the first embodiment of the present invention, the configuration of the basic encoder 10 will be described. FIG. 1 shows an internal block diagram of the basic encoder 10. FIG. 2 shows the structure of the moving image MI. FIG. 3 shows an external perspective view of the image pickup apparatus 1 including an encoder. A basic encoder 10 can be provided in the imaging apparatus 1. The imaging device 1 acquires a moving image MI by shooting. The moving image MI is composed of a plurality of frame images arranged in time series. The i-th frame image is represented by the symbol FI [i] (i is an integer). Each frame image forming the moving image MI is sequentially supplied to the basic encoder 10 as an input image. The basic encoder 10 is provided with each part referred by the codes | symbols 11-23. The buffer memory 31 and the recording medium 32 are provided in the imaging device 1. The basic encoder 10 encodes the image data of the moving image MI according to an arbitrary moving image compression method. The moving image compression method may be in accordance with the MPEG (Moving Picture Experts Group) standard, for example, in accordance with the “MPEG-4 AVC / H.264” standard.

  The basic encoder 10 divides each input image into a plurality of macro blocks, and executes each process in the basic encoder 10 (process of each part referenced by reference numerals 12 to 22) for each macro block. In the following, it is assumed that one macroblock is formed by (16 × 16) pixels.

Image data (hereinafter referred to as input image data) d _{A of the} input image is supplied to the input terminal 11. The subtraction unit 12 subtracts the image data (hereinafter referred to as prediction image data) d _B of the prediction image supplied from the prediction image generation unit 19 from the input image data d _A , thereby obtaining a difference image between the input image and the prediction image. Image data (hereinafter referred to as difference image data) d _C is generated. Since the predicted image is an image obtained by predicting the input image, and the difference image indicates a difference between the input image and the predicted image, the difference image data can also be referred to as prediction error data.

Orthogonal transform unit 13 generates the conversion data d _D by orthogonal transformation of the difference image data d _C. As the orthogonal transformation, an arbitrary orthogonal transformation such as a discrete cosine transform (hereinafter referred to as DCT) or a Karhunen-Loeve transformation is used (the same applies to the encoder 110 described later). The difference image data d _C is image data that represents the difference image on the spatial domain, while the conversion data d _D may be image data that represents the difference image on the frequency domain. For example, when orthogonal transform is performed in units of (8 × 8) pixel small blocks, a matrix having (8 × 8) transform coefficients (for example, DCT coefficients) for each small block. Conversion data d _D is obtained.

Quantization unit 14 quantizes the transform data d _D, to generate the converted data d _D after quantization as the quantization conversion data d _E. In the quantization, each transform coefficient forming the transform data d _D is divided by a value larger than 1 that is inversely proportional to the quantization scale, and the quantized transform data d _E is generated by rounding down the decimal point of each division result. .

Coding section 15 performs coding processing to encode the quantized transform data d _E, coded data (coded by adding additional information 40 to the encoded quantized transform data d _E Quantized transformation data d _E and encoded data including additional information 40) d _F are generated. The encoding unit 15 and the basic encoder 10 can output and record the encoded data d _F from the output terminal 23 to the recording medium 32 through the buffer memory 31. The additional information 40 will be described later.

The inverse quantization unit 16 generates transformed data d _D ′ by inversely quantizing the quantized transformed data d _E. Inverse orthogonal transform unit 17, 'restores the difference image data d _C by inverse orthogonal transform and thereby restored difference image data d _C' converts data d _D to generate. The adding unit 18 reconstructs (restores) the input image by adding the predicted image data d _B to the restored difference image data d _C ′. The reconstructed input image is referred to as a reconstructed image (restored input image), and the image data of the reconstructed image corresponding to the addition result of the data d _C ′ and d _B is referred to as reconstructed image data d _A ′. Inverse quantization is the inverse operation of quantization, and inverse orthogonal transformation is the inverse operation of orthogonal transformation. Therefore, if it is assumed that there is no decimal point truncation processing in the quantizing unit 14, the data d _D ′, d _C ′, and d _A ′ coincide with the data d _D , d _C , and d _A , respectively.

Prediction image generating unit 19, based on the image data of the moving picture MI, generates the prediction image and the prediction image data d _B by known methods according to the standard of moving picture compression scheme employed by the encoder 10. Prediction image generating unit 19 may generate a prediction image and the prediction image data d _B was made motion compensation was used to predict the motion of an object on a moving picture MI, the motion prediction result.

The difference degree deriving unit 20 derives a difference degree DIFF between the input image data d _A and the reconstructed image data d _A ′ (that is, a difference degree between the input image and the reconstructed image). The evaluation value deriving unit 21 included in the parameter control unit 22 evaluates based on the degree of difference DIFF and the code amount of the encoded data d _F (that is, the data amount of the encoded data d _F per unit time) VOL _F. The value EV is derived, and the parameter control unit 22 adjusts the encoding parameter 41 in the basic encoder 10 based on the evaluation value EV. still. The derivation unit 21 may be provided outside the control unit 22.

The encoding parameter 41 in the basic encoder 10 includes a prediction mode parameter indicating a prediction condition in the predicted image generation unit 19, a transform format parameter indicating an orthogonal transform condition in the orthogonal transform unit 13, and a quantization in the quantization unit 14. Quantization parameters indicating the conditions of
Prediction image generating unit 19 can generate the predictive image data d _B using a prediction mode selected from among a plurality of prediction mode candidates. Prediction mode parameter is a parameter that specifies whether to use the one prediction mode for generating the predictive image data d _B.
The orthogonal transform unit 13 can perform orthogonal transform in a transform format selected from a plurality of transform format candidates. The conversion format parameter is a parameter for designating which conversion format is used for generating the conversion data d _D.
The quantization unit 14 can perform quantization after setting the value of the quantization scale to any one of a plurality of candidate values. The quantization parameter is a parameter that specifies a quantization scale (that is, specifies how many values of the quantization scale are to be set).

Here, for the sake of concrete description, it is assumed that the difference DIFF has a larger value as the input image and the reconstructed image differ from each other. For example, an SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference) between the data d _A and d _A ′ can be obtained as the difference DIFF. Further, it is assumed that the evaluation value EV is obtained such that the evaluation value EV increases as the difference degree DIFF increases and the evaluation value EV increases as the code amount VOL _F increases. Then, the parameter control unit 22 can adjust the encoding parameter 41 so that the evaluation value EV is as small as possible. For example, the evaluation value EV can be derived by the following formula (1). The coefficients k _DIFF and k _VOL have predetermined values of 0 or more, and at least one of the coefficients k _DIFF and k _VOL is greater than 0.
EV = k _DIFF × DIFF [i] + k _VOL × VOL _F [i] (1)

For example, when focusing on a quantization scale, while the decrease in the degree of difference DIFF and the difference between the data d _C and d _C 'when the quantization scale is small becomes small can be expected, in the quantization conversion data d _E The data amount and the code amount VOL _F in the encoded data d _F increase. Conversely, when the value of the quantization scale increases, the difference between the data d _C and d _C ′ increases and the difference DIFF is expected to increase. On the other hand, the data amount and the encoded data in the quantized transformed data d _E The code amount VOL _{F at} d _F decreases. That is, the degree of difference DIFF and the code amount VOL _F are in a so-called trade-off relationship. The parameter control unit 22 can adjust the quantization scale in consideration of the balance between the difference degree DIFF and the code amount VOL _F.

Additional information included in the encoded data d _F 40 includes information for decoding the encoded data d _F. That is, the additional information 40 includes an encoding parameter 41, and further includes motion information indicating the motion, information indicating which part of which frame image is used in predicting the input image, and the like. Therefore, if the encoded data d _F read from the recording medium 32 is supplied to a decoder (not shown) corresponding to the basic encoder 10, the encoded data d _F can be decoded by a known method. The decoder corresponding to the basic encoder 10 uses the additional information 40 to encode the encoded data d _F (encoded quantized transform data d _E ), the encoding unit 15, the quantizing unit 14, and the orthogonal transform unit 13. Reconstructed difference image data is generated by performing inverse encoding, inverse quantization, and inverse orthogonal transformation corresponding to encoding, quantization, and orthogonal transformation of image data, and reconstruction is performed by adding predicted image data to the restored difference image data An image is generated (that is, encoded data d _F is decoded and an input image is restored). In the decoder, predicted image data is generated by a known prediction process using the reconstructed image as a reference image.

[About special encoders]
Next, the special encoder 110 will be described with reference to FIG. FIG. 4 is an internal block diagram of the special encoder 110. The special encoder 110 can be provided in the imaging apparatus 1. Each frame image forming the moving image MI is sequentially supplied to the special encoder 110 as an input image. The special encoder 110 includes parts referred to by reference numerals 111, 115 to 123, 130 and 131. The special encoder 110 encodes the image data of the moving image MI according to the same moving image compression method used by the basic encoder 10.

  Similar to the basic encoder 10, the special encoder 110 divides each input image into a plurality of macro blocks, and performs each process in the special encoder 110 (process of each part referenced by reference numerals 115 to 122 and 131) as a macro block. Run every time.

Image data of an input image, that is, input image data d _A is supplied to the input terminal 111. The special encoder 110 does not have portions corresponding to the subtracting unit 12, the orthogonal transforming unit 13, and the quantizing unit 14 of FIG. 1, and predetermined data d _P is used instead of the quantized transform data d _E of FIG. This is supplied to the encoding unit 115. The data d _P is held in the memory 130 as data corresponding to data obtained by orthogonal transformation and quantization of difference image data between the input image and the predicted image (that is, data d _{E in} FIG. 1). However, the data d _P is not data obtained by actually performing orthogonal transform and quantization in the special encoder 110, but data assumed as an example of the data d _{E in} FIG. 1 is set as the data d _P. The data d _P may be predetermined fixed data. In special encoder 110, the data d _P is considered to be the input image and the quantized transform data orthogonal transformation on the differential image data and obtained by applying the quantization of the prediction image. Thus, the data d _P also called quantized transform data d _P.

Encoding unit 115 performs encoding processing for encoding the quantized transform data d _P, coded data (coded by adding the encoded quantized transform data d _P in the additional information 140 generating encoded data) d _Q containing quantized transform data d _P and the additional information 140. Coding section 115 and the special encoder 110 can output and record the encoded data d _Q from the output terminal 123 to the recording medium 32 through the buffer memory 31. The additional information 140 will be described later.

The inverse quantization unit 116 generates the transformed data d _R that has not been quantized by inversely quantizing the quantized transformed data d _P. The inverse orthogonal transform unit 117 generates data d _S that is image data of the restored difference image by performing inverse orthogonal transform on the transform data d _R. The restored differential image based on the data d _{S corresponds to} a restored differential image between the input image and the predicted image. The adding unit 118 reconstructs (restores) the input image by adding the restored difference image data d _S to the image data (predicted image data) d _B of the predicted image supplied from the predicted image generating unit 119. The reconstructed input image is referred to as a reconstructed image (restored input image), and the image data of the reconstructed image corresponding to the addition result of the data d _S and d _B is referred to as reconstructed image data d _T. The inverse quantization in the inverse quantization unit 116 corresponds to the inverse operation of the above-described quantization, and the inverse orthogonal transform in the inverse orthogonal transform unit 17 corresponds to the inverse operation of the above-described orthogonal transform. The predicted image generation unit 119 is the same as the predicted image generation unit 19 in FIG. Significance of such symbols _{d T [1] ~d T [} n] shown in Figure 4 will be apparent from the following description.

The difference degree deriving unit 120 derives a difference degree DIFF _AT between the input image data d _A and the reconstructed image data d _T (that is, a difference degree between the input image and the reconstructed image). The evaluation value deriving unit 121 included in the combination selection unit 122 is based on the degree of difference DIFF _AT and the code amount of the encoded data d _Q (that is, the data amount of the encoded data d _Q per unit time) VOL _Q. An evaluation value EV _AT is derived. still. The derivation unit 121 may be provided outside the selection unit 122. The function of the selection unit 122 will be described later.

  The parameter control unit 131 controls the encoding parameter 141 in the special encoder 110. The encoding parameter 141 includes a prediction mode parameter indicating a prediction condition in the prediction image generation unit 119, an inverse quantization parameter indicating an inverse quantization condition in the inverse quantization unit 116, and an inverse orthogonal in the inverse orthogonal transform unit 17. Contains an inverse transformation format parameter that indicates the condition of the transformation.

In the moving image compression method employed in the encoders 10 and 110, a plurality of prediction mode candidates are prepared as prediction mode candidates for predicting an input image, and the predicted image generation unit 119 includes a plurality of prediction mode candidates. generating prediction image data d _B using a prediction mode selected from among.

  In the moving image compression method employed by the encoders 10 and 110, a plurality of candidate values are prepared as candidates for quantization scale values used in quantization, and the inverse quantization unit 116 selects one of the plurality of candidate values. Is set to the value of the quantization scale, and inverse quantization can be performed. The inverse quantization parameter is a parameter that specifies a quantization scale (that is, specifies how many values of the quantization scale are to be set). If the condition for inverse quantization is determined, the condition for quantization, which is the inverse operation of inverse quantization, is also determined. Therefore, the inverse quantization parameter is equivalent to the above-described quantization parameter.

In the moving image compression method employed in the encoders 10 and 110, a plurality of conversion format candidates are prepared as candidates for the orthogonal transform format, and the inverse orthogonal transform unit 117 performs conversion selected from the plurality of transform format candidates. Inverse orthogonal transform can be performed in the form. The inverse conversion format parameter is a parameter for designating which conversion format is used for generating the data d _S. If the conditions for inverse orthogonal transform (that is, the selective transform format) are determined, the conditions for orthogonal transform, which is the inverse operation of inverse orthogonal transform, are also determined. Therefore, the inverse conversion format parameter is equivalent to the conversion format parameter described above.

Included in the encoded data d _Q additional information 140 includes information for decoding the encoded data d _Q. That is, the additional information 140 includes an encoding parameter 141, and further includes motion information indicating the motion, information indicating which part of which frame image is used in predicting the input image, and the like. Therefore, if supplied to a decoder (not shown) corresponding to read from the recording medium 32 coded data d _Q special encoder 110, by a known method, it is possible to decode the coded data d _Q. In the decoder corresponding to the special encoder 110, using the additional information 140, the encoded data d _Q (encoded quantized transformation data d _P ) is subjected to inverse encoding corresponding to the encoding of the encoding unit 115, and Reconstructed difference image data is generated by performing inverse quantization and inverse orthogonal transform according to the encoding parameter 141, and a reconstructed image is generated by adding predicted image data to the restored difference image data (that is, encoded data). d _T is decoded and the input image is restored). In the decoder, predicted image data is generated by a known prediction process using the reconstructed image as a reference image.

Now, there is a candidate value of the first to n _A as a candidate value of the, and, assuming that the conversion format candidates of the first to n _B is present as the conversion format candidates described above. Here, each of n _A and n _B is an integer of 2 or more. The inverse quantization unit 116 performs inverse quantization by setting any one of the first to n _A candidate values as a quantization scale value, and the inverse orthogonal transform unit 117 performs the first to n _B transforms. Inverse orthogonal transformation is performed in one of the transformation formats of the format candidates. Then, there are (n _A × n _B ) combinations of inverse quantization and inverse orthogonal transform. In other words, there are (n _A × n _B ) combinations of inverse quantization parameters (inverse quantization parameters) and inverse orthogonal transformation parameters (inverse transformation format parameters). (N _A × n _B ) combinations are referred to as first to nth combinations (n = n _A × n _B ).

The parameter control unit 131 causes the inverse quantization unit 116 and the inverse orthogonal transform unit 117 to perform inverse quantization and inverse orthogonal transform for each of the first to nth combinations to obtain restored difference image data d _S. The reconstructed image data d _T based on the restored difference image data d _S obtained using the i-th combination is represented by the symbol d _T [i]. The difference DIFF _AT between the input image data d _A and the reconstructed image data d _T [i] is represented by the symbol DIFF _AT [i]. In this way, n reconstructed images (n pieces of data d _T [1] to d _T [n]) corresponding to the first to n-th combinations for each macroblock with respect to one input image. And n dissimilarities DIFF _AT [1] to DIFF _AT [n] are generated and derived.

The evaluation value deriving unit 121 derives an evaluation value EV _AT based on the degree of difference DIFF _AT and the code amount VOL _Q for each combination. The evaluation value EV _AT based on the degree of difference DIFF _AT [i] and the code amount VOL _Q [i] corresponding to the i-th combination is represented by the symbol EV _AT [i]. Code amount VOL _Q corresponding to the combination of the i [i], the coded data d _Q corresponding to the combination of the i (hereinafter, also referred to as encoded data d _Q [i]) is the amount of codes VOL _Q, a code amount VOL _Q when coding parameter 141 is added to the encoded data d _Q corresponds to the combination of the i. The encoded data d _Q [i] is obtained by encoding the data d _P into the inverse quantization parameter (inverse quantization parameter) and inverse orthogonal transformation parameter (inverse transformation format parameter) corresponding to the i-th combination. It can be obtained by adding additional information 140 including

Here, for the sake of concrete description, it is assumed that the difference DIFF _AT [i] has a larger value as the input image and the reconstructed image based on the data d _T [i] differ from each other. For example, an SSD (Sum of Squared Difference) or SAD (Sum of Absolute Difference) between the data d _A and d _T [i] can be obtained as the difference DIFF _AT [i]. The dissimilarity DIFF _AT [i] evaluation value if increased EV _AT [i] is also increased and the code amount VOL _Q [i] is them if the evaluation value EV _AT increases [i] so also increases, the evaluation value Assume that EV _AT [i] is required. For example, the evaluation value EV _AT [i] can be derived from the following equation (2).
EV _AT [i] = k _DIFF × DIFF _AT [i] + k _VOL × VOL _Q [i] (2)

Combination selection unit 122, a combination corresponding to the minimum evaluation value among the evaluation values _{EV AT [1] ~EV AT [} n], it is selected as a particular combination from among the combinations of the first to n. The encoder 110 may encoded data d _Q based on the encoded parameters 141 corresponding to the particular combination and data d _P, outputs and recorded on the recording medium 32 through the buffer memory 31. For example, the evaluation value EV _AT [1] of the ~EV _AT [n], if the evaluation value EV _AT [i] is the smallest, the combination of the i is selected as the specific combination, the encoded data d _Q [i ] Is recorded on the recording medium 32 via the buffer memory 31. Note that if the change is input images and / or predicted image, also changes the specific combination, if the change is a particular combination, also encoded data d _Q recorded to be generated recording medium 32 by the encoding unit 115 changes To do. Therefore, special encoder 110, although not to perform orthogonal transformation and quantization fact, and generates encoded data d _Q corresponding to the input image and the prediction image, and said.

[Comparison of basic encoder and special encoder]
As described above, the data d _P supplied to the encoding unit 115 is not data obtained by actually performing orthogonal transform and quantization in the special encoder 110, but data assumed as an example of quantized transform data. It is an example. Accordingly, with respect to a certain input image, the data d _P may differ greatly from the data d _E obtained from the quantizing unit 14 of FIG. 1, may be approximated, or may coincide. However, under certain circumstances, data d _P provides better results than data d _E. This will be described.

For simplicity of explanation, it is assumed that an image in one target macroblock is a flat image and an average luminance value of the image in the target macroblock is a value of image data. 5A and 5B corresponding to the encoders 10 and 110 are referred to. Assume that the data d _A of the target macroblock in the input image is “2” and the data d _B of the target macroblock in the predicted image is “1”. In this case, the data d _C of the target macroblock is “1” (see FIG. 5A). In the basic encoder 10, it is assumed that “2” data d _C ′ is obtained from the inverse orthogonal transform unit 17 (see FIG. 1) for the target macroblock. In this case, the basic encoder 10 obtains “3” data d _A ′ for the target macroblock, and as a result, the dissimilarity DIFF is “1”.

On the other hand, in the special encoder 110, a plurality of restored difference image data d _S corresponding to a plurality of combinations is obtained. In FIG. 5B, it is assumed that the four data d _S corresponding to the first to fourth combinations are “0”, “1”, “2”, and “3”, respectively. . Then, with respect to the macro block of interest, the special encoder 110 corresponds to the first to fourth combinations, and data d _T (d _T [1] to “1”, “2”, “3”, and “4”). d _T [4]) is obtained, and the difference DIFF _AT (DIFF _AT [1] to DIFF _AT [4]) of “1”, “0”, “1” and “2” is obtained as a result. Of these four dissimilarities DIFF _AT , the dissimilarity of “0” corresponding to the second combination is the smallest. For simplicity of explanation, it is assumed that the generated code amount is the same between the first to fourth combinations. Then, the second combination is selected as the specific combination, the encoded data d _Q is generated by the second coding parameters 141 according to the combination.

The difference degree of “0” (DIFF _AT [2]) corresponding to the second combination is smaller than that in the basic encoder 10, and the comparison between the encoders 10 and 110 is performed by the encoding unit (15, 115). Assuming that the supplied quantized transform data (d _E , d _P ) is the same between the encoders 10 and 110, the special encoder 110 has a smaller difference than the basic encoder 10 without increasing the code amount. (Encoded data corresponding to a smaller difference can be generated). If encoded data corresponding to a smaller degree of difference can be generated at the time of encoding, a reconstructed image having a smaller deviation from the input image can be generated at the time of decoding. That is, the image quality of the compressed image (recorded image, reproduced image) can be improved.

If the quantization scale is reduced in the basic encoder 10 or the conventional encoder, the quantization error is reduced and the deviation between the input image and the reconstructed image is usually reduced, but the code amount is increased instead. On the other hand, the special encoder 110, for encoding data d _Q from a preset data d _P is generated, affecting the code amount VOL _Q be the value of the quantization scale number is little or no (i.e., (There is little or no increase in the code amount VOL _Q associated with reducing the deviation). That is, according to the special encoder 110, it is possible to improve the image quality of the compressed image (recorded image, reproduced image) without substantially increasing the code amount. In addition, the special encoder 110 can omit the orthogonal transform and quantization processing.

Although it depends on the method of generating the coded data d _Q, as described above, the effect of the change in parameters has on the code amount VOL _Q is or not insignificant. Thus, the combination selecting unit 122, the code amount VOL _Q [1] Regardless ~VOL _Q [n], a combination corresponding to the smallest degree of difference among the dissimilarity _{DIFF AT [1] ~DIFF AT [} n], The specific combination may be selected from the first to nth combinations. In order to realize this, “k _DIFF > 0” and “k _VOL = 0” may be set in the above equation (2). In this case for example, of the difference degree _{DIFF AT [1] ~DIFF AT [} n], if the dissimilarity DIFF _AT [i] is the smallest, the combination of the i is selected as the specific combination, the combination of the i Corresponding encoded data d _Q [i] is recorded on the recording medium 32 via the buffer memory 31.

Again, the quantized transform data d _P in the special encoder 110 is not data obtained by actually performing orthogonal transform and quantization on the difference image data between the input image and the predicted image. However, if the reconstructed differential image data d _S obtained from the retained data d _P in the memory 130 results in a reconstructed image having a small difference from the input image, the retained data d _P If the data d _P well representing the difference between the input image and the predicted image is encoded and recorded, the code based on the data d _P at the time of decoding can be said. it is possible to obtain a good reconstruction image from data d _Q (i.e., non-implementation of the orthogonal transform and quantization is not a problem).

In particular, for example, when the brightness of the shooting region suddenly changes during the shooting period of the moving image MI, the difference DIFF by the basic encoder 10 may be considerably increased around the sudden change. DIFF> DIFF _AT [i] ”or“ EV> EV _AT [i] ”is often satisfied (that is, the situation shown in FIGS. 5A and 5B often occurs).

<< Second Embodiment >>
However, the encoding of the special encoder 110, in order to rely on holding data d _P a predetermined, not a panacea. Therefore, both the encoders 10 and 110 may be provided in the imaging apparatus 1 so that the pros / cons of both encoders are complemented. FIG. 6 is a block diagram of a part related to the encoder in the imaging apparatus 1. The imaging apparatus 1 according to the second embodiment includes an output selection unit 35 in addition to the encoders 10 and 110, the buffer memory 31, and the recording medium 32.

The output selection unit 35 is inserted between the encoders 10 and 110 and the buffer memory 31 and the recording medium 32. Thus, the basic encoder 10 supplies the encoded data d _Q to the output selection unit 35, a special encoder 110 supplies the output selection unit 35 the coded data d _Q corresponding to the specific combination. If the specific combination is the i-th combination, the encoded data d _Q [i] is supplied to the output selection unit 35.

The output selection unit 35 performs the following output selection process for each input image and for each macroblock. In the output selection process, the output selection unit 35 uses the minimum evaluation value among the evaluation values EV _AT [1] to EV _AT [n] derived by the special encoder 110 as the evaluation value derived by the basic encoder 10. Compare with EV. Now, the minimum evaluation value among the evaluation values _{EV AT [1] ~EV AT [} n] is assumed to be an evaluation value EV _AT [i]. In this case, in the output selection process, the output selection unit 35 selects the encoded data d _F supplied from the basic encoder 10 as the output encoded data when the inequality “EV <EV _AT [i]” is satisfied, When the inequality “EV> EV _AT [i]” is established, the encoded data d _Q (specifically, d _Q [i]) supplied from the special encoder 110 is selected as output encoded data, and the output encoded data is buffered. The data is output and recorded on the recording medium 32 via the memory 31. When “EV = EV _AT [i]” is established, the encoded data d _F or d _Q is selected as the output encoded data.

In order to make the comparison of the evaluation values meaningful, it is preferable to share the evaluation value derivation method based on the degree of difference and the code amount between the encoders 10 and 110. For example, the evaluation values EV and EV _AT [1] to EV _AT [n] may be calculated using the above formulas (1) and (2). At this time, as described in the first embodiment, “k _DIFF > 0” and “k _VOL = 0” may be set. In other words, the output selection unit 35 compares the difference degree DIFF _AT (that is, the minimum value of the difference degrees DIFF _AT [1] to DIFF _AT [n]) corresponding to the specific combination with the difference degree DIFF, and the former is the latter. if greater than while selecting coded data d _F as output encoded data, the former may be selected as the output encoded data if less encoded data d _Q than the latter.

  According to the present embodiment, it is possible to select and output encoded data that is determined to be better in terms of the degree of difference and / or the amount of code.

<< Third Embodiment >>
A third embodiment of the present invention will be described. In the third embodiment, the configuration of the imaging apparatus 1 will be described. FIG. 7 is a schematic configuration block diagram of the imaging apparatus 1. The imaging apparatus 1 includes the above-described recording medium 32 and each part referred to by reference numerals 201 to 206.

  The imaging unit 201 includes a solid-state imaging device such as an optical system formed by a lens and a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Get image data. This subject image is handled as an input image. However, the image data obtained by performing predetermined signal processing (noise reduction processing, demosaicing processing, etc.) on the image data acquired by the imaging unit 201 may be image data of the input image.

  Image data of the input image is supplied to an image processing unit (image processing apparatus) 202. The image processing unit 202 includes the special encoder 110, or includes the basic encoder 10, the special encoder 110, and the output selection unit 35 (see FIGS. 4 and 6). An internal memory 203 formed of SDRAM (Synchronous Dynamic Random Access Memory) or the like includes the buffer memory 31 and temporarily stores arbitrary data handled in the imaging apparatus 1. The recording medium 32 is formed by a nonvolatile memory such as a semiconductor memory or a magnetic disk. The display unit 204 is formed of a liquid crystal display panel or the like and displays an arbitrary image. The operation unit 205 includes a mechanical operation member such as a button and / or a touch panel, and accepts various operations and instructions from the user. The main control unit 206 comprehensively controls the operation of each part in the imaging apparatus 1 while following the operation content on the operation unit 205.

<< Deformation, etc. >>
The embodiment of the present invention can be appropriately modified in various ways within the scope of the technical idea shown in the claims. The above embodiment is merely an example of the embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the above embodiment. The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. As annotation items applicable to the above-described embodiment, notes 1 to 6 are described below. The contents described in each comment can be arbitrarily combined as long as there is no contradiction.

[Note 1]
In special encoder 110, the quantized transform data d _P supplied to the encoding unit 115 and the inverse quantization unit 116 may be a data dependent on the input image and predicted image. That is, for example, the quantized transform data d _P supplied from the memory 130 to the encoding unit 115 and the inverse quantization unit 116 may be changed according to the difference image data between the input image and the predicted image. This is expected to reduce the difference degree DIFF _AT [i]. In order to realize this, a look-up table (hereinafter referred to as “data d _P”) that holds a plurality of data and selectively outputs data corresponding to difference image data between the input image and the predicted image as data d _P from the plurality of data. The memory 130 may be formed in the first LUT).

[Note 2]
If the contents as well as inverse quantization and parameters of the inverse orthogonal transformation of the data d _P are determined, it determined the contents of the restored difference image data d _S. Since the special encoder 110 knows the contents of the data d _P in advance, the inverse quantization unit 116 and the inverse orthogonal transform unit 117 shown in FIG. 4 are formed by a lookup table (hereinafter referred to as a second LUT). May be. The 2LUT receives input data d _P, and outputs the data d _S according to the data d _P. Similarly, the encoding unit 115 may be formed by a lookup table (hereinafter referred to as a third LUT). The 3LUT receives input data d _P, and outputs the data d _Q according to the data d _P (coded data d _P). One LUT obtained by integrating the first to third LUTs described above may be provided in the special encoder 110. The one LUT functions as the memory 130, the encoding unit 115, the inverse quantization unit 116, and the inverse orthogonal transform unit 117, and data d _P , d _Q and d corresponding to the difference image data between the input image and the predicted image. _S may be output.

[Note 3]
Conversion format candidates of the aforementioned first to n _B, unless conversion format candidates of the first through n _B are mutually different conversion formats, is optional. The phrase “a plurality of transform formats are different from each other” includes that the size of a block to which orthogonal transform and inverse orthogonal transform are applied differs between the plurality of transform formats. For example, MPEG-4 AVC / H. In H.264, a plurality of conversion modes including a first conversion mode in which DCT is performed in units of blocks composed of (4 × 4) pixels and a second conversion mode in which DCT is performed in units of blocks composed of (8 × 8) pixels. Are prepared, and DCT and inverse DCT corresponding to DCT can be executed by selectively using any of the plurality of conversion modes. Conversion format according converted format and a second conversion mode according to the first conversion mode, it can be a two of conversion format candidates of the first to n _B.

[Note 4]
However, n _B may be 1. In this case, in the special encoder 110, the transform format of the inverse orthogonal transform is fixed to one transform format, and the number of combinations of inverse quantization and inverse orthogonal transform is (n _A × n _B ) = n _A ≧ 2. .

[Note 5]
Although not particularly considered in the above description, image data of each image such as an input image is formed from a plurality of types of signals (for example, luminance signal Y and color difference signals U and V), and includes an image processing unit including a special encoder 110. Each process in 202 can be executed for each type of signal.

[Note 6]
The target device that is the imaging device 1 or the image processing unit (image processing device) 202 can be configured by hardware such as an integrated circuit or a combination of hardware and software. Arbitrary specific functions that are all or part of the functions realized in the target device are described as a program, the program is stored in a flash memory that can be mounted on the target device, and the program is executed by the program execution device. The specific function may be realized by executing on a microcomputer (for example, a microcomputer that can be mounted on the target device). The program can be stored and fixed on an arbitrary recording medium. The recording medium for storing and fixing the program may be mounted or connected to a device (such as a server device) different from the target device.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Basic encoder 110 Special encoder 15, 115 Encoding part 20, 120 Dissimilarity derivation part 122 Combination selection part 35 Output selection part

Claims (6)

In an image processing apparatus including an encoder that generates encoded data corresponding to an input image and a predicted image of the input image,
The encoder is
Memory that stores predetermined data as data corresponding to data obtained by orthogonal transformation and quantization of difference image data between the input image and the predicted image;
A reconstructed image generating unit that generates a reconstructed image of the input image from the sum of the restored difference image data obtained by inverse quantization and inverse orthogonal transform of the retained data and the image data of the predicted image;
A degree-of-difference deriving unit for deriving a degree of difference between the input image and the reconstructed image;
An encoding unit that generates the encoded data, and using a plurality of combinations as a combination of the inverse quantization parameter and the inverse orthogonal transform parameter, a plurality of re-transmissions corresponding to the plurality of combinations. Generate and derive a construction image and multiple dissimilarities,
The encoder further includes a combination selection unit that selects a specific combination from the plurality of combinations based on the plurality of differences.
The image processing apparatus, wherein the encoding unit generates the encoded data based on a parameter corresponding to the specific combination and the retained data. The combination selection unit obtains a plurality of evaluation values by deriving an evaluation value based on the degree of difference and the code amount for each combination, and selects the specific combination based on the plurality of evaluation values,
The plurality of combinations include first to nth combinations, and the code amount corresponding to the i-th combination includes additional information including a parameter corresponding to the i-th combination in the data obtained by encoding the retained data. Code amount of the added data (n is an integer of 2 or more, i is a natural number of n or less)
The image processing apparatus according to claim 1. The image processing apparatus according to claim 1, wherein the combination selection unit selects a combination corresponding to a minimum difference among the plurality of differences as the specific combination. An orthogonal transform / quantization unit that generates quantized transform data by orthogonal transform and quantization of the difference image data, a second restored difference image obtained by inverse quantization and inverse orthogonal transform of the quantized transform data A second reconstructed image generator for generating a second reconstructed image of the input image from the sum of the data and the image data of the predicted image, and deriving a second difference between the input image and the second reconstructed image A second encoder having a second difference degree derivation unit, and a second encoding unit that generates second encoded data from the quantized transform data;
Based on a first evaluation value corresponding to the specific combination of the plurality of evaluation values, and a second evaluation value based on the second difference and a second code amount, the encoded data by the encoder or the An output selection unit that selects the second encoded data by the second encoder as output encoded data;
The second code amount is a code amount of data obtained by adding additional information including the orthogonal transform and quantization parameters in the second encoder to data obtained by encoding the quantized transform data. The image processing apparatus according to claim 2. An orthogonal transform / quantization unit that generates quantized transform data by orthogonal transform and quantization of the difference image data, a second restored difference image obtained by inverse quantization and inverse orthogonal transform of the quantized transform data A second reconstructed image generator for generating a second reconstructed image of the input image from the sum of the data and the image data of the predicted image, and deriving a second difference between the input image and the second reconstructed image A second encoder having a second difference degree derivation unit, and a second encoding unit that generates second encoded data from the quantized transform data;
Based on the first dissimilarity corresponding to the specific combination and the second dissimilarity among the plurality of dissimilarities, the encoded data by the encoder or the second encoded data by the second encoder is output. The image processing apparatus according to claim 3, further comprising an output selection unit that selects as encoded data. In an imaging device that acquires an input image by shooting,
An imaging apparatus comprising the image processing apparatus according to claim 1, which receives supply of image data of the input image.

Images