JP2011114369A - Image encoding and decoding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding and decoding apparatus, capable of performing image processings and frequency conversion, taking into consideration the frequency characteristics of a difference image between a local decoded image and an original image, of compressing the difference image by a higher encoding efficiency. <P>SOLUTION: There is generated a difference image between an input image and a local decoded image generated for motion compensation prediction by a first encoding unit; image processing or frequency conversion is applied to the difference image, by using spatial frequency information of the difference image; then, the difference image is encoded; a decoded image where encoded data of the input image is decoded is added to a decoded image, where the difference image is decoded; and switching between the decoded image of the input image and the added image is conducted and output as a reproduced image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image encoding / decoding device for storing a moving image such as a video camera or a television broadcast in a recording medium and outputting the stored image information at an image quality or a transmission rate according to the application.

  There are video recording devices such as DVD / hard disk recorders that store images and television images captured by a camera on a recording medium such as an optical or magnetic disk or semiconductor memory, and read and view them when requested. . In these apparatuses, when a moving image is stored in a recording medium, MPEG-2 or H.264 is stored. In general, compression is performed using an encoding method such as H.264. MPEG-2 and H.264 H.264 is classified as a system called lossy encoding, and it is possible to greatly reduce the data size of moving images by thinning out information that is not easily noticeable from an image using human visual characteristics. Make it possible.

However, since many video recording apparatuses such as DVD / hard disk recorders record images for a long time on a recording medium having a limited capacity, image compression is performed by increasing the compression rate until a visual deterioration is detected. Often to do.
In general, in lossy encoding, the compression rate and image quality are in a trade-off relationship, and image quality degradation is amplified as the compression rate is improved.

  When storing moving images on a recording medium, it is desirable to be able to record for a long time without any degradation in image quality. Increasing it leads to amplifying image quality degradation. Therefore, in a video recording apparatus such as a DVD / hard disk recorder, the compression rate is determined under the selection of whether the recording time is important or the image quality is important, and image encoding is performed.

  Whether the recording time or the image quality is emphasized is not determined only by the configuration and use of the apparatus, but changes every moment depending on the remaining amount of the recording medium, the user's interest in the recorded image, and the like. For example, if the remaining amount of the recording medium is large, it is possible to record for a long time without increasing the compression rate. In the case where there are few, there may be a case where it is desired to secure a long recording time even at the expense of image quality.

  Also, users are highly interested in the recorded video, and it is desirable to save images that are viewed repeatedly in a high quality state. It is not desirable to store a high quality image until the remaining amount of the recording medium is compressed.

  Whether the image quality or the recording time should be emphasized in this way changes with the state of the recording medium and the passage of time, and the image can be flexibly changed according to such changes. If it can be recorded, long-time recording can be realized while maintaining the image quality that the user is satisfied with.

  The relationship between the image quality and the bit amount can be changed by re-encoding the once recorded image at a different compression rate. However, for re-encoding, it is necessary to operate part or all of the apparatus for a certain period of time. In addition to a large amount of power consumption, there is a case where the time for re-encoding cannot be secured in the case of an apparatus that is kept operating for a long time.

  As an apparatus that can change the relationship between the image quality and the bit amount of a recorded image without re-encoding, there is an image encoded recording / reading apparatus disclosed in Patent Document 1. This apparatus includes a plurality of encoding units. The first encoding unit encodes an original image, and the second encoding unit includes a local decoded image generated by the first encoding unit and A difference image from the original image is encoded. High-quality image reproduction is realized by simultaneously decoding the two bit streams generated by the first encoding unit and the second encoding unit and outputting the added image of both. The bit stream generated by the second encoding unit (hereinafter referred to as a difference stream) can be erased. In this case, only the bit stream generated by the first encoding unit (hereinafter referred to as a basic stream) is used. It is possible to decrypt and reproduce.

  In the image encoded recording / reading apparatus disclosed in Patent Document 1, the basic stream generated by the first encoding unit is left while the difference stream generated by the second encoding unit is used as the remaining amount of the recording medium. By erasing according to the elapsed time, the relationship between the image quality of the recorded image and the bit amount can be changed without re-encoding. The second encoding unit determines whether or not it is a gazing point, and in the image area other than the gazing point, means for setting the value of the difference image to “0”, means for removing noise components, and degradation on the basic stream Means for determining the bit amount to be allocated to each frame of the difference stream according to the size of the difference stream, and is devised so that a difference stream with a low bit amount and a high image quality improvement effect can be generated.

International Publication WO06 / 100820

In the apparatus of Patent Document 1, since the second encoding unit performs compression without considering the characteristics of the difference image, whether the difference image can be compressed with low degradation and a low bit amount is determined by the second encoding unit. Depends on the compression capacity.
MPEG-2 and H.264 In many encoding methods such as H.264, when the coefficient distribution when the image is orthogonally transformed is biased toward the lower side of the spatial frequency, high compression capability can be exhibited. There is a tendency that the distribution of the coefficient of the above is dispersed mainly in the middle and high frequency components, and it is difficult to maintain high coding efficiency for such an image.
For this reason, even if the bit amount reduction means as described above is provided, the bit amount of the difference stream becomes larger than necessary, the difference stream itself presses down the remaining amount of the recording medium, and the difference stream is held in the recording medium. It sometimes causes a situation that is difficult to keep.
In addition, as a result of intentionally reducing the bit amount of the difference stream, encoding difference is caused in the difference image itself, and the generated added image may be deteriorated in image quality rather than the decoded image of the basic stream alone.

  The present invention has been made to solve the above-described problems. By performing image processing and frequency conversion in consideration of the frequency characteristics of the difference image between the locally decoded image and the original image, the difference image is increased. It is an object of the present invention to obtain an image encoding / decoding device capable of compression with encoding efficiency.

  The image encoding / decoding device according to the present invention includes a first encoding unit that encodes an input image, a difference between the input image and a locally decoded image generated by the first encoding unit for motion compensation prediction. A difference processing unit that generates an image, an image conversion unit that performs image processing or frequency conversion of the difference image using spatial frequency information of the difference image, and a second that encodes the difference image converted by the image conversion unit An encoding unit; a first decoding unit that decodes a bit stream of encoded data output from the first encoding unit; and a first decoding unit that decodes a bit stream of a difference image output from the second encoding unit. 2 decoding units, an addition processing unit for adding the decoded images decoded by the first and second decoding units to generate an added image, a decoded image decoded by the first decoding unit, and an addition processing unit Switch to the added image obtained in In which and a selection unit for outputting an image.

According to the present invention, a difference image between an input image and a locally decoded image generated by the first encoding unit for motion compensation prediction is generated, and an image of the difference image is generated using the spatial frequency information of the difference image. After processing or frequency conversion, this difference image is encoded, and the decoded image obtained by decoding the encoded data of the input image and the decoded image obtained by decoding the difference image are added, and the decoded image of the input image and the added image are added. And are output as a playback image.
With this configuration, it is possible to generate a difference stream with a high image quality improvement effect with a small amount of bits, and recording and playback of a moving image bit stream for a long time on a recording medium with a limited capacity There is an effect that it is possible to easily obtain a high-performance image encoding / decoding device that achieves both high image quality.
In addition, the difference image between the local decoded image and the original image generated by the first encoding unit can be compressed to a small bit amount using the spatial frequency information of the difference image, and the image quality is longer and longer than before. Time image recording becomes possible.
Furthermore, since it is possible to realize the internal configuration of the encoding unit and decoding unit without any modification, it is possible to easily construct a device using general-purpose encoding / decoding means. Become.

It is a block diagram which shows the structure of the image coding / decoding apparatus by Embodiment 1 of this invention. 2 is a block diagram illustrating a configuration of an image conversion unit according to Embodiment 1. FIG. It is a figure which shows an example of the average value in a picture of a DHT coefficient. It is a figure which shows an example of image processing of rotation and reduction. It is a figure which shows the change of generation | occurrence | production of an orthogonal transformation coefficient when rotating an image. 6 is a block diagram illustrating a configuration of an image conversion unit according to Embodiment 2. FIG. FIG. 10 is a block diagram illustrating a configuration of an image conversion unit according to Embodiment 3. FIG. 10 is a block diagram illustrating a configuration of an image conversion unit according to a fourth embodiment. FIG. 10 is a block diagram illustrating a configuration of an image conversion unit according to a fifth embodiment. It is a figure which shows an example of the component replacement | exchange of an orthogonal transformation coefficient. It is a block diagram which shows the structure of the image coding / decoding apparatus by Embodiment 6 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of an image encoding / decoding apparatus according to Embodiment 1 of the present invention. 1, the image encoding / decoding apparatus according to Embodiment 1 includes encoding units 1 and 4, a difference unit 2, an image conversion unit 3, a recording medium 5, decoding units 6 and 7, an image restoration unit 8, and an adder 9. Is provided. The main body of the image encoding / decoding apparatus according to the present invention is an image conversion unit 3 and an encoding unit 4, a decoding unit 7 and an image restoration unit 8 surrounded by a broken line in FIG. This is positioning of a peripheral device for operating the decoding / decoding device. Hereinafter, description will be made including peripheral devices.

  When the image data is input, the encoding unit (first encoding unit) 1 receives MPEG-2 or H.264. The image data is compression-encoded based on an encoding method such as H.264 and stored in a recording medium in the form of a bit stream (hereinafter referred to as a basic stream). At the same time, the encoding unit 1 outputs a locally decoded image generated for motion compensation prediction. If the orthogonal transform coefficient calculated in the encoding process can be output, the data is also output to the image conversion unit 3.

A differentiator (difference processing unit) 2 generates a difference image between an image input from the outside of the apparatus (hereinafter referred to as an input image) and a locally decoded image.
The image conversion unit 3 performs image processing and frequency conversion on the difference image, and details of the processing will be described later. The image processing and frequency conversion methods performed by the image conversion unit 3 are stored in the recording medium 5 as conversion information.
The encoding unit (second encoding unit) 4 compresses and encodes the differential image subjected to image processing and frequency conversion and stores it in the recording medium 5 in the form of a bit stream (hereinafter referred to as differential stream).

  In a state where the basic stream, the difference stream, and the conversion information are stored in the recording medium 5, the decoding unit 6 decodes the basic stream and generates a decoded image. At the same time, the decoding unit 7 decodes the difference stream and generates a decoded image. The image restoration unit 8 performs image processing and frequency conversion on the decoded image generated by the decoding unit 7. Image processing and frequency conversion performed by the image restoration unit 8 are inverse transformations of the processing of the image conversion unit 3. Inverse conversion is performed with reference to the conversion information stored in the recording medium 5.

  An adder (addition processing unit) 9 generates an addition image of the decoded image generated by the decoding unit 6 and the image processed or frequency-converted by the image restoration unit 8. The selector (selection unit) 10 selects one of the decoded image generated by the decoding unit 6 and the addition image generated by the adder 9 and outputs the selected image to the outside of the apparatus.

  The difference stream and the conversion information stored in the recording medium 5 can be deleted according to the remaining amount of the recording medium 5 and an instruction from the user. When erasing, the difference stream and the conversion information are erased as a set. When the difference stream and the conversion information are deleted, the decoding unit 7, the image restoration unit 8, and the adder 9 do not perform processing when the image is reproduced, and the decoding unit 6 decodes only the basic stream, Output from the selector 10.

The difference image generated by the differentiator 2 is a component corresponding to the image deterioration generated in the encoding unit 1, and a reproduction image equivalent to the input image can be obtained by adding the component to the decoded image of the basic stream. Can do. However, as in the first embodiment, if a difference image is also an encoding target, there is a possibility that image degradation due to encoding occurs.
In the image coding / decoding device according to the first embodiment, even when a difference image is an encoding target, the difference image can be compressed with low degradation and a low bit amount.

In FIG. 1, an image conversion unit 3 plays a role of realizing a function of compressing a difference image with low degradation and a low bit amount. Hereinafter, a configuration example of the image conversion unit 3 will be described.
FIG. 2 is a block diagram illustrating a configuration of the image conversion unit according to the first embodiment. In FIG. 2, the image conversion unit 3 includes a positive integer conversion unit 21, a block division unit 22, an orthogonal conversion unit 23, an optimum conversion determination unit 24, an image processing unit 25, and a bit width correction unit 26. The input of the image conversion unit 3 is a difference image generated by the differentiator 2 shown in FIG. 1, and the difference image is represented by positive / negative signed integer data. For example, when both the input image that is the input of the subtractor 2 and the locally decoded image are represented by integer data having a width of 8 bits, the output of the subtractor 2 is a 9-bit width difference image having a positive or negative integer value. .

The positive integer converting unit 21 converts the above-described difference image having positive and negative integer values into a positive integer. As a conversion method, a method of adding a constant offset value to each pixel data value, a method of handling a negative integer represented by a complement as an unsigned positive integer, or the like is used. The former adds, for example, an offset value of 2 to the power of (N−1) to a difference image having an N-bit width. In the latter case, for example, the binary complement value “111111111” of “−1” is read as an unsigned positive integer and treated as 511.
Here, a rounding process for reducing the bit width by 1 bit may be performed simultaneously with the conversion to a positive integer. For example, when the difference image has a 9-bit width, a bit shift operation in the right direction is performed and converted to an 8-bit width.

  The block division unit 22 divides the difference image represented by positive integer data into a block size of 8 × 8 pixels or 4 × 4 pixels. The orthogonal transform unit 23 performs orthogonal transform on each block divided by the block dividing unit 22 by DHT (Discrete Hadamard Transform), DCT (Discrete Cosine Transform), or the like. Hereinafter, a case where DHT is used for a block obtained by dividing a difference image into 8 × 8 pixels will be described as an example.

  The optimal transformation determination unit 24 calculates the statistical information by adding up the orthogonal transformation coefficients of the 8 × 8 matrix obtained by the orthogonal transformation unit 23 in units of pictures, slices, and the like. As this statistical information, for example, an in-picture average value of absolute values of orthogonal transform coefficients (DHT coefficients) as shown in FIG. 3 is obtained. The average value in the picture represents the spatial frequency characteristics of the difference image. In the example shown in FIG. 3, the value of the orthogonal transform coefficient on the diagonal line from the (0, 0) component to the (7, 7) component. It can be seen that power is concentrated on the coefficient. This means that the change in pixels is large with respect to the direction of 45 degrees diagonally of the difference image (the angle when the horizontal direction is 0 degree). The optimum transformation determination unit 24 obtains an area where such power of the orthogonal transformation coefficient is concentrated as an angle.

The angle of the region where the power of the orthogonal transform coefficient is concentrated is determined by, for example, obtaining H0, H67.5, H45, H22.5, and H90 in the following equation (1) and minimizing the angle. Here, when H0 is minimum, 0 degree, when H22.5 is minimum, 22.5 degrees, when H45 is minimum, 45 degrees, when H67.5 is minimum, 67.5 degrees, H90 is minimum In this case, the angle is 90 degrees. In the following equation (1), Q (x, y) is an (x, y) component [0 ≦ x, y ≦ 7] in the average value of orthogonal transform coefficients in a picture.
H90 = | Q (0,0) | + | Q (1,0) | + | Q (2,0) | + | Q (3,0) | + | Q (4,0) | + | Q ( 5,0) | + | Q (6,0) | + | Q (7,0) |
H67.5 = | Q (0,0) | + | Q (1,0) | + | Q (2,1) | + | Q (3,1) | + | Q (4,2) | + | Q (5,2) | + | Q (6,3) | + | Q (7,3) |
H45 = | Q (0,0) | + | Q (1,1) | + | Q (2,2) | + | Q (3,3) | + | Q (4,4) | + | Q ( 5,5) | + | Q (6,6) | + | Q (7,7) |
H22.5 = | Q (0,0) | + | Q (0,1) | + | Q (1,2) | + | Q (1,3) | + | Q (2,4) | + | Q (2,5) | + | Q (3,6) | + | Q (3,7) |
H0 = | Q (0,0) | + | Q (0,1) | + | Q (0,2) | + | Q (0,3) | + | Q (0,4) | + | Q ( 0,5) | + | Q (0,6) | + | Q (0,7) |
... (1)

In addition, the optimal transformation determination unit 24 calculates the amount of pixel change in the horizontal direction and the vertical direction in the difference image based on the average value in the picture of the orthogonal transformation coefficient. The pixel change amount is calculated according to the following formula (2), for example. In the following formula (2), when XH is greater than or equal to a specific threshold value α, it is considered that the horizontal pixel change is large. Further, when XH is less than the threshold value and XH + XM exceeds a specific threshold value β, the horizontal pixel change is regarded as moderate. If none of them is satisfied, the horizontal pixel change is considered small. Similarly, when YH is greater than or equal to a specific threshold value α, the vertical pixel change is considered large, and when YH is less than the threshold value and YH + YM exceeds a specific threshold value β, the vertical pixel change is moderate. If neither is satisfied, the pixel change in the vertical direction is considered small. Here, α and β are arbitrary integer values determined based on experimental data or the like.
XH = | Q (7,0) | + | Q (7,1) | + | Q (7,2) | + | Q (7,3) | + | Q (7,4) | + | Q ( 7,5) | + | Q (7,6) | + | Q (7,7) | + | Q (6,0) | + | Q (6,1) | + | Q (6,2) | + | Q (6,3) | + | Q (6,4) | + | Q (6,5) | + | Q (6,6) | + | Q (6,7) |
XM = | Q (5,0) | + | Q (5,1) | + | Q (5,2) | + | Q (5,3) | + | Q (5,4) | + | Q ( 5,5) | + | Q (5,6) | + | Q (5,7) | + | Q (4,0) | + | Q (4,1) | + | Q (4,2) | + | Q (4,3) | + | Q (4,4) | + | Q (4,5) | + | Q (4,6) | + | Q (4,7) |
YH = | Q (0,7) | + | Q (1,7) | + | Q (2,7) | + | Q (3,7) | + | Q (4,7) | + | Q ( 5,7) | + | Q (6,7) | + | Q (7,7) | + | Q (0,6) | + | Q (1,6) | + | Q (2,6) | + | Q (3,6) | + | Q (4,6) | + | Q (5,6) | + | Q (6,6) | + | Q (7,6) |
YM = | Q (0,5) | + | Q (1,5) | + | Q (2,5) | + | Q (3,5) | + | Q (4,5) | + | Q ( 5,5) | + | Q (6,5) | + | Q (7,5) | + | Q (0,4) | + | Q (1,4) | + | Q (2,4) | + | Q (3,4) | + | Q (4,4) | + | Q (5,4) | + | Q (6,4) | + | Q (7,4) |
... (2)

  In the above description, with respect to the orthogonal transformation coefficient obtained by dividing the difference image into 8 × 8 pixel blocks, a method for obtaining the angle of the region where the power of the orthogonal transformation coefficient is concentrated and the pixel change amount in the horizontal and vertical directions. However, the unit of block division may be an arbitrary unit such as 4 × 4 pixels or 16 × 16 pixels. Further, instead of DHT, DCT, DST (discrete sine transform) or the like may be used as orthogonal transform.

As the statistical information of the orthogonal transform coefficients, a total value, an intermediate value, a maximum value, or the like can be used instead of the average value of the absolute values of the orthogonal transform coefficients aggregated in units of pictures and slices.
Statistical information may be calculated for each unit using a unit smaller than a slice or a GOP composed of a plurality of pictures as a unit.
Note that the angle and pixel change amount calculated by the optimum conversion determination unit 24 are stored in the recording medium 5 of FIG. 1 as conversion information.

The image processing unit 25 performs rotation processing and reduction processing as shown in FIG. 4 on the difference image. As a means for rotating the image, a technique such as Hough transform is used. The rotation angle is an angle obtained by the optimum conversion determination unit 24.
FIG. 5 is a diagram illustrating changes in generation of orthogonal transform coefficients when an image is rotated. In FIG. 5, the orthogonal transform coefficient is changed to a distribution that is biased toward the low frequency component in the horizontal direction due to the rotation. If such a coefficient distribution is used, a general-purpose code such as MPEG-2 is included in the encoding unit 4. High compression performance can be obtained even when the conversion method is used. When the image processing unit 25 rotates as described above, the image restoration unit 8 shown in FIG. 1 rotates the same angle in the reverse direction, thereby restoring the original difference image.

When the image processing unit 25 reduces the difference image, a general-purpose down-conversion method is used as the means. However, the reduction magnification is determined based on the horizontal and vertical pixel change amounts obtained by the optimum conversion determination unit 24. If it is determined that the amount of pixel change in the horizontal direction is large, if the image is reduced, many of the difference image components may be lost, so if the horizontal reduction ratio is kept large and the change amount is small Reduce the reduction ratio. The same applies to the vertical direction.
By reducing the reduction magnification and converting the difference image into a reduced image, the bit amount when the difference image is encoded can be greatly reduced. The reduced image is restored to the original size by the image restoration unit 8 shown in FIG.

  The bit width correction unit 26 performs a rounding process to match the processed difference image with the corresponding bit width in the encoding unit 4. For example, when MPEG-2 is adopted as the encoding method of the encoding unit 4, only 8-bit width image data can be used, so that a difference image represented by 9 bits cannot be input as it is, and the bit width Reduction is required. In addition, the bit width may increase due to computations associated with image processing.

  The bit width correcting unit 26 is a bit width adjusting unit for dealing with such a case, and performs a bit shift operation and a clipping process on each pixel value of the processed difference image to reduce it to a predetermined bit width. .

  Note that rotation and reduction do not necessarily have to be executed, but only one or both can be executed, and whether or not to execute can be selected according to the capacity of the recording medium and user instructions. Keep it.

As described above, according to the first embodiment, the encoding unit 1 generates a difference image between the local decoded image generated for motion compensation prediction and the input image, and uses the spatial frequency information of the difference image. After performing the image processing or frequency conversion of the difference image, the difference image is encoded, the decoded image obtained by decoding the encoded data of the input image, and the decoded image obtained by decoding the difference image are added to decode the input image. The image and the added image are switched and output as a reproduced image.
With this configuration, it is possible to generate a differential stream with a high image quality improvement effect with a small amount of bits. In addition, it is possible to easily obtain a high-performance image encoding / decoding device that can record a bit stream of a moving image for a long time and achieve high image quality of a reproduced image.

Embodiment 2. FIG.
The image encoding / decoding device according to the second embodiment is the same as the configuration of FIG. 1 shown in the first embodiment, but the configuration of the image conversion unit 3 is different.
FIG. 6 is a block diagram showing a configuration of an image conversion unit according to Embodiment 2 of the present invention. 6, the image conversion unit 3 according to the second embodiment excludes the block division unit 22 and the orthogonal transformation unit 23 from the configuration of FIG. 2 described in the first embodiment. In this case, the orthogonal transform coefficient is output from the outside of the image conversion unit 3, that is, from the encoding unit 1 shown in FIG.

Note that the orthogonal transform coefficient calculated by the encoding unit 1 usually does not correspond to the difference image but corresponds to the input image (original image), and the orthogonal transform coefficient of the difference image as illustrated above Are likely to have different numerical characteristics.
However, with regard to the rotation angle and pixel change amount, it is often possible to obtain an approximate calculation result even if the orthogonal transformation coefficient of the input image is used, and the use of this configuration reduces the amount of calculation and simplifies the processing configuration. Therefore, it is also effective to adopt this configuration.

Embodiment 3 FIG.
The image encoding / decoding device according to the third embodiment is the same as the configuration of FIG. 1 shown in the first embodiment, but the configuration of the image conversion unit 3 is different.
FIG. 7 is a block diagram showing a configuration of an image conversion unit according to Embodiment 3 of the present invention. In FIG. 7, the image conversion unit 3 according to the third embodiment differs from the configuration of FIG. 2 described in the first embodiment in that the image processing unit 25 is arranged after the positive integer converting unit 21, and the image processing unit 25, an image selection unit 27 is added.

In this configuration, the image processing unit 25 simultaneously generates a plurality of processed images having different rotation angles and reduction magnifications, and the block dividing unit 22 and the orthogonal transform unit 23 perform the processing on all images generated in the image processing unit 25. Perform block division and orthogonal transformation. The optimum conversion determination unit 24 performs calculation according to, for example, the following formula (3) instead of the above formula (1) or the above formula (2). The following equation (3) calculates the sum of absolute values of orthogonal transform coefficients excluding DC components.
HH = Σ | Q (x, y) | (3)
x = 1-7, y = 1-7

The optimum conversion determination unit 24 selects a processed image that minimizes the value of the above expression (3), and stores information (rotation angle, reduction ratio) of the selected processing method in the recording medium 5.
The image selection unit 27 outputs the processed image selected by the optimum conversion determination unit 24 to the bit width correction unit 26 at the subsequent stage. The operations of the positive integer converting unit 21 and the bit width correcting unit 26 are the same as those of the image converting unit 3 described with reference to FIG. 2 in the first embodiment.

  According to the third embodiment, since an optimum processing method is selected while directly referring to the image-processed orthogonal transform coefficient, it is possible to generate a differential image that provides higher compression performance than the configuration of FIG. The evaluation criterion for the processed image does not necessarily have to be the above formula (3), but the basic determination criterion is to select an image with few frequency components in the middle and high frequencies.

Embodiment 4 FIG.
The image encoding / decoding device according to the fourth embodiment is the same as the configuration of FIG. 1 shown in the first embodiment, but the configuration of the image conversion unit 3 is different.
FIG. 8 is a block diagram showing a configuration of an image conversion unit according to Embodiment 4 of the present invention. In FIG. 8, the image conversion unit 3 of the fourth embodiment differs from the configuration of FIG. 2 shown in the first embodiment in that the block division unit 22 and the orthogonal transformation unit 23 are removed, and a frequency analysis unit 28 is added instead. ing.

The frequency analysis unit 28 analyzes the frequency characteristics in the difference image using a high-pass filter (HPF) instead of calculating the orthogonal transform coefficient. The following formula (4) is a calculation formula for the pixel change amount and the rotation angle in the horizontal direction and the vertical direction in this configuration example.
Horizontal pixel change amount = Σ | 2 × p (x, y) −p (x−1, y) −p (x + 1, y) |
x = 1 to h-2, y = 1 to v-2
Vertical pixel change amount = Σ | 2 × p (x, y) −p (x, y−1) −p (x, y + 1) |
x = 1 to h-2, y = 1 to v-2
h: horizontal picture size, v: vertical picture size, rotation angle = tan (vertical pixel change amount / horizontal pixel change amount)
... (4)

  In the fourth embodiment, since the orthogonal transform coefficient is not calculated, there is a possibility that the image processing that contributes to the improvement of the compression performance cannot be realized with high accuracy, but it is less than in the case of performing block division or orthogonal transform. It is possible to obtain the pixel change amount and the rotation angle with the calculation amount.

Embodiment 5 FIG.
The image encoding / decoding device according to the fifth embodiment is the same as the configuration of FIG. 1 shown in the first embodiment, but the configuration of the image conversion unit 3 is different.
FIG. 9 is a block diagram showing a configuration of an image conversion unit according to Embodiment 5 of the present invention. In FIG. 9, the image conversion unit 3 according to the fifth embodiment includes a positive integer conversion unit 21, a block division unit 22, an orthogonal transformation unit 23, an optimum transformation determination unit 24, a frequency substitution unit 28, an orthogonal inverse transformation unit 29, and a bit width. A correction unit 26 is provided. Here, operations of the positive integer converting unit 21, the block dividing unit 22, the orthogonal transforming unit 23, and the bit width correcting unit 26 are the same as those of the image converting unit 3 described with reference to FIG. 2 in the first embodiment.

  In this configuration example, a frequency replacement unit 28 is newly added. Here, the components of the orthogonal transform coefficient are replaced as shown in FIG. As a replacement method, for example, as in the replacement method (a) of FIG. 10, the low frequency component is obtained by giving an index number in zigzag scan order to each component of the orthogonal transform coefficient and inverting the size of the index number. And the method of inverting the relationship between the high frequency components. In addition, there is a method of shifting the frequency component from the middle range to the high range by shifting the large index number to the small index number as in the replacement method (b). Furthermore, as in the replacement method (c), there is a method of translating middle to high frequency components on an orthogonal transformation matrix. These methods are effective.

In the case of the replacement method (b), which index number and subsequent components are shifted to a low frequency range (index number 1) is determined using, for example, the average value in the picture of the absolute value of the orthogonal transform coefficient. When the average value in the picture of the absolute value of the orthogonal transform coefficient given in the format illustrated in FIG. 4 in the first embodiment is searched in zigzag scan order, when a component whose absolute value is equal to or greater than a specific threshold is found, The index numbers after the component are to be shifted.
For example, when the index number X (1 ≦ X ≦ 64) and the average absolute value in the picture is equal to or greater than the threshold value and there are N components after the index number, each component from the index number X to X + N−1 is indexed. Shift from position 1 to position N.
Here, what is actually shifted is not the average value in the picture, but the orthogonal transform coefficient in each block. After the shift, “0” is filled in for the component whose index number is N + 1 or later (index number 35 or later in FIG. 10B).

  In the case of the replacement method (c), the procedure is almost the same as that of the replacement method (b). Which index number is to be shifted is determined by the same procedure as in the replacement method (b), and on the matrix for components whose horizontal and vertical frequencies are higher than the index number that is the shift target. Move in parallel to the (0,0) component. Components that have not been assigned values after the parallel movement are filled with “0” as in the replacement method (b).

  When the replacement methods (a) to (c) are performed, the coefficient having a minute value among the orthogonal transform coefficients may be replaced with “0” in order to reduce the bit amount in encoding the difference image. It is an effective means. Note that the minute value is a value whose sum of absolute values of coefficients is, for example, “1” or less.

According to the fifth embodiment, the orthogonal transform coefficient can be forcibly biased to the low frequency side by the frequency replacement as described above. This is converted back to the image data format by the orthogonal inverse transform unit 29 and further subjected to the same bit width correction as the image transform unit 3 in FIG. 2, and then input to the coding unit 4 in FIG. The unit 4 can encode the difference image while maintaining high compression performance.
When frequency replacement is performed, information regarding the replacement method is stored in the recording medium 5, and the image restoration unit 8 performs processing to return the orthogonal exchange coefficient to the state before replacement based on the information.

In the description here, the index numbers assigned in the zigzag scan order are used as the index numbers of the orthogonal transform coefficients, but it is possible to deal with index numbers given in any way.
Also, as described in the description of the image conversion unit 3 shown in FIG. 2, instead of using the average value within a picture as the spatial frequency information, an average value obtained by summing up a unit smaller than a picture or a unit of a plurality of pictures is used. There may be. Further, instead of the average value, a maximum value or an intermediate value can be substituted.

Embodiment 6 FIG.
FIG. 11 is a block diagram showing a configuration of an image encoding / decoding apparatus according to Embodiment 6 of the present invention. In FIG. 11, the image encoding / decoding device according to the sixth embodiment removes the image conversion unit 3 and the image restoration unit 8 from the configuration described with reference to FIG. 1 in the first embodiment. The operation of each component is the same as that in the first embodiment, but the encoding unit 4 includes the image conversion unit 3 shown in any of the first to fifth embodiments, and decodes The unit 7 is characterized in that it includes the image restoration unit 8 in the first embodiment.

  The encoding unit 4 is partially different from the encoding unit 4 of the first embodiment, and the orthogonal transform coefficients are converted into a scan order reverse to the conventional one (from high frequency to low frequency components). (In order). Further, the encoding unit 4 is allowed to scan in an arbitrary order, and is the same as the frequency replacement process exemplified in FIG. 10 in the fifth embodiment by changing the scan order flexibly. An effect is obtained. Similarly, the decoding unit 7 is used corresponding to the change of the scan order of orthogonal transform.

  As described above, according to the sixth embodiment, it is necessary to apply a dedicated encoding method to the encoding unit 4 and the decoding unit 7, but the changed part is limited to only the orthogonal transform unit. Can be realized easily. Further, since the conversion information can be output in a form multiplexed with the difference stream, the data management in the recording medium 5 can be performed as compared with the case where the difference stream and the conversion information are individually generated as in the first embodiment. Is easy.

  1, 4 encoding unit, 2 subtractor, 3 image converting unit, 5 recording medium, 6, 7 decoding unit, 8 image restoring unit, 9 adder, 10 selector, 21 positive integer converting unit, 22 block dividing unit, 23 Orthogonal transformation unit, 24 Optimal transformation determination unit, 25 Image processing unit, 26 Bit width correction unit, 27 Image selection unit, 28 Frequency transformation unit, 29 Orthogonal inverse transformation unit.

Claims (13)

A first encoding unit that encodes an input image;
A difference processing unit that generates a difference image between the input image and the locally decoded image generated by the first encoding unit for motion compensation prediction;
An image conversion unit that performs image processing or frequency conversion of the difference image using spatial frequency information of the difference image;
A second encoding unit that encodes the difference image converted by the image conversion unit;
A first decoding unit that decodes a bit stream of encoded data output from the first encoding unit;
A second decoding unit for decoding the bitstream of the difference image output from the second encoding unit;
An addition processing unit that generates an added image by adding the decoded images respectively decoded by the first and second decoding units;
An image encoding / decoding device comprising: a selection unit that switches a decoded image decoded by the first decoding unit and an addition image obtained by the addition processing unit and outputs the switched image as a reproduction image.   The image conversion unit divides the difference image into blocks composed of a plurality of pixels, and uses statistical information of orthogonal transform coefficients obtained by orthogonally transforming the block as spatial frequency information of the difference image. The image encoding / decoding device according to claim 1.   The image conversion unit uses a value obtained by averaging or summing absolute values of orthogonal transform coefficients for each frequency component, or an intermediate value of orthogonal transform coefficients in each frequency component, as spatial frequency information of the difference image. The image encoding / decoding device according to claim 2.   The image encoding / decoding apparatus according to any one of claims 1 to 3, wherein the image conversion unit rotates the difference image.   The image encoding / decoding apparatus according to claim 4, wherein the image conversion unit determines a rotation angle in rotation processing of the difference image using spatial frequency information of the difference image.   The image encoding / decoding apparatus according to claim 1, wherein the image conversion unit reduces the difference image.   The image encoding / decoding apparatus according to claim 6, wherein the image conversion unit determines a reduction ratio in the reduction processing of the difference image using a pixel change amount calculated from the spatial frequency information of the difference image. . The image conversion unit
A block dividing unit that divides the difference image into blocks composed of a plurality of pixels;
An orthogonal transformation unit that orthogonally transforms the blocks divided by the block division unit;
A frequency replacement unit that replaces a specific frequency component in the orthogonal transform coefficient with another frequency component;
The orthogonal inverse transformation part which performs an inverse orthogonal transformation with respect to the orthogonal transformation coefficient by which the frequency component was substituted by the said frequency substitution part was provided, The any one of Claims 1-7 characterized by the above-mentioned. Image encoding / decoding apparatus.   The image coding / decoding apparatus according to claim 8, wherein the frequency replacement unit replaces the frequency component by inverting the size of the index number assigned to the orthogonal transform coefficient.   The frequency replacement unit shifts N components from a specific index number obtained from the spatial frequency information of the difference image, out of the index numbers assigned to the orthogonal transform coefficients, from an index number 1 to an index number N position. 9. The image coding / decoding apparatus according to claim 8, wherein the frequency component is replaced by the image coding / decoding apparatus.   The frequency replacement unit horizontally replaces a component having a frequency equal to or higher than a specific frequency obtained from the spatial frequency information of the difference image to a position of a (0, 0) component of an orthogonal transform coefficient, and replaces the frequency component. The image encoding / decoding device according to claim 8.   The image encoding according to any one of claims 1 to 11, wherein the second encoding unit uses an encoding method capable of encoding orthogonal transform coefficients in an arbitrary order. Decoding device.   13. The image encoding / decoding apparatus according to claim 12, wherein the second encoding unit uses an encoding method capable of encoding orthogonal transform coefficients in order from a high frequency to a low frequency.

Images