JP2004248124A - Hierarchical image encoder and hierarchical image decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device capable of improving the subjective quality of a decoded image by performing encoding corresponding to an image characteristic in hierarchical image encoding/decoding. <P>SOLUTION: A first encoder 11 of a hierarchical image encoder 1 encodes an original image signal and outputs the base layer encoded data 12. A first decoder 13 decodes the base layer encoded data 12 to generate a base layer decoded image signal 14, and a subtractor 15 subtracts the base layer decoded image signal 14 from the original image signal to generate a difference image signal 16. A second encoder 17 divides each small area of the differential image signal 16 into classes according to the predefined reference, an encoding priority is imparted to a classified small area, an encoding parameter is determined in accordance with an encoding priority to perform encoding, and enhancement layer encoded data 18 are outputted. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image encoding technology and an image decoding technology, and in particular, to a hierarchical image encoding for improving the subjective quality of a decoded image by performing encoding according to the characteristics of the image with a simple configuration. The present invention relates to an apparatus and a hierarchical image decoding apparatus.
[0002]
[Prior art]
As a method of encoding an image efficiently, there is a hierarchical image encoding method. In this method, a first encoder and a second encoder are prepared, and the encoded data generated for each small area by the first encoder is decoded, so that base layer decoding of an image is performed. By obtaining an image and using both the encoded data generated by the first encoder and the encoded data generated for each small area by the second encoder, a decoded image with higher image quality can be obtained. It is a method that is done.
[0003]
As a method of realizing such hierarchical coding, a method of coding a difference between an original image and a decoded image of coded data obtained as a result of the first coding by a second coder has been considered. .
[0004]
At this time, in the conventional hierarchical image encoding method, when the second encoder encodes the differential image signal, the amount of generated code in each small area is controlled to be substantially constant. There are many. FIG. 6 shows a configuration example of this method.
[0005]
In the hierarchical image encoding device 7 shown in FIG. 6, an original image signal is encoded by a first encoder 71, and base layer encoded data 72 is generated. Further, the first decoder 73 decodes the base layer encoded data 72 to generate a base layer decoded image signal 74, and the subtractor 75 subtracts the base layer decoded image signal 74 from the original image signal to obtain a difference image signal 76 is generated. The difference image signal 76 is subjected to variable-length encoding such as Huffman encoding in a second encoder 77, so that enhancement layer encoded data 78 is generated. At this time, the difference image signal 76 is divided into small areas of an appropriate size and encoded until the code amount allocated to each small area is reached.
[0006]
As a document describing the above-described hierarchical image coding method, there is, for example, Patent Document 1 below. According to the technique described in Patent Document 1, encoded data is given a hierarchy, and only a selected area is reproduced with various image qualities from a part of the encoded data, or a low-quality image is reproduced from a part of the encoded data. Or play it.
[0007]
Also, although not hierarchical image coding, an image coding apparatus for coding an input image signal by changing a coding control parameter in accordance with a class identification signal for each region of an image is disclosed in Japanese Patent Application Laid-Open No. H10-163,086. It is described in. In this technique, an edge or the like of an input image is detected and divided into regions, each region is classified into classes according to a predetermined standard, and a class identification signal is output for each region to encode an image signal according to the class identification signal. It is carried out. However, the image coding apparatus of Patent Document 2 performs coding by selectively switching any of a plurality of coding methods according to a class identification signal of each area. It does not perform hierarchical image encoding such that the difference signal between the decoded image signal whose output has been decoded and the original image signal is encoded by the second encoder. That is, it does not output the base layer encoded data and the enhancement layer encoded data as a result of the hierarchical image encoding.
[0008]
The MPEG-4 encoding method conventionally used as an image encoding method is described in Non-Patent Document 1 below.
[0009]
[Patent Document 1]
JP 2002-44671 A [Patent Document 2]
JP 2001-45494 A [Non-Patent Document 1]
ISO / IEC 14496-2 Coding of Audio-Visual Objects, Part-2
[0010]
[Problems to be solved by the invention]
In general, the amount of information of a differential image signal in the above-described hierarchical image coding is large in a region where the original image signal is complex and small in a region where the original image signal is simple. Depends on. In the method according to the related art, since the code amount is not allocated according to the properties of the original image or the base layer decoded image area, in order to improve the subjective quality, a larger code amount is required for the enhancement layer coded data. Will be needed.
[0011]
SUMMARY OF THE INVENTION It is an object of the present invention to provide a hierarchical image encoding / decoding device that solves the above-described problem of increasing the required code amount of an enhancement layer by a simple process and configuration.
[0012]
[Means for Solving the Problems]
The present invention employs the following means to solve the above problems. One means is to divide a digital original image signal into a plurality of small areas in a first encoder, perform encoding processing for each of the small areas, generate base layer encoded data, and generate encoded data. And generating a difference image signal from the base layer decoded image signal obtained by decoding the image signal and the original image signal. The second encoder divides the difference image signal into a plurality of small regions corresponding to the original image signal small regions. A hierarchical image encoding apparatus that performs encoding processing for each differential image signal small region to generate enhancement layer encoded data, wherein the second encoder encodes the difference image signal according to a predetermined reference. A class classification unit that classifies areas and generates a class identification number indicating the class of each small area, and an order in which coding priority is assigned to each of the classified small areas And attaching portion, and a control unit determining a coding parameter according to an applied coded priority, a hierarchical image coding apparatus, characterized in that it comprises.
[0013]
When adopting this configuration, the above-described class classification unit may have a function of performing classification based on activity in a small region of the original image signal. Also, the class classification unit may have a function of classifying based on the activity in the small region of the difference image signal. The activity is, for example, a value indicating the degree of variation of the pixel value in the small area of the original image signal, the number of pixels whose luminance value in the small area of the original image signal is larger than a predetermined threshold, or the activity in the small area of the difference image signal. This is the sum of the absolute values of the pixel values.
[0014]
When the above-described configuration is adopted, the control unit determines whether or not to perform the encoding process on each of the small regions, assigns the code amount, assigns the quantization value, and the code, in accordance with the encoding priority given to each of the small regions. May be provided with a function of determining a coding pattern or a coding order.
[0015]
On the other hand, the hierarchical image decoding apparatus of the present invention generates a base layer decoded image by decoding base layer coded data with the coded data generated by the hierarchical image coding apparatus of the present invention as a processing target. , A differential image signal is generated by decoding the enhancement layer coded data, and an image signal is generated by adding the two. In the hierarchical image decoding apparatus, the region where the enhancement layer coded data is obtained is obtained. , A decoding process is performed to generate a differential image signal.
[0016]
In addition, a function of generating a decoded image using only the corresponding region of the base layer decoded image signal may be provided for an area where the enhancement layer encoded data is not obtained.
[0017]
In addition, a function may be provided for a region in which the enhancement layer coded data is not obtained, using the region corresponding to the difference image signal of the previous frame as a difference image signal of the frame.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating an outline of a hierarchical image encoding device according to an embodiment of the present invention. As illustrated, the hierarchical image encoding device 1 includes a first encoder 11, a first decoder 13, a subtractor 15, and a second encoder 17. An input original image signal is encoded in a first encoder 11. The first encoder 11 outputs base layer encoded data 12 as a result of encoding the original image signal. The first decoder 13 decodes the base layer coded data 12 and generates a base layer decoded image signal 14. Next, the subtractor 15 subtracts the base layer decoded image signal 14 from the original image signal to generate a difference image signal 16.
[0019]
The second encoder 17 receives the original image signal and the difference image signal 16 as input, divides the difference image signal 16 into small areas, and assigns area identification numbers and priorities in accordance with predetermined criteria. The encoding parameter is determined and encoded according to the priority, and the enhancement layer encoded data 18 is output.
[0020]
FIG. 2 is a block diagram illustrating an outline of a hierarchical image decoding device according to an embodiment of the present invention. As shown in the figure, the hierarchical image decoding device 2 includes a first decoder 21, a second decoder 22, and an adder 23. The base layer encoded data 12 which is an input is decoded by the first decoder 21. The first decoder 21 outputs a decoded base layer image signal 24. The enhancement layer encoded data 18 is decoded in the second decoder 22. The second decoder 22 outputs a difference image signal 25 resulting from decoding the enhancement layer encoded data 18. The base layer image signal 24 and the difference image signal 25 are added in the adder 23 and output as an enhancement image signal 26.
[0021]
In the present embodiment, there may be an area where the enhancement layer encoded data 18 cannot be obtained in the second decoder 22. In this case, the second decoder 22 performs a decoding process on the area where the enhancement layer coded data 18 is obtained to generate the differential image signal 25, but does not obtain the enhancement layer coded data 18. For the region, the enhancement image signal 26 is generated using only the region corresponding to the base layer image signal 24 as the output of the adder 23.
[0022]
As another method, the second decoder 22 performs a decoding process on a region where the enhancement layer encoded data 18 is obtained to generate a difference image signal 25, and the enhancement layer encoded data 18 For a region not obtained, a region corresponding to the difference image signal of the previous frame may be used as the difference image signal 25 of the frame.
[0023]
Next, the configuration of the present embodiment will be described more specifically. As the first encoder 11 and the first decoder 13 of the hierarchical image encoding device 1 of FIG. 1, for example, an MPEG-4 encoding method can be used.
[0024]
FIG. 3 is a block diagram showing an outline of the MPEG-4 encoder. In the MPEG-4 encoder 3 shown in FIG. 3, a subtractor 301, a DCT transformer 302, a quantizer 303, a variable-length encoder 304, a buffer 305, and a controller 306 shown in FIG. It corresponds to the first encoder 11. Further, since a decoder is also required for the encoder, it can be said that the first decoder 13 in FIG. 1 is already included in the MPEG-4 encoder 3, and the inverse quantization The unit 309, the inverse DCT transform unit 310, the adder 311, the motion compensation prediction unit 312, and the image memory 313 correspond to the first decoder 13.
[0025]
In the MPEG-4 encoder 3, when an original image signal is input, a subtractor 301 subtracts the image signal generated by the motion compensation prediction unit 312 from the original image signal. Then, the difference image signal resulting from the subtraction is subjected to DCT in the DCT transformer 302 and quantized in the quantizer 303, and the resulting quantized DCT coefficient 308 is subjected to variable length encoding in the variable length encoder 304. , Base layer encoded data 12 in the buffer 305.
[0026]
The control unit 306 controls the generated code amount of the encoded data stored in the buffer 305 and controls the fineness (quantization parameter) of the quantization in the quantization unit 303 in order to keep the rate of the encoded data constant. I do. This configuration is an example in which a quantization parameter is used as the encoding parameter 307. The quantization parameter is determined, for example, in a coding unit called a macroblock divided into 16 × 16 pixels.
[0027]
Further, the quantized DCT coefficient 308 is inversely quantized by the inverse quantization unit 309 and inverse DCT-transformed by the inverse DCT transform unit 310, and the adder 311 adds the image signal generated by the motion compensation prediction unit 312. And stored in the image memory 313. The details of the MPEG-4 encoding method are described in Non-Patent Document 1.
[0028]
FIG. 4 shows a specific example of the second encoder 17. The class classification unit 402 obtains a statistic called activity for the small area in the difference image signal 16 or the original image signal. The class identification number 403 of each small area is generated based on this activity.
[0029]
The ranking unit 404 assigns the encoding priority 405 to each small area according to the class identification number 403. The control unit 406 and the encoding unit 401 determine the presence or absence of the encoding process of each small area and the encoding parameter 407 such as the code amount allocation according to the encoding priority 405, and encode the difference image signal 16, It outputs the enhancement layer encoded data 18. Other examples of the encoding parameter 407 include a quantization value and an encoding pattern.
[0030]
As encoding means in the encoding unit 401, the difference image signal 16 is subjected to variable length encoding from the upper layer by bit plane expansion, and output up to the assigned code amount, which is one of the encoding parameters 407. There is a method of quantizing the image signal 16 and performing variable length coding on a quantized representative value.
[0031]
Next, the concept of the activity in the second encoder 17 will be described. In general, an activity indicates a characteristic of a pixel value distribution in a certain area. The user's consciousness can easily focus on semantically important parts such as a person's face and telop in the displayed image. In addition, according to human visual characteristics, it has been found that an error in a region where the change is large is slightly inconspicuous even if the error is small, and conversely it is very noticeable in a region where the change is small even if the error is small. Alternatively, the outline of the object is easily blurred due to lossy coding of the image. Therefore, if the outline is sharpened by adding the residual signal, the subjective quality is further improved. From the above, an example of the definition of the activity and how to obtain the activity will be described below.
[0032]
(A) In a small area of an original image signal, a maximum value of a luminance difference absolute value between a certain pixel and eight adjacent pixels is obtained. The number of pixels whose maximum value is larger than a certain threshold Th1 is counted as activity A. At this time, the activity A indicates the degree of change in the pixel value of the small area, and the area is classified according to the size of A as shown in FIG. Here, Th2 and Th3 in the figure are thresholds for classifying the area. The ranking unit 404 assigns priorities to the corresponding small regions of the difference image signal 16 in the order of edge region, flat region, and complex region.
[0033]
(B) In the small area of the original image signal, the number of pixels in which the luminance value of a certain pixel is larger than the threshold value Th4 is counted as activity A. An area where the activity A is larger than a certain threshold Th5 is defined as a bright area, and a small area is classified as a normal area. The ranking unit 404 assigns priorities to the corresponding small regions of the difference image signal 16 in the order of a bright region and a normal region.
[0034]
(C) In the small area of the difference image signal 16, the sum of the absolute values of the pixel values is defined as the activity A, and is classified into an area smaller than a certain threshold Th6 and a larger area. The ranking unit 404 assigns priorities in the order of smaller area and larger area. This is because it can generally be said that in the first encoder 11, a complicated area of the original image signal is roughly quantized by a large quantization value, and the encoded residual signal also becomes large.
[0035]
Next, a processing example in the control unit 406 will be described. For the small area to which the priority is given by the above processing, the threshold of the generated code amount is increased according to the priority, and the fineness of the quantization is adjusted, and the number of bit planes to be encoded is reduced. increase. Alternatively, the small areas are encoded in the order of priority and output as the enhancement layer encoded data 18. In a small area to which a large code amount is allocated, more differential image signal information can be encoded and transmitted, and the image quality of the enhancement image signal 26 obtained on the decoding side is improved. In addition, the data obtained by encoding the small areas in the order of priority is reduced from the end of the data, so that the transmission side can control the data amount and quality improvement according to the capacity of the communication path.
[0036]
【The invention's effect】
As described above, the present invention performs coding according to the characteristics of an image by using the activity of an original image signal or a difference image signal, and achieves subjective quality even with the same code amount as that of the conventional technology. Can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an outline of a hierarchical image encoding device.
FIG. 2 is a block diagram illustrating an outline of a hierarchical image decoding device.
FIG. 3 is a block diagram illustrating a configuration example of an MPEG-4 encoder.
FIG. 4 is a block diagram illustrating a specific configuration example of a second encoder.
FIG. 5 is a diagram showing an example of area classification by activity.
FIG. 6 is a block diagram illustrating an outline of a conventional hierarchical image encoding device.
[Explanation of symbols]
1,7 Hierarchical image encoder 2 Hierarchical image decoder 3 MPEG-4 encoder 11,71 First encoder 12,72 Base layer encoded data 13,21,73 First decoder 14 , 74 Base layer decoded image signal 15, 75 Subtractor 16, 25, 76 Difference image signal 17, 77 Second encoder 18, 78 Enhancement layer encoded data 22 Second decoder 23 Adder 24 Base layer image Signal 26 Enhancement image signal 301 Subtractor 302 DCT transformer 303 Quantizer 304 Variable length encoder 305 Buffer 306, 406 Controller 307, 407 Coding parameter 308 Quantized DCT coefficient 309 Dequantizer 310 Inverse DCT transformer 311 Adder 312 Motion compensation prediction unit 313 Image memory 401 Encoding unit 402 Class classification unit 4 3 class identification number 404 prioritization unit 405 coding priority

Claims (6)

A first encoder divides the original image signal into a plurality of small areas, performs encoding processing for each of the small areas, generates base layer encoded data, and decodes the encoded data into a base layer. A difference image signal is generated from the decoded image signal and the original image signal, and the second encoder divides the difference image signal into a plurality of small regions corresponding to the small regions of the original image signal. In a hierarchical image encoding device that performs an encoding process for each small region of an image signal and generates it as enhancement layer encoded data,
The second encoder is:
A class classification unit that classifies the small areas of the difference image signal according to a predetermined criterion and generates a class identification number indicating the class of each small area;
A ranking unit for assigning an encoding priority to each of the classified small areas;
A control unit for determining an encoding parameter according to the assigned encoding priority. The hierarchical image encoding device according to claim 1,
The hierarchical image encoding apparatus according to claim 1, wherein the class classification unit performs classification based on an activity indicating a characteristic of a pixel value distribution in a small region of the original image signal or the difference image signal. The hierarchical image encoding device according to claim 2,
The activity is a value indicating a degree of variation of a pixel value in a small area of the original image signal, the number of pixels whose luminance value in the small area of the original image signal is larger than a predetermined threshold value, or a value in a small area of the difference image signal. A hierarchical image encoding device, which is a sum of absolute values of pixel values. The hierarchical image encoding device according to any one of claims 1 to 3,
The control unit determines the presence / absence of an encoding process for each small area, allocation of a code amount, a quantization value, an encoding pattern, or an encoding order according to the encoding priority given to each of the small areas. A hierarchical image encoding device characterized by the above-mentioned. A base layer coded data and an enhancement layer coded data coded by the hierarchical image coding apparatus according to any one of claims 1 to 4, are input, and the base layer coded data is A hierarchical image decoding device that generates a base layer decoded image by decoding, generates a difference image signal by decoding the enhancement layer coded data, and generates an image signal by adding the two. At
In the area where the enhancement layer encoded data is obtained in the second decoder, a decoding process is performed to generate a differential image signal,
A hierarchical image decoding apparatus characterized in that a region in which enhancement layer encoded data is not obtained is generated as a decoded image using only a region corresponding to a base layer decoded image signal. A base layer coded data and an enhancement layer coded data which are coded by the hierarchical image coding apparatus according to any one of claims 1 to 4, and are inputted to a first decoder. By decoding the base layer coded data to generate a base layer decoded image, decoding the enhancement layer coded data in a second decoder to generate a differential image signal, and adding the two, In a hierarchical image decoding device that generates image signals,
In the area where the enhancement layer encoded data is obtained in the second decoder, a decoding process is performed to generate a differential image signal,
A hierarchical image decoding apparatus characterized in that a region corresponding to a difference image signal of a previous frame is used as a difference image signal of the frame for a region where the enhancement layer encoded data is not obtained.

Images