JP2003304538A - Image encoder, image decoder, and method for them - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image decoder to which an in-loop filter can be mounted even with a low processing capability. <P>SOLUTION: In an intra-mode where a frame picture is encoded as an I picture, the frame picture is compression-coded by an orthogonal transform section 113, a quantization section 114, and a variable length encoding section 117 and the resulting picture is outputted via a post-processing section 120. A quantized value outputted from the quantization section 114 is decoded into a frame picture by a dequantization section 121 and an inverse orthogonal transform section 122. The decoded frame picture (decoded image) is stored in a frame memory 141 after applying filter processing to eliminate block distortion by the in-loop filter 140 when switches 131, 132 are closed under the control of a filter processing control section 160 or stored in the frame memory 141 without being subjected to the filter processing when the switches 131, 132 are turned off. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding device,
The present invention relates to image decoding devices and methods thereof, and particularly to a technique for encoding and decoding frame images repeatedly input.

[0002]

2. Description of the Related Art ITU (International Telecommunication Union Electronics) has been used as a standard technique for continuously encoding a moving image with a low bit rate (high compression ratio) and high image quality and for decoding the encoded moving image. H.C. 261,
H. 263, ISO (International Organization for Standardization) MPEG (M
moving Picture Experts Gro
up) -1, MPEG-2, MPEG-4, etc. have been developed. As a next-generation technology that further develops while inheriting the technologies such as MPEG-1 to 4, H.264 standardized by ITU at present. There is 26L. This H. In 26L, in order to further enhance the image quality improving effect at a low bit rate (high compression rate), an in-loop filter that removes noise in block units from a reference image used in inter-picture predictive coding and decoding is adopted.

[0003] FIG. It is a block diagram which shows the function structure of the conventional image coding apparatus 500 disclosed by 26L. As shown in the figure, the image encoding device 500
Is a subtraction unit 512, an orthogonal transformation unit 513, and a quantization unit 5
14, a variable length coding unit 517, and an inverse quantization unit 521.
An inverse orthogonal transformation unit 522, an addition unit 524, an in-loop filter 540, a frame memory 541, a motion detection unit 542, and a motion compensation unit 543.

The image coding device 50 configured as described above
At 0, in the intra mode (in-plane coding) in which the frame image is coded as an I picture, the input image is compression-coded into frequency components by the orthogonal transformation by the orthogonal transformation unit 513 and quantized by the quantization unit 514. Is compressed and encoded into a quantized value. This quantized value is compression-coded into a variable length by Huffman coding by the variable-length coding unit 517, and is output as an I-picture coded signal.

On the other hand, the quantized value output from the quantizing unit 514 is decoded into frequency components by the inverse orthogonal transform by the inverse quantizing unit 521, and decoded into a frame image by the inverse orthogonal transform by the inverse orthogonal transform unit 522. To be done. The decoded frame image (decoded image) is stored in the frame memory 541 after being subjected to filter processing for removing block distortion by the in-loop filter 540.

In the inter mode (inter-frame coding) in which an input image is coded as a P picture and a B picture, a motion vector is generated by a motion detection section 542, and a motion compensation image (predicted image) is generated by a motion compensation section 543. Is generated, and the subtraction unit 512 generates a motion compensation error (difference image). The motion detection unit 542
Searches for a forward, backward, or bidirectional reference image having the smallest difference from the input image in the decoded image stored in the frame memory 541. The motion detection unit 542 outputs the motion amount of the difference pixel between the searched reference image and the input image as a motion vector, and determines whether the referenced image is the front image, the rear image, or the average value of the bidirectional images. The block prediction type indicating whether or not is output. Also, the motion compensation unit 543 performs a calculation indicated by the motion vector output from the motion detection unit 542 and the block prediction type on the difference pixel to generate a motion compensation image. Then, the subtraction unit 512 subtracts the input image from the motion compensation image generated by the motion compensation unit 543,
A motion compensation error (difference image) is generated.

The motion compensation error (difference image) output from the subtraction unit 512 is compression-encoded into frequency components by the orthogonal transformation by the orthogonal transformation unit 513, and compressed into a quantized value by the quantization by the quantization unit 514. To be done. This quantized value is compression-coded into a variable length by Huffman coding by the variable-length coding unit 517, and is output as a P-picture or B-picture coded signal together with a motion vector and the like.

On the other hand, the quantized value of the P picture or B picture output from the quantizing unit 514 is the inverse quantizing unit 521.
Is decoded into a frequency component by the inverse orthogonal transform by the above, and is decoded into a motion compensation error (difference image) by the inverse orthogonal transform by the inverse orthogonal transform unit 522. Then, the addition unit 524
Thus, by adding the motion compensation error (difference image) and the motion compensation image, the frame image is decoded.
The decoded frame image (decoded image) is stored in the frame memory 541 after being subjected to filter processing for removing block distortion by the in-loop filter 540.

As described above, in the image coding apparatus 500, the decoded image is constantly subjected to the filter processing by the in-loop filter 540, and the decoded image subjected to the filter processing is stored in the frame memory 541. Therefore, block distortion is not accumulated in the decoded image stored in the frame memory 541, the prediction efficiency by the motion detection unit 542 and the motion compensation unit 543 is improved, and MPE is improved.
It is possible to reduce the image quality deterioration as compared with the G technique.

FIG. 9 shows the above H.264. It is a block diagram which shows the function structure of the conventional image decoding apparatus 600 disclosed by 26L. As shown in the figure, the image decoding device 60
0, which decodes an encoded signal output from the image encoding device 500, and includes a variable length decoding unit 617, an inverse quantization unit 621, an inverse orthogonal transformation unit 622, and an addition unit 62.
4, the in-loop filter 640, and the frame memory 64
1 and a motion compensation unit 643.

The image decoding apparatus 60 configured as described above
In the case of 0, in the intra mode in which the coded signal of the I picture is decoded into the frame image, the coded signal is inverse Huffman coded by the variable length decoding unit 617 to be decoded into a quantized value, and the dequantization unit The frequency component is expanded and decoded by the inverse orthogonal transform of 621, and the inverse orthogonal transform unit 6
It is decoded into a frame image (decoded image) by the inverse orthogonal transform by 22. The decoded frame image (decoded image) is filtered by the in-loop filter 640 to remove block distortion, and then stored in the frame memory 641 and output to a monitor or the like.

In the inter mode in which the coded signals of the P picture and the B picture of the frame image are decoded into the frame image, the coded signal is the variable length decoding unit 617.
Inverse Huffman coded by and decoded into quantized values,
The inverse quantization unit 621 expands and decodes the frequency component by the inverse orthogonal transform, and the inverse orthogonal transform unit 622 performs the inverse orthogonal transform to decode the motion compensation error (difference image).

On the other hand, the motion compensation unit 643 generates a motion compensation image (predicted image). The motion compensation unit 64
3 performs a calculation indicated by the motion vector and the block prediction type on the difference pixel of the reference image read from the frame memory 641 to generate a motion compensation image.

Then, the addition unit 624 adds the motion compensation error (difference image) and the motion compensation image to decode the frame image. The decoded frame image (decoded image) is filtered by the in-loop filter 640 to remove block distortion, and then stored in the frame memory 641 and output to a monitor or the like.

As described above, in the image decoding apparatus 600, the in-loop filter 640 constantly applies the filtering process, and the decoded image subjected to the filtering process is stored in the frame memory 641. Therefore, the block distortion is not accumulated in the decoded image stored in the frame memory 641, the prediction efficiency by the motion compensation unit 643 is improved, and the image quality deterioration can be reduced as compared with the MPEG technique.

[0016]

However, the amount of calculation in the in-loop filters 540 and 640 used in the conventional image coding apparatus 500 and image decoding apparatus 600 is extremely large. This is because, for example, the process of calculating an average value of a certain pixel and its surrounding pixels and replacing the certain pixel with the calculated average value must be executed one by one for the pixels near the block boundary.

Therefore, if an in-loop filter is mounted on an image coding apparatus or an image decoding apparatus having a small processing capacity, all the coding / decoding processing can be used for a predetermined time (one frame can be processed). However, there is a problem in that smooth image coding and decoding cannot be performed.

Therefore, in the conventional image encoding device / image decoding device, a hardware encoder (decoder) having a CPU having a very high computing power is mounted on the device, or otherwise, an in-loop filter is not mounted. There was only one of them. Such a problem is particularly problematic when the program is installed in a computer device having a low processing capacity such as a personal computer or a portable information terminal, and an image encoding device or an image decoding device is constructed by the installed program. Become.

The present invention has been made to solve the above problems, and an image decoding apparatus capable of implementing an in-loop filter even in an image decoding apparatus (image decoding apparatus) having a low processing capacity. It is an object to provide an (image decoding device) and these methods.

[0020]

In order to achieve the above object, an image coding apparatus according to the present invention is an image coding apparatus or the like for coding a frame image that is repeatedly input, and a predetermined frame image is used. Encoding means for performing encoding by performing the conversion processing of 1., inverse conversion means for performing the inverse conversion processing of the conversion processing on the frame image encoded by the encoding means, and filtering processing for the frame image. A filtering unit, a storage unit for storing the frame image, and a frame image obtained by the inverse conversion processing by the inverse conversion unit, after the filtering process by the filtering unit, the frame image is stored by the storage unit. Control for storing the frame image in the storage unit or storing the frame image in the storage unit without performing the filtering process by the filtering unit. And a stage, said encoding means,
It is characterized in that the frame image is encoded while referring to a past frame image stored in the storage means.

Here, the control means performs the filter processing on the frame image when the importance of the frame image is high, and the control means performs the filtering process on the frame image when the importance of the frame image is low. You may control so that the said filter process may not be performed. For example, the control unit determines that the frame image has a high degree of importance when the frame image is subjected to in-plane encoding by the encoding unit, and performs the filtering process on the frame image. If the frame image is subjected to inter-plane coding by the coding means, it is determined that the importance of the frame is low, and the filtering process is not performed on the frame image. To control
When the frame image has been subjected to forward prediction encoding by the encoding means, it is determined that the frame has a high degree of importance, the frame image is subjected to the filter processing, and the frame image When the bidirectional predictive coding is performed by the encoding means, it is determined that the importance of the frame is low, or control not to perform the filtering process on the frame image, When the frame image has been subjected to base layer encoding by the encoding means, it is determined that the frame has a high degree of importance, the frame image is subjected to the filter processing, and the frame image is obtained. Is that the enhancement layer encoding has been performed by the encoding means, it is determined that the importance of the frame is low, and May be controlled not subjected to the filtering process on the frame image.

Further, the control means monitors a margin regarding the processing capacity of the image coding apparatus, and if there is a margin, performs a filtering process on a frame image of lower importance, and if there is no margin, May be controlled not to perform the filtering process on the frame images of higher importance. For example, the frame image is associated with a priority order corresponding to the degree of importance, and the control unit monitors the margin by monitoring the operation rate of the CPU included in the image encoding device, and the operation is performed. If the rate is high,
It is also possible to perform control processing so that only frame images with high priority are subjected to filter processing, and when the operation rate is low, filter processing is also performed with respect to frame images with low priority.

In order to achieve the above object, the image decoding apparatus according to the present invention is an image decoding apparatus for decoding repeatedly input frame images, and applies predetermined conversion processing to the frame images. Decoding means for performing decoding by doing so, filtering means for filtering the frame image, storage means for storing the frame image,
The frame image obtained by the decoding by the decoding unit is subjected to the filtering process by the filtering unit and then the frame image is stored in the storage unit, or the filtering process by the filtering unit is not performed. And a control means for controlling the frame image to be stored in the storage means, wherein the decoding means decodes the frame image while referring to a past frame image stored in the storage means. Is characterized by. Then, the image decoding apparatus may include the same characteristic means as the image encoding apparatus.

The present invention can be realized not only as such an image coding apparatus and image decoding apparatus, but also as an image coding method and an image decoding method in which the characteristic means of these apparatuses are used as steps. It can also be realized as a computerized method or as a program that causes a computer to execute those steps. Needless to say, the program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. (Embodiment 1) FIG. 1 is a block diagram showing a functional configuration of an image coding apparatus 100 according to Embodiment 1 of the present invention.

The image encoding device 100 is realized by a computer device including a CPU, a memory, a hard disk (HD) in which a program for image encoding is installed, and its function is an operation receiving unit 105 and
Preprocessing unit 110, subtraction unit 112, and orthogonal transformation unit 113
A quantization unit 114, a variable length coding unit 117, a post-processing unit 120, an inverse quantization unit 121, and an inverse orthogonal transformation unit 12.
2, an addition unit 124, a switch unit 130, an in-loop filter 140, a frame memory 141, a motion detection unit 142, a motion compensation unit 143, and a priority order determination unit 15.
0 and a filter processing controller 160.

The operation receiving unit 105 receives the input operation of the operator. The preprocessing unit 110 includes a format conversion unit that performs format conversion of the input image signal into a spatial resolution designated by the operation of the operation reception unit 105, a screen rearrangement unit that rearranges screens according to picture types, and the like.
The picture type and the frame image for each frame are sequentially output.

As the picture type, an I picture (Intr) without a reference image created in the intra mode is used.
a Picture: an in-plane coded image, a P picture (Predictive Picture: a predictable coded image) created in inter mode and referencing only the forward image, and a B picture (Bi) that can also refer to the backward image.
directionally predictive
Picture: bidirectionally predictable image), and the motion detection unit 1 is used for motion detection in the inter mode.
The referenceable decoded image stored in the frame memory 141 by 42 is limited.

When the frame image is encoded, 3
Mode for encoding using one picture (this mode is also referred to as “IPB encoding mode” hereinafter)
And a mode in the case of encoding using only two of I picture and P picture. This I picture and P
The mode for encoding using only two pictures is a mode for encoding P pictures that may be referred to and P pictures that are not likely to be referred to (hereinafter, this mode will be referred to as "old IP encoding mode"). ]), And the basic P
There is a mode for encoding a picture, a P picture that can be referred to, and a P picture that cannot be referred to (this mode is also referred to as “new IP encoding mode” hereinafter).

In the case of the old IP encoding mode,
"Possible" or "not possible" in frame image
Information is added, and "possibility" or "no possibility" information is added to the picture type. Further, in the case of the new IP encoding mode, the information "basic", "possible" or "not possible" is added to the frame image, and "basic" is added to the picture type.
Information of “possible” or “not possible” is added.

The subtraction unit 112 outputs the frame image output from the pre-processing unit 110 as it is in the intra mode, and the difference value between the frame image and the motion compensation image output from the motion compensation unit 143 in the inter mode. Calculate the motion compensation error (residual image).

The orthogonal transformation unit 113 transforms the frame image in the intra mode and the motion compensation error in the inter mode output from the subtraction unit 112 into the frequency domain by performing orthogonal transformation such as discrete cosine transform. The frequency component is output.

The quantizing unit 114 quantizes the frequency component output from the orthogonal transforming unit 113 and outputs a quantized value. The variable length coding unit 117 includes a quantization unit 11
By using a variable length code (Huffman code) that assigns a code length according to the frequency of occurrence to the quantized value output from No. 4, a coded signal subjected to further information compression is output.

The post-processing unit 120 includes a buffer for temporarily storing coded signals and the like, a rate control unit for controlling the quantization width in the quantizing unit 114, and the like, and the motion vector, picture type, etc. , And converts the encoded signal output from the variable length encoding unit 117 into an encoded signal of a bit stream and outputs the encoded signal. The inverse quantizer 121 decodes the frequency component by inversely quantizing the quantized value generated by the quantizer 114.

The inverse orthogonal transform unit 122 includes an inverse quantization unit 121.
By inverse orthogonal transforming the frequency component decoded by, the frame image is decoded in the intra mode, and the motion compensation error (residual image) which is the difference value of the pixels in the inter mode is decoded.

The adding unit 124 outputs the frame image (decoded image) decoded by the inverse orthogonal transform unit 122 as it is in the intra mode, and the residual image decoded by the inverse orthogonal transform unit 122 in the inter mode,
The frame image is decoded by adding the motion compensation image generated by the motion compensation unit 143.

The switch section 130 is composed of a pair of switches 131 and 132 for switching the switching mode in synchronization with the switch ON / OFF control for each picture by the filter processing control section 160.
0 is incorporated in the loop or removed from the loop, that is, the processing by the in-loop filter 140 is skipped.

The in-loop filter 140 includes the switch 13
When 1, 132 is ON, the decoded image output from the addition unit 124 is spatially low-pass filtered on a block-by-block basis to generate a decoded image without block distortion. For example, if the difference between a pixel and its surrounding pixels is calculated and the difference between a pixel and its surrounding pixels is within a predetermined range, the process of replacing that pixel with the calculated average value is performed in the vicinity of the block boundary. Perform one pixel at a time.

The frame memory 141 includes a switch unit 13
The decoded image output from 0 is stored for a plurality of frames. This makes it possible to monitor the decoded image in the same state as the image decoding device that decodes the coded signal output from the post-processing unit 120, or use the decoded image as a reference image in the inter mode. The old IP
In the coding mode and the new IP coding mode, the decoded image of the P picture to which the information with the possibility of reference is added is always stored in the frame memory 141, and the decoded picture of the P picture with the information without the possibility of reference is added. The decoded image does not need to be stored in the frame memory 141. For this reason, the information of the possibility / impossible is to use the decoded image in the frame memory 14
The same meaning is stored or not.

The motion detection unit 142 searches the decoded image stored in the frame memory 141 in the inter mode for a reference image having the smallest difference from the frame image output from the preprocessing unit 110, and detects the motion of the difference pixel. Output a motion vector that is a quantity. When outputting a motion vector, the block prediction type is output whether the image to be referred to is a front image, a rear image, or an average value of both images.

The motion compensator 143 performs the operation indicated by the motion vector and the block prediction type to generate a motion compensated image. The priority order determination unit 150 outputs the priority level of the picture according to the picture type, the base layer, and the enhancement layer. The filter processing control unit 160 controls ON / OFF of the switches 131 and 132 according to the priority and the CPU operating rate output from the priority order determining unit 150.

FIG. 2 shows the priority order determining unit 1 shown in FIG.
It is a block diagram which shows the detailed functional structure of 50. As shown in the figure, the priority determining section 150 outputs the priority of the picture according to the picture type, the base layer, and the enhancement layer. As shown in FIG. Tables 151-153
, A selector 154, and a decision processing unit 155.
In the case of a P picture in the old and new IP coding modes, information indicating "basic", "possible to be referenced" or "none" is added to the picture type.

The table 151 is a table that is selected when the IPB coding mode is designated by the operation of the operation receiving unit 105 and associates the picture type of the frame image with the priority. In the I picture, the priority is set. Is "0", the priority is set to "1" for P pictures, and the priority is set to "2" for B pictures. Note that the priority is set to decrease as the numerical value increases.

The table 152 is a table that is selected when the old IP encoding mode is specified by the operation of the operation receiving unit 105 and associates the picture type of the frame image with the priority. The degree is “0”, the priority is set to “1” in the P (possibly referred to) picture, and the priority is set to “2” in the P (not possibly referred to) picture.

The table 153 is a table selected when the I, P (basic, possible reference, no reference possibility) coding mode of the new standard (multi-frame reference coding) is designated. In the I picture, the priority is “0”, and in the P (basic) picture, the priority is “1”, P
The priority is set to “2” for the (possibly referenced) picture, and the priority is set to “3” for the P (not referenced) picture.

The selector 154 has tables 151 to 15 based on the coding mode (IPB coding mode, old and new IP coding mode) designated by the operation receiving unit 105.
Select any one of 3.

The decision processing unit 155 refers to the table selected by the selector 154 and refers to the preprocessing unit 110.
The priority level is determined according to the picture type, the base layer, and the enhancement layer output from, and the determined priority level is output.

Specifically, when the IPB coding mode is designated, the selector 154 selects the table 151, and the decision processing unit 155 outputs the picture type from the preprocessing unit 110 each time. I picture, P
The priority associated with the picture and B picture is output.

When the old IP encoding mode is designated, the selector 154 selects the table 152, and the decision processing unit 155 determines the picture type and the data ("possibility") added to the P picture. Output, based on "Yes", "No possibility").

Further, when the new IP coding mode is designated, the selector 154 selects the table 153, and the decision processing unit 155 determines that the picture type and P
Based on the data attached to the picture type ("basic", "possible", "not possible" of reference),
Output the priority.

FIG. 3 is a block diagram showing a detailed functional configuration of the filter processing controller 160 shown in FIG. As shown in FIG. 3, the filter processing control unit 160 controls ON / OFF of the switches 131 and 132 according to the priority and the CPU operating rate output from the priority determination unit 150. As shown, it comprises three tables 161-163, a selector 164, and a switch switching processing unit 165.

The table 161 is a table which is selected when the IPB encoding mode is designated and shows a combination of the priority and the CPU operating rate when the filtering process is performed, and the CPU operating rate is 70%. When the priority is 0 or less, the switch is turned on when the priority is 0 to 2, and the CPU operation rate is 7
When 0% or more and less than 80%, the switch is turned on only when the priorities are 0 and 1, and when the CPU operation rate is 80% or more, the switch is turned on only when the priority is 0.

The table 162 is a table which is selected when the old IP encoding mode is designated and shows a combination of the priority and the CPU operating rate when the filtering process is performed, and the CPU operating rate is 70. If it is less than%, the switch is turned on when the priority is 0 to 2, and the CPU operation rate is 7
When 0% or more and less than 80%, the switch is turned on only when the priorities are 0 and 1, and when the CPU operation rate is 80% or more, the switch is turned on only when the priority is 0.

A table 163 is a table which is selected when the new IP encoding mode is designated and shows a combination of the priority and the CPU operating rate when the filtering process is performed. The CPU operating rate is 70%. If it is less than%, the switch is turned on when the priority is 0 to 3, and the CPU operation rate is 7
When 0% or more and less than 80%, the switch is turned on only when the priority is 0, 1 and 2, and when the CPU operating rate is 80% or more, the switch is turned on only when the priority is 0 or 1. There is.

The selector 164 has tables 161 to 16 based on the coding mode (IPB coding mode, old and new IP coding mode) designated by the operation receiving unit 105.
Select any one of 3.

The switch changeover processing unit 165 has the selector 1
Based on the priority output from the priority determination unit 150 and the CPU operating rate acquired for each picture, the table selected by 64 is output, and a switch ON or OFF signal is output, and the switch 131 of the switch unit 130 is output. , 132 is ON / OFF controlled.

Specifically, when the IPB coding mode is designated, the selector 164 selects the table 161, and the switch switching processing unit 165, when the CPU operating rate is less than 70%, I picture, P picture, B
A switch ON signal is output for all the pictures. When the CPU operating rate is 70% or more and less than 80%, the switch switching processing unit 165 outputs the switch ON signal only for the I picture and the P picture. When the CPU operating rate is 80% or more, the switch switching processing unit 165 outputs the switch ON signal only in the case of the I picture.

When the old IP coding mode is designated, the selector 164 selects the table 162, and the switch switching processing section 165, when the CPU operating rate is less than 70%, the I picture and the P picture. The switch ON signal is output for all (possible) and P pictures (not possible). Also, the CPU operation rate is 70%
When the ratio is less than 80%, the switch switching processing unit 165
Outputs a switch ON signal only in the case of an I picture and a P picture (possibly). Also, CPU
When the operation rate is 80% or more, the switch switching processing unit 165
Outputs a switch ON signal only in the case of an I picture.

Further, when the new IP coding mode is designated, the selector 164 has selected the table 163, and the switch switching processing section 165 has the I picture,
A switch ON signal is output for all of the P picture (basic), P picture (possible), and P picture (possible). Also, if the CPU operation rate is 70% or more, 80
If it is less than%, the switch switching processing unit 165 outputs the switch ON signal only in the case of I picture, P picture (basic), and P picture (possible).
When the CPU operating rate is 80% or more, the switch switching processing unit 165 outputs the switch ON signal only in the case of the I picture and the P picture (basic).

Next, the operation of the image coding apparatus 100 thus configured will be described. In the intra mode in which a frame image is encoded as an I picture, the preprocessing unit 1
The frame image output from 10 is the orthogonal transform unit 113.
Is compressed and coded into frequency components by orthogonal transformation by
The quantization value is compressed and encoded by the quantization unit 114. This quantized value is compression-encoded into a variable length by Huffman encoding by the variable-length encoding unit 117,
The post-processing unit 120 converts the I-picture bitstream into an encoded signal and stores it in a storage medium such as a hard disk.

On the other hand, the quantized value output from the quantizing unit 114 is decoded into frequency components by the inverse orthogonal transform by the inverse quantizing unit 121, and decoded into a frame image by the inverse orthogonal transform by the inverse orthogonal transform unit 122. To be done. In the decoded frame image (decoded image), the switches 131 and 132 are turned on under the control of the filter processing control unit 160.
If it is, the filter processing for removing the block distortion is performed by the in-loop filter 140 and then stored in the frame memory 141, and the switches 131 and 132 are stored.
If is turned off, it is stored in the frame memory 141 without being subjected to the filtering process.

In the inter mode in which the frame image is encoded as a P picture and a B picture, the motion detecting unit 142 generates a motion vector, the motion compensating unit 143 generates a motion compensating image (predictive image), and the subtracting unit. A motion compensation error (difference image) is generated by 112. The motion detection unit 142 selects the preprocessing unit 1 from the decoded images stored in the frame memory 141.
The forward, backward, or bidirectional reference image having the smallest difference from the frame image output from 10 is searched under a predetermined restriction.

FIG. 4 is a diagram showing the reference relationship of the frame images stored in the frame memory 141. In particular,
FIG. 4A is a diagram showing a reference image in the prediction in the case of the IPB system, FIG. 4B is a diagram showing a reference image in the prediction in the case of the old IP system, and FIG. IP
It is a figure which shows the reference image in the prediction in the case of a system. Note that the priority order (priority) associated with each picture is shown in the lower column of each picture of each method.

P in the case of the IPB method of FIG. 4 (a)
In predicting a picture, the preceding I picture and P picture can be referred to. In prediction of a B picture, it is possible to refer to a preceding I picture or P picture, and it is possible to refer to one backward or temporally closest I picture or P picture.

H. In the prediction of 26L B picture, in addition to I picture and P picture, B picture can be referred to as a forward image. In the case of the mode in which this B picture is used as a reference image, information of "possible" or "not possible" is added to the frame image of the B picture, and "possible" or "possible" is added to the picture type. Information of "no sex" is added. In this mode, the decoded image of the B picture to which the information with the possibility of reference is added is always stored in the frame memory 141, and the decoded image of the B picture with the information without the possibility of reference is stored in the frame memory 141. It need not be stored in 141.

P in the case of the old IP method of FIG. 4 (b)
Prediction of (possible reference) pictures can refer to forward I pictures and P (possible reference) pictures. Prediction of P (no reference possible) pictures can refer to the preceding I or P (possible reference) pictures.

P in the case of the new IP system of FIG. 4 (c)
For prediction of (basic) pictures, the forward I picture and P
The (basic) picture can be referenced. In predicting a P (possible reference) picture, a preceding I picture and a P (basic) picture can be referred to. In predicting a P (no possibility of reference) picture, a plurality of forward I pictures, P (basic) pictures or P (possible reference) pictures can be referenced.

For convenience of explanation, a case where the IPB coding mode is designated will be described. Under such a limitation, the motion detection unit 142 outputs the motion amount of the difference pixel between the searched reference image and the frame image output from the preprocessing unit 110 as a motion vector, and the referenced image is the front image. A block prediction type indicating whether the image is a backward image or an average value of bidirectional images is output. In addition, the motion compensation unit 143 performs a calculation indicated by the motion vector output from the motion detection unit 142 and the block prediction type on the difference pixel to generate a motion compensation image. Then, the subtraction unit 112 generates a motion compensation error (difference image) by subtracting the frame image output from the preprocessing unit 110 and the motion compensation image generated by the motion compensation unit 143.

The motion compensation error (difference image) output from the subtraction unit 112 is compression-encoded into frequency components by the orthogonal transformation by the orthogonal transformation unit 113, and compressed into a quantized value by the quantization by the quantization unit 114. To be done. This quantized value is compression-coded into a variable length by Huffman coding by the variable-length coding unit 117, and converted into a coded signal of a P-picture or B-picture bitstream together with a motion vector or the like by the post-processing unit 120, It is stored in a storage medium such as a hard disk.

On the other hand, the quantized value of the P picture or B picture output from the quantizing unit 114, which has the possibility of reference, is decoded into frequency components by the inverse orthogonal transform by the inverse quantizing unit 121, and the inverse orthogonal transform is performed. The motion compensation error (difference image) is decoded by the inverse orthogonal transform by the unit 122.
Then, the addition unit 124 adds the motion compensation error (difference image) and the motion compensation image to decode the frame image. This decoded frame image (decoded image) is subjected to a filtering process for removing block distortion by the in-loop filter 140 when the switches 131 and 132 are turned on under the control of the filtering process control unit 160. , Are stored in the frame memory 141, and when the switches 131 and 132 are turned off, they are stored in the frame memory 141 without being filtered.

Here, the ON / OFF control of the switches 131 and 132 by the filter processing controller 160 will be described in more detail. FIG. 5 is a flowchart showing the switch drive processing executed by the switch switching processing section 165 of the filter processing control section 160.

By the way, the decision processing unit 155 of the priority order decision unit 150 refers to the table 151 selected by the selector 154 and decides the priority according to the picture type for each picture output from the preprocessing unit 110. , The determined priority is output. Specifically, IP
When the B encoding mode is designated, the selector 154
Has selected the table 151, and the decision processing unit 15
5, each time the picture type is output from the preprocessing unit 110, the priority is “0” for an I picture, “1” for a P picture, and “1” for a B picture. 2 ”is output.

The switch change-over processing unit 165 of the filter processing control unit 160 has a priority of the picture for each coding of the picture and a CPU included in the image coding apparatus 100.
And the operating rate of (S21), and the entry to be referenced in the table (in the example of FIG. 3, table 161) is determined (S22).

Specifically, if the CPU operating rate is less than 70%, the entry to be referred to is determined to be the first line, and the CPU
If the operating rate is 70% or more and less than 80%, the entry to be referred to is determined as the second line, and if the CPU operating rate is 80% or more, the entry to be referred to is determined as the third line.

When the entry to be referred to is determined, the switch switching processing section 165 reads the right column of the entry (S23), and determines whether the priority set in the picture type of the decoded image is in the right column. (S24).
If it is in the right column (Yes in S24), the switch switching processing unit 165 sends a switch-on signal to the switches 131 and 13
2 is output (S25). As a result, the decoded image is filtered, and the filtered decoded image is stored in the frame memory 141.

On the other hand, if it is not in the right column (No in S24), the switch switching processing section 165 outputs a switch-off signal to the switches 131 and 132 (S26).
As a result, the filtering process for the decoded image is skipped, and the decoded image is stored in the frame memory 141 without performing the filtering process. Such control is performed for each picture, and the decoded image that has been subjected to the filter processing and the decoded image that has not been subjected to the filter processing are sequentially stored in the frame memory 141.

Therefore, in the image coding, the in-loop filter such as noise removal is not always applied, but the in-loop filter can be selectively applied as needed, so that the image quality is greatly affected, for example. By applying the in-loop filter only to the frame image, the image quality of the important frame stored in the frame memory is maintained and the decoded image stored in the frame memory is maintained even if the image coding device has a small processing capacity. Block distortion is less likely to be accumulated, the prediction efficiency by the motion compensation unit is improved, image quality deterioration can be reduced as compared with the MPEG technique, and a high image quality improvement effect can be obtained at a low bit rate.

That is, a frame image having a large influence on other frame images, that is, an in-plane coded frame image, a forward predictive coded frame image, a base layer frame image, etc. is preferentially filtered in the loop. Therefore, it is possible to more effectively obtain an image quality improvement effect such as noise removal by the in-loop filter with respect to the same increase in processing load.

Further, since the ON / OFF of the filter processing can be controlled so that the processing capability of the image coding apparatus can be fully utilized, the CPU is used with high efficiency, and the same hardware resources are used. Even in this case, high quality encoding is realized.

(Embodiment 2) Next, an image decoding apparatus according to an embodiment of the present invention will be described. Figure 6
It is a block diagram which shows the functional structure of the image decoding apparatus 200 which concerns on Embodiment 2 of this invention. This image decoding device 2
00 is for decoding the coded signal coded by the image coding apparatus 100 shown in FIG.
It is realized by a computer device including a memory, a hard disk (HD) in which a program for image decoding is installed, and the function thereof is the preprocessing unit 210.
A variable length decoding unit 217, an inverse quantization unit 221, an inverse orthogonal transformation unit 222, an addition unit 224, and a switch unit 23.
0, the in-loop filter 240, the post-processing unit 270,
It includes a frame memory 241, a motion compensation unit 243, a priority order determination unit 250, and a filter processing control unit 260.

The pre-processing unit 210 is provided with a buffer for temporarily storing the coded signal, and separates and outputs the picture type, the motion vector, and the coded signal of the image itself included in the coded signal. When the encoded signal of the image is in the old IP encoding mode, the information of “possible” or “not possible” is added to the frame image, and the “probable” or “possible” is added to the picture type. The information “no possibility” is added. Further, in the case of the new IP encoding mode, "basic", "possible" or "not possible" information is added to the frame image, and "basic" or "possible" is added to the picture type. Alternatively, “no possibility” information is added.

The variable length decoding unit 217 has a preprocessing unit 210.
The fixed-length quantized value is output by decoding (Huffman decoding) the coded signal output from.
The dequantization unit 221 decodes the frequency component by dequantizing the quantized value output from the variable length decoding unit 217.

The inverse orthogonal transform unit 222 includes an inverse quantization unit 221.
By inverse orthogonal transforming the frequency component decoded by, the frame image is decoded in the intra mode, and the motion compensation error (residual image) which is the difference value of the pixels in the inter mode is decoded.

The adder 224 outputs the frame image (decoded image) decoded by the inverse orthogonal transform unit 222 as it is in the intra mode, and the motion compensation error (residual residual) decoded by the inverse orthogonal transform unit 222 in the inter mode. The frame image is decoded by adding the difference image) and the motion compensation image generated by the motion compensation unit 243.

The switch unit 230 is composed of a pair of switches 231 and 232 for switching the switching mode in synchronization by the switch ON / OFF control for each picture by the filter processing control unit 260.
0 is included in the loop or is excluded from the loop, that is, the processing by the in-loop filter 240 is skipped.

The in-loop filter 240 includes the switch 23.
When 1 and 232 are ON, the decoded image output from the addition unit 224 is spatially low-pass filtered in block units to generate a decoded image without block distortion. For example, if the difference between a pixel and its surrounding pixels is calculated and the difference between a pixel and its surrounding pixels is within a predetermined range, the process of replacing that pixel with the calculated average value is performed in the vicinity of the block boundary. Perform one pixel at a time.

The post-processing unit 270 includes a format conversion unit for converting the format to a predetermined spatial resolution, a screen order returning unit for returning the screens rearranged according to the picture type to the original order, and monitors the decoded image. And output to. The frame memory 241 stores a plurality of frames of decoded images output from the switch unit 230 that may be referenced.

The motion compensating section 243 performs the calculation indicated by the motion vector and block prediction type output from the pre-processing section 210 on the decoded image stored in the frame memory 241 to generate a motion compensating image.

The priority order determining section 250 has the same configuration as the priority order determining section 150 shown in FIG.
Picture type output from 0, base layer,
The priority of the picture is output according to the enhancement layer.

The filter processing control unit 260 has the same structure as the filter processing control unit 160 shown in FIG. 3, and according to the priority output from the priority determination unit 250 and the CPU operating rate obtained by monitoring. The switches 231 and 232 of the switch unit 230 are ON / OFF controlled.

Next, the operation of the image decoding apparatus 200 configured as above will be described. The image encoding device 1
For convenience of explanation in combination with 00, a case where the IPB coding mode is designated will be described.

In the intra mode for decoding the coded signal of the I picture into the frame image, the coded signal output from the preprocessing unit 210 is inverse Huffman coded by the variable length decoding unit 217 and decoded into a quantized value. Are decompressed into frequency components by inverse orthogonal transform by the inverse quantization unit 221, and decoded into frame images (decoded images) by inverse orthogonal transform by the inverse orthogonal transform unit 222. The decoded frame image (decoded image) is output by the switches 231 and 232 under the control of the filter processing control unit 260.
If it is N, the filter processing for removing the block distortion is performed by the in-loop filter 240 and then stored in the frame memory 241 and the post-processing unit 27.
At 0, the order of the images is restored, or the format is converted and output to a monitor or the like. On the other hand, when the switches 231 and 232 are turned off, the decoded image is stored in the frame memory 241 without being subjected to the filtering process, and the post-processing unit 270 determines the order of the images. Returned, converted format,
Output to monitor etc.

In the inter mode in which the coded signals of P pictures and B pictures are decoded into frame images, the coded signals output from the preprocessing section 210 are inverse Huffman coded by the variable length decoding section 217. It is decoded into a quantized value, expanded and decoded into frequency components by the inverse orthogonal transform by the inverse quantizer 221, and the inverse orthogonal transformer 22
Motion compensation error (differential image) by inverse orthogonal transform by 2
Will be decrypted.

On the other hand, the motion compensation unit 243 generates a motion compensation image (predicted image). The motion compensation unit 24
3 is a motion vector output from the preprocessing unit 210,
The operation indicated by the block prediction type is performed on the difference pixel of the reference image read from the frame memory 241, and the motion compensation image is generated.

Then, the addition unit 224 adds the motion compensation error (difference image) and the motion compensation image to decode the frame image. The decoded frame image (decoded image) has a filtering process control unit 260.
When the switches 231 and 232 are turned on under the control of 1., the post-processing unit 270 performs the filtering process for removing the block distortion by the in-loop filter 240.
In step (1), the order of the images is restored, the format is converted, and the image is output to a monitor or the like, and the decoded image having the reference Kanase is stored in the frame memory 241. On the other hand, when the switches 231 and 232 are off, the post-processing unit 2 does not perform the filtering process.
In 70, the order of the images is restored, the format is converted, the image is output to a monitor or the like, and the decoded image having reference possibility is stored in the frame memory 241.

Here, as in the case of the switches 131 and 132 of the image coding apparatus 100, the switches 231 and 23
2 is ON / OFF controlled by the filter processing control unit 260.

That is, the switch switching processing unit of the filter processing control unit 260 has the priority of the picture for each encoding of the picture and the C provided in the image decoding apparatus 200.
The operation rate of the PU is acquired, the entry to be referred to in the table for the IPB coding mode is determined, the right column of the entry is read, and whether or not the priority set for the picture type of the decoded image is in the right column to decide. If it is in the right column, the switch switching processing unit of the filter processing control unit 260 outputs a switch-on signal to the switches 231 and 232. This filters the decoded image,
The decoded image subjected to the filter processing is the frame memory 24.
It is stored in 1. On the other hand, if it is not in the right column, the switch switching processing unit of the filter processing control unit 260 outputs a switch-off signal to the switches 231 and 232. As a result, the filtering process for the decoded image is skipped, and the decoded image is stored in the frame memory 241 without performing the filtering process.

Such control is performed for each picture,
The decoded image that has been subjected to the filter processing and the decoded image that has not been subjected to the filter processing are sequentially stored in the frame memory 241.

Therefore, in the image decoding, the in-loop filter for noise removal or the like is not always applied, but the in-loop filter can be selectively applied as needed, so that the image quality is greatly affected, for example. By applying the in-loop filter only to the frame image, the image quality of the important frames stored in the frame memory is maintained and the decoded image stored in the frame memory is maintained even if the image decoding device has a small processing capacity. Block distortion is less likely to be accumulated, the prediction efficiency by the motion compensation unit is improved, image quality deterioration can be reduced as compared with the MPEG technique, and a high image quality improvement effect can be obtained at a low bit rate.

That is, a frame image having a large influence on other frame images, that is, an in-plane coded frame image, a forward predictive-coded frame image, a base layer frame image, etc. is preferentially filtered in the loop. Therefore, it is possible to more effectively obtain an image quality improvement effect such as noise removal by the in-loop filter with respect to the same increase in processing load.

Further, since the ON / OFF of the filter processing can be controlled so that the processing capability of the image decoding apparatus is fully exerted, the CPU is used with high efficiency, and the same hardware resources are used. Even with this, high-quality decoding is realized.

The present invention can be realized not only as such an image encoding device or image decoding device, but also as an image encoding method or image decoding method in which the characteristic means of these devices are taken as steps. It can also be realized as a computerized method or as a program that causes a computer to execute those steps. Needless to say, the program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

Here, a case will be described in which the above program is recorded in a storage medium such as a flexible disk so as to be easily executed in an independent computer system.

(Third Embodiment) FIG. 7 uses a flexible disk storing a program for realizing the image coding apparatus 100 of the first embodiment and the image decoding apparatus 200 of the second embodiment. It is a figure showing the case where it implements with a computer system. In particular, FIG. 7A shows an example of the physical format of a flexible disk which is the main body of the recording medium, FIG. 7B shows the external appearance of the flexible disk as viewed from the front, the sectional structure, and the flexible disk, and FIG. It is a figure which each shows the structure of the computer apparatus for recording / reproducing the program stored in the flexible disk.

The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the case F, and each track has 16 sectors Se in the angular direction. Is divided into Therefore, in the flexible disk storing the program, the image coding method and the image decoding method as the program are recorded in the area allocated on the flexible disk FD.

The above program is executed on the flexible disk F.
In the case of recording in D, the image encoding method and the image decoding method as the above programs are written from the computer system Cs via the flexible disk drive.
When the image coding method and the image decoding method are constructed in the computer system by the program in the flexible disk, the program is read from the flexible disk by the flexible disk drive and transferred to the computer system.

In the above description, a flexible disk is used as a recording medium, but an optical disk may be used as well. Moreover, the recording medium is not limited to this, and any recording medium such as a CD-ROM, a memory card, and a ROM cassette can be used as long as the program can be recorded.

In the first and second embodiments, the processing capability (CPU operation rate) is taken into consideration. However, the switch is turned on / off only by the picture type, that is, the filter processing or the skip processing is controlled. Good.

In the first and second embodiments, the filter processing is skipped when the importance of the picture is low and the CPU utilization rate is high. However, the loop filter having a lighter processing load than the loop filter 140 is used. One or more of them may be provided to perform the filtering process by the in-loop filter having a light load when the priority of the picture is low or when the CPU utilization rate is high in the case of the low priority picture. The same applies to the case of image decoding.

[0110]

As is apparent from the above description, the image coding apparatus (and the image decoding apparatus) according to the present invention is an image coding apparatus or the like for coding repeatedly input frame images, Encoding means for performing encoding by performing a predetermined conversion process on the frame image, inverse conversion means for performing an inverse conversion process of the conversion process on the frame image encoded by the encoding means, and frame image Filtering means for performing a filtering process, storage means for storing a frame image, and a frame image obtained by the inverse transforming process by the inverse transforming device after the filtering process by the filtering means Is stored in the storage means, or the frame image is stored in the storage means without performing the filtering process by the filter means. And a Gosuru control means, said encoding means, characterized in that encoding the frame image with reference to a past frame image stored in the storage means.

By this, in image coding (and image decoding), an in-loop filter such as noise removal is not always applied, but an in-loop filter can be selectively applied as necessary. For example, by applying the in-loop filter only to the frame image that greatly affects the image quality, it becomes possible to use the in-loop filter even in an image encoding device (image decoding device) having a small processing capacity. A high image quality improvement effect can be obtained at a low bit rate.

Here, the control means performs the filtering process on the frame image when the importance of the frame image is high, and the control means performs the filtering process on the frame image when the importance of the frame image is low. You may control so that the said filter process may not be performed. For example, the control unit determines that the frame image has a high degree of importance when the frame image is subjected to in-plane encoding by the encoding unit, and performs the filtering process on the frame image. If the frame image is subjected to inter-plane coding by the coding means, it is determined that the importance of the frame is low, and the filtering process is not performed on the frame image. To control
When the frame image has been subjected to forward prediction encoding by the encoding means, it is determined that the frame has a high degree of importance, the frame image is subjected to the filter processing, and the frame image When the bidirectional predictive coding is performed by the encoding means, it is determined that the importance of the frame is low, or control not to perform the filtering process on the frame image, When the frame image has been subjected to base layer encoding by the encoding means, it is determined that the frame has a high degree of importance, the frame image is subjected to the filter processing, and the frame image is obtained. Is that the enhancement layer encoding has been performed by the encoding means, it is determined that the importance of the frame is low, and May be controlled not subjected to the filtering process on the frame image.

As a result, a frame image having a large influence on other frame images, that is, an intra-frame coded frame image, a forward prediction coded frame image, a base layer frame image, etc., is preferentially placed in the loop. Since the filter is applied, it is possible to more effectively obtain the image quality improving effect such as noise removal by the in-loop filter with respect to the same increase in the processing load.

Further, the control means monitors a margin regarding the processing capacity of the image coding apparatus (image decoding apparatus), and if there is a margin, performs a filtering process on a frame image of lower importance. If there is no margin, control may be performed not to perform the filtering process on frame images of higher importance. For example, the frame image is associated with a priority order corresponding to the degree of importance, and the control unit monitors the availability by monitoring the operation rate of the CPU included in the image encoding device, and the operation is performed. When the rate is high, the filter processing may be performed only on the frame images having a high priority, and when the operation rate is low, the frame images having a low priority may be subjected to the filter processing.

As a result, ON / OFF of the filter processing can be controlled so that the processing capability of the image encoding device (image decoding device) is fully exerted, so that the CPU is used with high efficiency. This means that high-quality encoding (decoding) can be realized even with the same hardware resource.

As described above, according to the present invention, high-quality image coding and image decoding can be realized at a low bit rate (high compression rate), and in particular, image coding by software under limited hardware resources. It can be said that the practical value is extremely high in the present day when the information communication technology and the computer have been widely spread due to the high image quality improving effect in the digitalization processing and the image decoding processing.

[Brief description of drawings]

FIG. 1 is an image coding device 1 according to a first embodiment of the present invention.
It is a block diagram showing a functional configuration of 00.

2 is a block diagram showing a detailed functional configuration of a priority order determination unit 150 shown in FIG.

3 is a block diagram showing a detailed functional configuration of a filter processing control unit 160 shown in FIG.

4 is a diagram showing a reference relationship of frame images stored in a frame memory 141 shown in FIG.

5 is a flowchart showing a switch driving process executed by a switch switching processing unit 165 shown in FIG.

FIG. 6 is an image decoding device 2 according to Embodiment 2 of the present invention.
It is a block diagram showing a functional configuration of 00.

FIG. 7 is a diagram showing a case where a flexible disk storing a program for realizing the image encoding device and the image decoding device is used to implement the program by a computer system.

FIG. 8 is a block diagram showing a functional configuration of a conventional image encoding device.

FIG. 9 is a block diagram showing a functional configuration of a conventional image decoding device.

[Explanation of symbols]

100 image coding apparatus 112 Subtraction unit 113 Orthogonal transformation unit 114 quantizer 117, 217 Variable length coding unit 121,221 Inverse quantizer 122, 222 inverse orthogonal transform unit 124,224 adder 130,230 switch 131,132,231,232 switches 140,240 In-loop filter 141,241 frame memory 142 Motion detector 143, 243 Motion compensation unit 150,250 Priority decision section 160, 260 Filter processing control unit 200 Image Coding Device

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Makoto Hakai             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. (72) Inventor Kiyoshi Abe             1006 Kadoma, Kadoma-shi, Osaka Matsushita Electric             Sangyo Co., Ltd. F term (reference) 5C059 KK03 KK15 MA00 MA05 MA14                       MA23 MA32 MC11 MC38 ME02                       NN01 NN21 PP04 SS10 TA69                       TB04 TC00 TC27 TD15 UA02                       UA05 UA16 UA33 UA38 UA39                 5J064 AA02 AA03 BA16 BB01 BB03                       BC01 BC11 BC27 BD03

Claims (22)

[Claims] 1. An image encoding apparatus for encoding a frame image repeatedly input, comprising: an encoding unit that performs encoding by performing a predetermined conversion process on the frame image; and an encoding unit that performs the encoding. Inverse conversion means for performing the inverse conversion processing of the conversion processing on the generated frame image, filtering means for performing the filtering processing on the frame image, storage means for storing the frame image, and inverse conversion processing by the inverse conversion means The frame image obtained in step 1 is stored in the storage means after being subjected to the filter processing by the filter means, or the frame image is stored without being subjected to the filter processing by the filter means. Control means for controlling so that the frame is stored in the storage means, and the encoding means stores the past frames stored in the storage means. An image encoding apparatus, wherein the frame image is encoded while referring to the image. 2. The control means performs the filtering process on the frame image when the importance of the frame image is high, and performs the filtering process on the frame image when the importance of the frame image is low. The image coding apparatus according to claim 1, wherein the image coding apparatus is controlled so that the filtering process is not performed. 3. The encoding means performs encoding including in-plane encoding and inter-plane encoding on the frame image, and the control means causes the frame image to be in-plane encoded by the encoding means. If the frame has been encoded, it is determined that the importance of the frame is high, the frame image is subjected to the filtering process, and the frame image is subjected to the inter-plane encoding by the encoding means. 3. The image coding apparatus according to claim 2, wherein the frame is judged to have a low degree of importance in the case where the frame is skipped, and the frame image is controlled not to be subjected to the filtering process. 4. The encoding means performs encoding including forward predictive encoding and bidirectional predictive encoding on the frame image, and the control means controls the frame image by the encoding means. If forward predictive coding has been performed,
If it is determined that the frame has a high degree of importance, the frame image is subjected to the filter processing, and the frame image is bidirectionally predictive coded by the coding unit, the frame The image coding apparatus according to claim 2, wherein the image coding apparatus determines that the degree of importance is low and controls the frame image so as not to perform the filtering process. 5. The encoding means adds base layer encoding and enhancement to the frame image.
Performing encoding including layer encoding, the control means, when the frame image has been subjected to base layer encoding by the encoding means, determines that the importance of the frame is high, When the frame image is subjected to the filter processing and the frame image has been subjected to enhancement layer encoding by the encoding means, it is determined that the importance of the frame is low, and the frame image The image coding apparatus according to claim 2, wherein the image coding apparatus is controlled so that the filter processing is not performed on the. 6. The control means monitors a margin regarding the processing capacity of the image coding apparatus, and if there is a margin, performs a filtering process on a frame image of lower importance, and if there is no margin. 3. The image coding apparatus according to claim 2, wherein the control is performed so that a frame image having a higher degree of importance is not filtered. 7. The frame image is associated with a priority order corresponding to the degree of importance, and the control means monitors the margin by monitoring an operation rate of a CPU included in the image encoding device. When the operation rate is high, only the frame image with a high priority is subjected to the filter processing, and when the operation rate is low, the frame image with a low priority is subjected to the filter processing. The image encoding device according to claim 6. 8. The frame image belongs to any one of a plurality of types, and the control unit associates the type of the frame image with the priority.
An information table and a second information table indicating a combination of the priority and the operating rate when the filtering process is performed, and the first information table based on a type of a target frame image.
The priority of the frame image is determined by referring to the information table, and the frame image is filtered by the second information table based on the combination of the determined priority and the operating rate. The image coding apparatus according to claim 7, wherein it is determined whether or not to perform processing. 9. The control unit has a plurality of the first information tables and a plurality of the second information tables, and refers to one first information table selected from the plurality of the first information tables. Whether to apply the filtering process to the frame image by determining the priority of the frame image by referring to one second information table selected from the plurality of second information tables. 9. The image encoding device according to claim 8, wherein 10. An image decoding apparatus for decoding a frame image that is repeatedly input, the decoding means performing decoding by performing a predetermined conversion process on the frame image, and performing a filtering process on the frame image. Filter means, storage means for storing a frame image, and a frame image obtained by the decoding by the decoding means, after the frame processing is performed by the filter means, the frame image is stored in the storage means. Or a control unit for controlling the frame image to be stored in the storage unit without performing a filtering process by the filter unit, wherein the decoding unit stores the past image stored in the storage unit. An image decoding apparatus, characterized in that the frame image is decoded while referring to the frame image. 11. The control means performs the filtering process on the frame image when the importance of the frame image is high, and controls the frame image when the importance of the frame image is low. The image decoding apparatus according to claim 10, wherein the image decoding apparatus is controlled so that the filtering process is not performed. 12. The decoding means performs decoding including in-plane decoding and inter-plane decoding on the frame image, and the control means controls the frame image to be in-plane by the decoding means. If the frame has been decoded, it is determined that the importance of the frame is high, the filtering process is performed on the frame image, and the frame image is subjected to the inter-plane decoding by the decoding unit. 12. The image decoding apparatus according to claim 11, wherein the frame is judged to be of low importance when the frame is skipped, and the frame image is controlled not to be subjected to the filtering process. 13. The decoding means performs decoding including forward predictive decoding and bidirectional predictive decoding on the frame image, and the control means controls the frame image by the decoding means. If forward predictive decoding has been performed,
When it is determined that the frame is of high importance, the filtering process is performed on the frame image, and the frame image is backward predictive decoded by the decoding unit, the importance of the frame is determined. The image decoding apparatus according to claim 11, wherein it is determined that the degree of frequency is low, and control is performed so that the filtering process is not performed on the frame image. 14. The decoding means performs decoding including base layer decoding and enhancement layer decoding on the frame image, and the control means controls the frame image by the decoding means. When the base layer decoding has been performed, it is determined that the frame has a high degree of importance, the filtering process is performed on the frame image, and the frame image is enhanced by the decoding unit. The image according to claim 11, wherein when the frame has been decoded, it is determined that the importance of the frame is low, and control is performed so that the filtering process is not performed on the frame image. Decoding device. 15. The control means monitors a margin of processing capacity of the image decoding apparatus, and if there is a margin, performs a filtering process on a frame image of lower importance, and if there is no margin, 12. The image decoding apparatus according to claim 11, wherein the control is performed so that a frame image having a higher degree of importance is not filtered. 16. The frame image is associated with a priority order corresponding to the degree of importance, and the control unit monitors the margin by monitoring an operating rate of a CPU included in the image decoding apparatus. When the operation rate is high, only the frame image with a high priority is subjected to the filter processing, and when the operation rate is low, the frame image with a low priority is subjected to the filter processing. The image decoding device according to claim 15. 17. The frame image belongs to any one of a plurality of types, and the control unit associates the type of the frame image with the priority level.
An information table and a second information table indicating a combination of the priority and the operating rate when the filtering process is performed, and the first information table based on a type of a target frame image.
The priority of the frame image is determined by referring to the information table, and the frame image is filtered by the second information table based on the combination of the determined priority and the operating rate. The image decoding apparatus according to claim 16, wherein it is determined whether or not to perform processing. 18. The control means has a plurality of the first information tables and a plurality of the second information tables, and refers to one first information table selected from among the plurality of first information tables. Whether to apply the filtering process to the frame image by determining the priority of the frame image by referring to one second information table selected from the plurality of second information tables. 18. The image decoding device according to claim 17, wherein 19. An image encoding method for encoding an image by referring to an image obtained by performing a filtering process on an encoded image, wherein the encoded image of the frame to be encoded is high when the importance of the frame is high. The image coding method is characterized in that a filtering process is performed on the frame, and when the importance of the frame to be coded is low, the coded image of the frame is not filtered. 20. An image decoding method for decoding a coded image by referring to an image obtained by performing a filtering process on a decoded image, wherein when the frame to be decoded has a high importance, An image decoding method characterized in that a decoded image is subjected to a filter process, and when the decoded frame has a low importance, the decoded image of the frame is not subjected to the filter process. 21. A program for an image encoding device that encodes a frame image that is repeatedly input, wherein the image encoding device according to any one of claims 1 to 9 includes a computer as a means. A program characterized by making a function of. 22. A program for an image decoding device that decodes a frame image that is repeatedly input, wherein the image decoding device according to any one of claims 10 to 18 is provided with a computer. A program characterized by making a function of.