DE60203492T2 - Image Coding Methods and Image Decoding Methods - Google Patents

Abstract

A picture coding apparatus (300) is a picture coding apparatus that codes a difference between picture data (Img) representing an input picture and predictive picture data (Pred) representing a predictive picture for the input picture and generates coded picture data, and includes a picture decoding unit (104) for decoding coded picture data (Img) after the picture data (Img) is coded; inter pixel filters A and B (303 and 304) for performing inter pixel filter operation for decoded picture data (Recon) obtained by the picture decoding unit (104); switches (301 and 302) for selecting one of the inter pixel filters; and an inter picture predicting unit (108) for generating the predictive picture data (Pred) for the input picture data (Img) using filtered decoded picture data (FilteredImg1), as reference picture data (Ref), obtained by the selected inter pixel filter.

Description

technical area

The The present invention relates to a picture coding method Image decoding method, an image coding apparatus, a Image decoding apparatus, a program for executing such Method in the form of software and a recording medium, on which the program is recorded.

technical background

With The development of multimedia applications is the last Years usual become information of all kinds of media, such as audio, video and text to handle in an integrated way. This will it is possible to handle the media integrated by digitizing all media. However, since digitized images have an enormous amount of data, Information compression techniques are an absolute necessity for their Storage and transmission. On the other hand, for interoperation of compressed image data, one is Standardization of compression techniques is also important. Standards for Image compression include H.261 and H.263, recommended by ITU-T International Telecommunication Union Telecommunication Standardization Sector and MPEG (Moving Picture Experts Group) -1, MPEG-2 and MPEG-4 ISO (International Organization for Standardization).

1 Fig. 10 is a block diagram showing a construction of a conventional picture coding apparatus 100 shows. It should be noted that in the present invention, the unit consisting of one page of a picture is referred to as an image. In a nested image signal, an image denotes a field or a frame, and in a progressive image signal, an image denotes a frame.

The picture coding device 100 includes a difference calculator 101 a picture coding unit 102 , a variable-length coding unit 103 , an image coding unit 104 , an adder 105 , an inter-pixel filter 106 , a frame store 107 , an inter-picture prediction unit 108 and an inter-image prediction estimation unit 109 , The difference calculator 101 subtracts a predictive image input from a negative-sign input terminal from an image input to a positive-sign input terminal to output the difference image. The picture coding unit 102 encodes the entered difference image. For example, the image coding unit encodes 102 the input data by performing a frequency transformation thereof using DCT or the like and quantizing the frequency data as the transformation result. The variable-length coding unit 103 performs a variable length coding of the coded difference image and predictive parameters from the inter-image prediction estimation unit 109 thereto, and adds relevant data, such as a header describing information relevant to the resultant coded data, formats it into an output coded bit stream, and outputs it outside the picture coding apparatus 100 out. The picture decoding unit 104 decodes that through the picture coding unit 102 coded difference image by performing processing opposite to the coding by the image coding unit 102 for the difference image. For example, the image decoding unit performs 104 after performing the inverse quantization of the coded difference image, inverse frequency transformation such as inverse DCT to decode the difference between the input image and the predicted image. The adder 105 adds the decoded difference image and the predicted image to decode the input image. The inter-pixel filter 106 performs filtering such as the suppression of coding noise in the high-frequency components of the decoded input image. The image memory 107 holds the image data of at least one image out of the one by the adder 105 decoded pictures as a reference picture. The inter-image prediction unit 108 reads a predictive image from the reference image in the image memory 107 based on the by the inter-image prediction estimation unit 109 obtained prediction result. The inter-image prediction estimation unit 109 derives a predictive parameter PredParam, which is the amount of change in the movement of the input image relative to the reference image.

Specifically, image data Img is externally input to the image coding apparatus 100 entered. The image data Img is input to the input terminal of the differential calculator 101 entered. The difference calculator 101 calculates the difference between the pixel values of these image data Img and the predictive image data Pred inputted to the negative-sign input terminal to output the result as difference image data Res. These predictive image data Pred are obtained in the following manner. First, reference image data Ref, that is, an image of an already coded image which has already been coded once and then decoded to be an image of an image, is stored in the image memory 107 saved. Next, from this reference image data Ref, based on the predictive parameter PredParam, data is extracted representing an image corresponding to each block in the inputted image data Img. The data representing this block of each block is the predictive image data Pred. The picture coding device 100 stores plural pages of coded pictures as reference picture data Ref for prediction in the picture memory 107 and the inter-image prediction unit 108 generates predictive image data from the Pred in the image memory 107 stored reference image data Ref. The inter-image prediction estimation unit 109 obtains predictive parameter data PredParam (for example, motion vector information used in the MPEG image coding method or the like) from the image data Img and the reference image data Ref. It is to be noted that pixel values of a predictive image in the case of intra-picture coding are "0" should.

The picture coding unit 102 encodes differential image data Res and outputs them as encoded reference image data CodedRes. The picture decoding unit 104 decodes the encoded difference image data CodedRes and outputs them as decoded difference image data ReconRes to be used in a reference image for inter-image prediction. The pixel values indicated by these decoded difference image data ReconRes and values indicated by the predictive image data Pred are provided by the adder 105 added and output as decoded image data Recon. The inter-pixel filter 106 performs filter operation processing on the decoded recon and stores it as filtered decoded image data FilteredImg in the image memory 107 ,

The inter-pixel filter 106 has the effect of reducing the coding noise of the decoded image data Recon and improving the predictive efficiency when the image is used as a reference image. As an example of the inter-pixel filter 106 ITU-T recommends an H.261 loop filter. The filtered decoded pixel data FilteredImg obtained by the inter-pixel filtering operation of the inter-pixel filter 106 are executed in the image memory 107 stored and used as a reference image when the following images are encoded. The variable-length coding unit 103 executes a variable-length coding of the coded difference image data CodedRes and the predictive parameter data PredParam, and combines them into a coded data bit stream for outputting the result outside the image coding apparatus. 2 Fig. 10 is a block diagram showing the structure of a conventional picture decoding apparatus 200 shows. The image decoding device 200 includes a variable-length decoding unit 201 a picture decoding unit 202 , an adder 203 , an inter-pixel filter 204 , a frame store 205 and an inter-image prediction unit 206 , The bitstream of coded data is transferred from outside to the picture decoding apparatus 200 entered. The variable-length decoding unit 201 performs variable length decoding of the input bitstream of encoded data and separates it into coded differential image data CodedRes and predictive parameter data PredParam. The picture decoding unit 202 decodes the encoded difference image data CodedRes and outputs them as decoded difference image data ReconRes. It is to be noted that an image designated as an image output as the decoded difference image data ReconRes, that is, an image corresponding to the reference image data Ref in the image encoding device 100 , already decoded and in the image memory 107 was saved. Therefore, the inter-image prediction unit generates 206 Predictive image data Pred from the reference image data Ref based on the predictive parameter data PredParam. The adder 203 adds the predictive image data Pred and the decoded difference image data ReconRes to output the result as decoded image data Recon. The inter-pixel filter 204 performs an inter-pixel filtering operation on the decoded image data Recon and outputs the result as filtered decoded image data FilteredImg to the outside of the image decoding apparatus 200 out. Outside the picture decoding device 200 means a display device such as a TV. And the filtered decoded image data FilteredImg are stored in the image memory 205 stored and referred to as reference image data Ref for the following images.

At the Consider the case of the transfer moving pictures by a mobile phone or the like, however, it is desirable that Energy consumption of the mobile device as a whole, to minimize the available time per charge extend, so it for the device not wanted is, a high processing ability to show which a larger amount of energy required for image processing. Therefore, when using a mobile device, which only one processing unit with low processing capability for image processing sometimes, on which there is only one inter-pixel filter can use which one for a low processing load is needed. Meanwhile, will sometimes required to use a high-performance inter-pixel filter for some applications which is a transmission moving pictures of high quality and require a high coding efficiency, even if an operating unit with high processing load is required. If a coding process is able to respond flexibly to such requirements, It is in the expanding range of applications in one Picture encoding device and picture decoding device according to such a coding method and decoding method.

Japanese Patent JP06 / 038197A be draws on coding and decoding of picture signals. In this document a filter control is disclosed. The filter controller calculates an absolute sum of image data and determines whether the buzzer is greater or less than a threshold. According to this determination, a control signal is output to switch a loop filter for a prediction picture between different conditions.

The Abstract of Japanese Patent JP 03-136586 discloses a Device for efficiently removing noise, which is in an inter-frame prediction coding loop by applying ON / OFF control of a bank limiting filter, a bank limit to an inter-frame prediction decode signal apply. The band limitation is applied when many picture elements with a larger amplitude difference (Distortion) between an input signal and a decoding signal.

The US Patent No. 6,249,610B1 discloses an image coding apparatus, in which a picture reproduction section in the latter part of the Picture encoding device is provided. Furthermore, there is noise captured in a reproduced image and a suitable filter to correct the reproduced image is selected. The Picture coding and the associated Filtering is done in the form of a computer method using Implemented recording media.

The European Patent Application 0 603 878 A2 discloses an interframe coding system, which in a picture signal contained higher frequency components with an adaptive filter provided in a coding loop effectively and adaptively eliminated.

The The present invention has been made in view of these conventional problems conceived and intended to provide a picture coding device, which an inter-pixel filter selectively depending on different situations used to generate encoded data and an image decoding device, which decodes the encoded data.

epiphany the invention

The Image decoding / coding method according to the present invention, As set forth in claim 1, various image data decode / encode a difference between input image data which is an input image data Picture picture, and predictive Image data, which is a predictive Picture for represent an image in the input image, to be encoded To generate image data.

The Image decoding / encoding method according to the present invention are able to find a suitable inter-pixel filter depending on it suitable to switch whether an image is addressed or not, about a predictive Create picture. Although the picture coding method is the inter-pixel filter the picture decoding method may also undergo decoding Use of a corresponding inter-pixel filter flawlessly To run.

Thereby is the inter-pixel filtering, such as noise removal, at the Image encoding (or image decoding) is not always done, but you can selectively run as needed so it too for an image coding apparatus (or a picture decoding apparatus) with a lower processability becomes an inter-pixel filter by accepting the Inter-pixel filtering only for an image that has a significant impact on image quality, and Thus, the effect of significantly improving the picture quality at a low bit rate can be obtained.

Here you can run the controller so that the filtering is performed when a level of significance of the image is high and filtering not executed when the significance level of the image is low. For example you can decide that the significance level of the image is high is and run a controller like this that filtering for the Picture executed when the picture is coded by the coding unit intra-picture is, and decide that the significance level of the image is low is and run a controller like this that filtering is not for a picture is executed when the picture is encoded by the inter-picture coding unit is. Or you can decide that the significance level of the image is high and the controller will do that filtering for the image accomplished when the picture is predictively coded by the coding unit is, and can decide that the significance level of the image is low and the controller will do that filtering for the image not executed when the image is bi-predictively coded by the coding unit is. Or you can decide that the significance level of the image is high and perform a control so that filtering for the image accomplished is when the image is encoded by the encoding unit base layer is, and can decide that the significance level of the image is low and the controller will do that filtering for the image not executed when the picture is expanded layer-coded by the coding unit is.

This implements inter-pixel filtering for an image having a significant impact on other images, that is, an intra-image encoded one Image, a forward predictive-coded image, a base-layer image, or the like by priority, so that the effect of improving image quality such as noise removal can be more efficiently obtained by an inter-pixel filter, even with the same increase in processing load.

As described above, according to the present Invention the picture coding and picture decoding at a low Bitrate (high compression rate) realized for a high image quality and in particular the significant effect of improving image quality may be at Image encoding processing and image decoding processing by software be realized under limited hardware resources and thus can be said that the practical value of the present invention currently extraordinarily is high, since information communication technology and computer are widespread.

Summary the drawings

1 Fig. 10 is a block diagram showing the construction of a conventional picture coding apparatus.

2 Fig. 10 is a block diagram showing the construction of a conventional picture decoding apparatus.

3 Fig. 10 is a block diagram showing the construction of an image coding apparatus according to a first example.

4 Fig. 10 is a block diagram showing the construction of an image coding apparatus which switches an inter-pixel filter for each slice of an input image.

5A Fig. 12 is a diagram showing the stream construction of a coded data bit stream output from the image coding apparatus of the present invention. 5B Fig. 12 is a diagram showing the stream construction of a coded data bit stream which is output when the image coding apparatus of the present invention switches a pixel filter in the unit of a disk.

6 FIG. 15 is a block diagram showing the construction of a picture decoding apparatus which decodes the coded data Bitstream1 generated by the picture coding apparatus according to the first embodiment.

7 Fig. 10 is a block diagram showing the construction of a picture decoding apparatus which uses a built-in inter-pixel filter instead of a certain inter-pixel filter when it is not included in the picture decoding apparatus.

8th Fig. 10 is a block diagram showing the construction of an image coding apparatus according to a first embodiment of the present invention.

9 FIG. 13 is a diagram showing details of operation with a deblocking filter which is an example of the inter-pixel filter as in FIG 8th shown. A is a diagram showing pixel values near the boundary between blocks before filtering. B is a diagram showing pixel values near the boundary between blocks after filtering.

10 Fig. 10 is a flowchart showing a flow of filter processing by an inter-pixel filter.

11 Fig. 10 is a block diagram showing the construction of a picture decoding apparatus which can select whether to perform inter-pixel filter processing or not.

12 Fig. 10 is a block diagram showing the construction of a picture decoding apparatus equipped with an inter-pixel filter which can further select an output stage.

13 Fig. 10 is a block diagram showing the construction of a picture decoding apparatus which can select an inter-pixel filter depending on the picture type of each picture.

14 Fig. 10 is a block diagram showing the functional construction of a picture coding apparatus according to a second embodiment of the present invention.

15 FIG. 12 is a block diagram showing the detailed functional structure of a priority determination unit as in FIG 14 shown, shows.

16 FIG. 15 is a block diagram showing the detailed functional structure of a filter processing control unit as shown in FIG 14 shown, shows.

17 FIG. 15 is a diagram showing reference relationships between images stored in an image memory such as 14 shown, shows.

18 FIG. 10 is a flowchart showing an activation processing which is performed by an as in FIG 16 shown switching processing unit is executed.

19 Fig. 10 is a block diagram showing the functional construction of a picture decoding apparatus according to a third embodiment of the present invention.

20 Fig. 12 is an explanatory diagram in the case of implementation by a computer system using a floppy disk on which the image coding method or the picture decoding method of the above-mentioned first embodiment to third embodiment is stored. A shows an example of a physical format of a floppy disk which is a recording medium body. B shows the front view of the appearance of the floppy disk, the cross-sectional view thereof and the floppy disk, and C shows the construction for recording and reproducing the above program on the floppy disk FD.

21 Fig. 10 is a block diagram showing the overall configuration of a content providing system that provides content distribution.

22 Fig. 13 is a diagram showing an example of the appearance of a cellular phone.

23 is a block diagram showing the structure of the mobile phone.

24 Fig. 12 is a diagram explaining apparatuses which perform the coding processing or decoding processing as shown in the above-mentioned embodiments and a system using these apparatuses.

Best mode for running the invention

(First example)

The following explains concrete examples and embodiments of the present invention with reference to the figures. 3 Fig. 10 is a block diagram showing the construction of an image coding apparatus 300 according to the first example of the present invention. In this figure, the same reference numerals are assigned to the same elements and data as those of Figs 1 shown conventional image coding apparatus 100 and their explanations are omitted since they have already been explained. Likewise, in the following figures the same reference numerals are assigned to the elements and data which have already been explained and their explanations are omitted. The picture coding device 300 includes the difference calculator 101 , the picture coding unit 102 , the variable-length coding unit 305 , the picture decoding unit 104 , the adder 105 , the image memory 107 , the inter-image prediction unit 108 , the inter-picture prediction estimation unit 109 , a switch 301 , a switch 302 , an inter-pixel filter A 303 and an inter-pixel filter B 304 , The desk 301 and the switch 302 are switches to the corresponding selected connection with either the terminal 1 or the connection 2 depending on a value of the filter type information Filter-Type1. The desk 301 is between the output terminal of the adder 105 and the input terminals of the inter-pixel filter A 303 and the inter-pixel filter B 304 intended. And the switch 302 is between the input port of the image memory 107 and the output terminals of the inter-pixel filter A 303 and the inter-pixel filter B 304 intended. The inter pixel filter A 303 and the inter-pixel filter B 304 For example, deblocking filters for smoothing high-frequency noise near the boundary between blocks for removing the block distortion, for example, and their smoothing levels are different. Furthermore, their operational processing loads for smoothing are different according to the smoothing levels. It should be noted that the elements shown in this figure, such as the switch 301 and the switch 302 can be implemented either as hardware or as software. This example also applies to other figures.

The variable-length coding unit 305 executes a variable-length coding, for example, a suffix coding of the inputted filter type information FilterType1, the coded difference image data CodedRes and the predictive parameter data PredParam, and adds them to a coded data bit stream 1 for outputting the coded data outside the image coding apparatus 300 together.

The operation of the image encoding apparatus constructed as above 300 In particular, a part having a new structure will be described in more detail by comparing with the conventional picture coding apparatus 100 explained. First, the filter type information FilterType1 is externally input to the image coding device 300 entered. The input from outside here means, for example, a user input using a user interface such as a keyboard from outside the picture coding device or data set for a device, and a value determined by the device depending on a bit rate (compression rate) or picture size. This filter type information FilterType1 will be in the switch 301 and the switch 302 entered. The desk 301 and the switch 302 switch to either "Port 1" or "Port 2", depending on the value of this filter type information FilterType1. For example, if the value of the filter type information FilterType1 is "1", the switch will turn on 301 and the switch 302 both to port "1". In In this case, the filtering by the inter pixel filter A 303 on the adder 105 output decoded data Recon applied. If the value of the filter type information FilterType1 is "2", the switch will turn on 301 and the switch 302 both to the port "2" side and filtering by the inter-pixel filter B 304 gets to that of the adder 105 output decoded image data Recon applied. The filtered, decoded image data FilteredImg1, which by the inter-pixel filter A 303 or the inter-pixel filter B 304 are filtered in the image memory 107 stored and used as a reference picture for predictive coding of the following pictures. Also, the filter type information FilterType1 indicating an inter-pixel filter is used together with the coded difference image data CodedRes and the predictive parameter data PredParam of the scan image to perform variable-length coding of this data into the variable-length coding unit 305 entered. The variable length coding result of the filter type information FilterType1 is stored in the coded data Bitstream1 by assigning the variable length coding results of these coded differential image data CodedRes and predictive parameter data Pred-Param, and recorded on a recording medium or transferred to a picture decoding apparatus. In other words, the type of the inter-pixel filter applied to the decoded image data Recon of each image is notified to the image decoding apparatus by the filter type information FilterType1 stored in the coded data Bitstream1. Therefore, since the picture decoding apparatus that decodes the coded data Bitstream1 can determine the inter-pixel filter which refers to the decoded picture data Recon of each picture in the picture coding apparatus 300 is applied, they use the same filter for the decoded picture of each picture. It is to be noted that although the value of the filter type information FilterType1 in the above explanation is "1" or "2", this is a value defined for the sake of explanation only and any other value can be used when a plurality of filters by such Values can be distinguished.

As described above, the image coding device 300 , including a plurality of inter-pixel filters having different prediction performance and processing load as an inter-pixel filter, using them by switching depending on the externally input filter type information FilterType1. There are the following advantages in using the inter-pixel filters with different prediction performance and processing load by switching them. First, for the explanation, it is assumed that the inter pixel filter A 303 has a lower processing load than the inter-pixel filter B 304 while the inter-pixel filter B 304 has higher noise suppression effect than inter-pixel filter A 303 to improve the efficiency of predictive coding. As an image decoding device, which is the one of the encoding device 300 of the present invention, two types of image decoding apparatuses are adopted: an image decoding apparatus A including only the inter-pixel filter A; 303 and a picture decoding apparatus B including the inter-pixel filter A 303 and the inter-pixel filter B 304 , The former image decoding apparatus A, which requires less processing load, is suitable for a device having a lower processing capability. The latter image decoding apparatus B is suitable for a device having a higher processing capability. The latter picture decoding apparatus B may also be the one using either the inter-pixel filter A 303 or the inter-pixel filter B 304 In this case, the picture coding device can decode and output coded data which is upwardly compatible with the former picture decoding device A. 300 act as an image coding device that supports both of these two types of image decoding devices. In other words, by constructing the picture coding device 300 such that the inter-pixel filter is selected with proper predictive power and processing load depending on the processing capability of the target picture decoding apparatus, the coding method can be performed using the same inter-pixel filter as in the picture coding apparatus 300 be applied to a wide variety of devices (to decode the encoded data bitstream1).

The inter-pixel filter may also depend on the processing capability of the image encoding device 300 not only for generating the coded data depending on the processing capability of the image decoding apparatus but also for other uses. For example, when an image size and a frame rate to be encoded are large, the processing load required for the entire encoding processing becomes large. Therefore, the inter-pixel filter B 304 used with a required high processing capability, when the image size to be encoded and the frame rate are fixed values or less, and the inter-pixel filter A 303 with a required lower processing capability is used when the image size to be encoded and the frame rate are fixed values or more in order to reduce the processing load required for the entire encoding processing. Or, if the image coding is realized by a timeshare system in which a device performs a plurality of timeshare processes, there is a possibility that the processing load, which associated with the image encoding, changes dynamically under the influence of other processes. Therefore, when the processing load associated with the image coding is a fixed value or more, the inter pixel filter B becomes 304 used with a high processing load, while when the processing load associated with the image coding is a fixed value or less, the inter pixel filter A 303 with lower processing load than the inter-pixel filter B 304 can be used.

To the Timing of switching the inter-pixel filter can use this by providing a plurality of inter-pixel filters, which for Images with certain characteristics are suitable on a frame by frame basis dependent be switched from the property of each picture. If for example Edge information is important, as in the case of characters, becomes one Inter-pixel filter used with good edge retention. You can go through an automatic assessment using image processing techniques such as edge detection and character recognition are switched, or a user may explicitly use an inter-pixel filter suitable for natural images, one for Sign suitable inter-pixel filter and one suitable for borders Select inter-pixel filter. When a plurality of inter-pixel filters are switched in this manner can be one for the image properties suitable filter can be selected and thus the Prediction efficiency will be further improved. In other words it is also effective to switch filters such that the picture quality improved, not dependent on them to switch from the processing load. In the present example was a switch of filters depending on the processing load explains but you can also be switched to improve picture quality.

In addition is the unit of switching of inter-pixel filters does not occur a picture-by-picture basis limited, but inter-pixel filters can in the unit of a smaller picture area than a picture such as a disc, a macroblock and an MPEG block, or in the unit of an area including at least one pixel, because image properties may vary in the part of an image.

4 is a block diagram showing the structure of an image coding device 400 shows which inter-pixel filter toggles at each slice of an input image. The picture coding device 400 is an image coding apparatus that filters a decoded image by switching inter-pixel filters in the unit of a slice MPEG and includes the difference calculator 101 , the picture coding unit 102 , the picture decoding unit 104 , the adder 105 , the image memory 107 , the inter-picture prediction unit 108 , the inter-picture prediction estimation unit 109 , a switch 403 , a switch 404 , the inter-pixel filter A 303 , the inter-pixel filter B 304 , the variable-length coding unit 305 , a filter switching position judgment unit 401 and a switch 402 , The filter switching position judgment unit 401 detects slices of an input image in externally input image data Img, and gives to the switch 402 Switch control data SetFType for outputting a pulse, for example every time the detected slices are switched. The desk 402 is a switch which interrupts between terminals while the filter switch control data SetFType are not outputted and brings the externally input filter type information FilterType1 in connection with the switch 403 and the switch 404 for a transition torque while outputting the filter switching control data SetFType. The desk 403 and the switch 404 respectively connect their terminals "1" or terminals "2" depending on the value of the filter type information FilterType1 which is inputted immediately every time the slices of the input image data Img are changed and maintains the connection state. In other words, while the switch 402 is disconnected, the inter-pixel filters are not switched. As a result, a new inter-pixel filter corresponding to the filter type information FilterType1 is selected each time the slices of the input image data Img are switched, and thus the inter-pixel filters in the center of the slice can be prevented from switching.

As explained above it becomes according to the picture coding device the present invention possible encoded data using an inter-pixel filter depending on the processability an image decoding apparatus that generates the images of the Image coding apparatus of the present invention issued restores encoded data. The inter-pixel filter may also depend on the processability the picture coding device are selected.

It It should be noted that although the image coding apparatus in the present example has two inter-pixel filters, they three or more inter-pixel filters may have. In the same way as the present example becomes each one of the three or more inter-pixel filters selected and and the type of inter-pixel filter used indicating filter type information may be included in the coded data.

In addition, filters can be switched to improve the picture quality and not dependent on the processing load to be switched.

It should be noted that in the image coding apparatus 400 the filter switching control data SetFType are a pulse waveform indicating the value "1" every switching of detected slices and "0" during the period other than the switching torque, but the present invention is not limited to such a case and may be, for example, a rectangular wave which is inverted each time a disk is switched, or any other waveform. The desk 402 Further, a switch which interrupts between the terminals while the filter switching control data SetFType are in the same phase, but the present invention is not limited to such a case and the filter switching position judging circuit 401 For example, the filter switching control data SetFType may output the value for interrupting the switch 402 indicates at positions other than the filter switching position. In addition, in the picture coding apparatus 400 the inter-pixel filters are switched on a slice-by-slice basis of the image data Img, but the inter-pixel filters may be switched on a frame-by-frame basis or may include a block, a macroblock, or in-unit a fixed number of pixels are switched.

5A Fig. 12 is a diagram showing the stream construction of a coded data bit stream output from the picture coding apparatus of the present invention. 5B Fig. 15 is a diagram showing the stream construction of a coded data bit stream which is output when the picture coding apparatus of the present invention switches inter-pixel filters in the unit of a disk. The feature of the encoded data of the present invention is that the encoded data bit stream includes a filter type information FilterType which determines one of a plurality of inter-pixel filters. By virtue of this stream construction, the image decoding apparatus of the present invention which decodes the coded data bit stream can use the same inter pixel filter as that used for coding by checking the filter type information FilterType contained in the bit stream of coded data.

In the bitstream of encoded data, as in 5A 1, the value of the filter type information FilterType indicating the inter-pixel filter used for filtering is in (a diagonally hatched area of, for example) the header 901 which is attached to the entire bitstream of coded data. This coded data bit stream corresponds to the coded data Bitstream1 output from the picture coding device 300 , as in 3 shown. Also in the bitstream of encoded data, as in 5B 1, the value of the filter type information FilterType indicating the inter-pixel filter used to filter the disk is in (a diagonally hatched area, for example,) of the disk header provided in each disk 902 described. This coded data bit stream corresponds to the coded data Bitstream1 output from the picture coding device 400 , as in 4 shown. As just described, by storing the filter type information, FilterType can be stored in the header 901 at the head of the bitstream encoded data or the disk header 902 at the head of each disk, which is not only the basic unit of recording and transmitting data but also the unit for correcting and modifying errors, the picture decoding apparatus selects the filtering type of a disk from decoding the disk by receiving the bit stream of encoded data from the header 901 or the disc.

It should be noted that it has been explained here that an inter-pixel filter is switched in the unit of a disk, but it can be switched not only in the unit of a disk but also in the unit of an image area smaller than one Disk (the unit may be an area containing one or more pixels, such as a macroblock or a block of MPEG). It can also be switched frame by frame, which is a unit larger than a slice. In this case, the value of the filter type information FilterType1 corresponding to each picture may not only be in (a diagonally hatched area of, for example) the header 901 be described as in 5A but also, for example, in an image header provided for all encoded image data. In addition, when switching a filtering method in the unit of a macroblock or a block, the value of the filter type information FilterType1 of each macroblock or block per slice can be composed and described in the slice header.

Furthermore, to transmit the coded data in the form of packets or the like, the header and the data may be transmitted separately. In this case, the header and the data are not as in 5 shown contained in a bit stream. However, in the case of packet transmission, the header associated with the data is currently being transmitted in another packet, even if the packets are transmitted slightly out of order, and thus the concept is the same as in the case of FIG 5 explained bitstream; even if they are not transmitted as a bit stream.

As explained above it is set by setting the value of the filter type information FilterType1, input to the image coding apparatus of the present invention, around an inter-pixel filter depending on the processing capability the image decoding device, possible, coded data depending on the processability the image decoding apparatus, which generates the of the Image coding apparatus of the present invention issued decoded coded data. An inter pixel filter can also be used selected become dependent from the processing ability the picture coding device.

In addition, can switch a filter to improve image quality, and not dependent be switched from the processing load.

6 Fig. 10 is a block diagram showing the structure of a picture decoding apparatus 1000 1 which decodes the coded data Bitstream1 generated by the picture coding apparatus according to the first example. The image decoding device 1000 Fig. 10 is an image decoding apparatus which switches an image-by-image or slice by slice inter-pixel filter according to the filter type information FilterType1 described in the header of the input coded data Bitstream1 and decodes the coded data in the coded data Bitstream1 and comprises a variable-length decoding unit 201 , the picture decoding unit 202 , the adder 203 , the image memory 205 , the inter-picture prediction unit 206 , a switch 1001 , a switch 1002 , an inter-pixel filter A 1003 and an inter-pixel filter B 1004 ,

The coded data Bitstream1 is externally input to the picture decoding device 1000 entered. The encoded data Bitstream1 are, for example, the image encoding device 300 or the picture coding device 400 in the first example coded data. The variable-length decoding unit 201 performs variable-length decoding of the input coded data Bitstream1, separates them into coded difference image data CodedRes, prediction parameter data PredParam and filter type information FilterType1, and outputs the coded difference image data CodedRes to the image decoding unit 202 , the prediction parameter data PredParam to the inter-picture prediction unit 206 and the filter type information FilterType1 to the switch 1001 and the switch 1002 out. When the value "1" is input as the filter type information FilterType1, the switch changes 1001 and the switch 1002 the connection to the side of the terminal "1" and apply a filtering by the inter-pixel filter A. 1003 to the decoded image data Recon. When the value "2" is entered as the filter type information FilterType1, the switch changes 1001 and the switch 1002 the connection to the side of the terminal "2" and apply filtering by the inter pixel filter B 1004 to the decoded image data Recon. Regardless of whether an inter-pixel filtering operation is performed by one of the inter-pixel filters, the filtered decoded image data FilteredImg1 is stored in the image memory 205 stored and outside the image decoding device 1000 , for example, to a display device or the like.

As explained above, according to the image decoding apparatus 1000 The present invention makes it possible to decode the coded data Bitstream1, including the filter type information FilterType1 indicating the type of the inter-pixel filter in the header.

It It should be noted that although the image decoding apparatus of the In this example, two inter-pixel filters are included three or more inter-pixel filters may contain. In this case, in the same way as in in accordance with the present example, one of the three or more pixel filters the filter type information is selected in the coded data bit stream and be used.

It It should be noted that, as shown in the first example, when the Filter type in the unit of an image or the unit of a smaller one Image area is switched as a picture, the inter-pixel filter Switches at the time when the filter type is changed.

7 Fig. 10 is a block diagram showing the structure of a picture decoding apparatus 1100 which uses a built-in inter-pixel filter instead of a particular inter-pixel filter when such a filter is not integrated. The image decoding device 1100 is characterized by using any one of the inter-pixel filters incorporated therein when the inter-pixel filter selected by the filter type information included in the coded data is not incorporated. The image decoding device 1100 includes the variable-length decoding unit 201 , the picture decoding unit 202 , the adder 203 , the image memory 205 , the inter-picture prediction unit 206 , the switch 1001 , the switch 1002 , the inter-pixel filter A 1003 , the inter-pixel filter B 1004 and a filter information conversion unit 1101 ,

For example, it is assumed that the image decoding device 1100 only two types of the inter-pixel filter A 1003 and the inter-pixel filter B 1004 contains, indicated by the value "1" and the value "2" of the filter type information FilterType1. Coded data Bitstream3 are externally input to the picture decoding device 1100 entered. The variable-length decoding unit 201 performs variable length decoding of the input coded data Bitstream3, separates it into coded difference image data CodedRes, Predictive Parameter Data PredParam and Filter Type Information FilterType3, and outputs the coded difference image data CodedRes to the image decoding unit 202 from, the predictive parameter data PredParam to the inter-image prediction unit 206 and the filter type information FilterType3 to the filter type information conversion unit 1101 , If the value of the filter type information FilterType3 is the value "3" which does not enter the image decoding device 1100 The built-in interpixel filter converts the filter type information conversion unit 1101 the value "3" of the filter type information FilterType3 into the value "2" indicating the inter-pixel filter whose smoothing level is that of the designated inter-pixel filter below that in the image decoding device 1100 for example, built-in interpixel filters come closest and output it as filter type information Filter-Type4.

By performing this conversion processing, decoding processing for restoring an image close to the original decoded image is possible, although the image quality is deteriorated to some extent because of other than the inter-pixel filter used for encoding in the image decoding apparatus 1100 is used so that there is sufficient availability as a simple image decoding function. If the value "1" is entered as the filter type information FilterType4, change the switch 1001 and the switch 1002 both connect to the side of port "1" and apply the filtering through the inter pixel filter A 1003 to the decoded image data Recon. If the value "2" is entered as the filter type information FilterType4, the switch will change 1001 and the switch 1002 both connect to the side of port "2" and apply the filtering through the inter pixel filter B 1004 to the decoded image data Recon. Filtered decoded image data Filtered Img3, which is the processing result by the inter pixel filter, becomes a display device or the like outside the image decoding device 1100 output.

As explained above, according to the image decoding apparatus 1100 even if the filter type information FilterType3 contained in the input coded data Bitstream3 indicates an inter-pixel filter which does not enter the image decoding apparatus 1100 built-in, they are decoded using a built-in inter-pixel filter instead. Therefore, the coded data bit stream can be decoded without substantially degrading the picture quality.

It should be noted that when the image decoding device 1100 an inter-pixel filter (including the case of no inter-pixel filtering operation), forced-use decoding of this inter-pixel filter can be performed.

It should be noted that the image decoding apparatus of the present example is provided with two inter-pixel filters (the case of no inter-pixel filtering operation is counted as a filter), but the image decoding apparatus equipped with three or more inter-pixel filters can do the same processing. In other words, the processing of storing the decoded image data Recon in the image memory 205 as they are, without performing an inter-pixel filtering operation may be included as a processing of the inter-pixel filter.

It It should be noted that, as shown in the first example, when the Filter type in the unit of an image or the unit of a smaller one Image area than that of an image, an inter-pixel filter is switched at the point of the change of the filter type.

Operation of the inter-pixel filter 303 . 304 . 1003 and 1004 becomes more detailed using 9 and 10 described. 9 Fig. 12 is a diagram showing the operational details of a deblocking filter which is an example of an inter-pixel filter. 9A Fig. 13 is a diagram showing pixel values near the boundary between blocks before filtering. 9B Fig. 13 is a diagram showing pixel values near the boundary between blocks after filtering. 10 Fig. 10 is a flowchart showing a flow of filter processing by an inter-pixel filter. 9A shows pixel values of the corresponding pixels 601 ~ 608 in a horizontal scan line. The pixels 601 ~ 604 are all pixels in a block 610 but the pixels 605 ~ 608 are pixels in a block 611 , adjacent to block 610 , The pixel values of the pixels 601 ~ 604 are respectively p3, p2, p1 and p0 and the pixel values of the pixels 605 ~ 608 are respectively q0, q1, q2 and q3. In an image coding apparatus, processing such as inter-pixel prediction, image coding, variable-length coding and image decoding is generally performed in the unit of a block (or macroblock). Therefore, the coding noise is likely to occur in the higher frequencies between pixels, such as the pixels 604 and 605 , across the boundary between adjacent blocks (or macroblocks), such as the block 610 and the block 611 , For example, there is a tendency that the difference between the pixel value p0 of the pixel 604 and the pixel value q0 of the image point 605 probably increases due to the influence of the coding noise. Therefore, an inter-pixel filter is a filter that can be determined using a plurality of parameters, for example, a filter corresponding to the parameters α and β for determining the filter, and pixel values of a group of pixels across the block boundaries are determined by this parameter. Pixel filter filtered.

As in 10 First, an inter-pixel filter first calculates the absolute value of the difference of the pixel values (p0-q0) between the pixel 604 and the pixel 605 over the limit and judges whether the calculated absolute value is smaller than the value of the parameter α or not (S701). As a result of the judgment, the inter-pixel filter does not perform deblock processing for the pixel value represented by the decoded image data Recon when the absolute value of the difference (p0-q0) between the pixel values is the value of the parameter α or more ( S704). On the other hand, as a result of the judgment in step S701, the inter-pixel filter calculates the absolute value of the difference (p1-p0) between the values of the pixel 604 and the pixel 603 when the absolute value of the difference (p0 - q0) between the values of adjacent pixels above the block boundary is smaller than the value of the parameter α and judges whether the calculated absolute value is smaller than the value of the parameter β or not (S702). The pixel 604 and the pixel 603 here are adjacent pixels in a block 610 , As a result of the judgment, the inter pixel filter results 503 a deblocking filter processing for the pixel value represented by the decoded image data Recon, if the absolute value of the difference (p0-p1) between the pixel values is the value of the parameter β or more (S704). As a result of the judgment, it also calculates the absolute value of the difference (q1-q0) between the pixel 605 and the pixel 606 and judges whether or not the calculated absolute value is less than the value of the parameter β when the absolute value of the difference (p1-p0) between the pixel values is less than the parameter β (S703). The pixel 605 and the pixel 606 here are adjacent pixels in a block 611 , As a result of the judgment, when the absolute value of the difference (q1-q0) of the pixel values is the value of the parameter β or more, the inter-pixel filter does not perform the deblocking filter processing for the pixel value represented in the decoded image data Recon (S704 ). On the other hand, if the absolute value of the difference (q1-q0) between the pixel values is less than the value of the parameter β, the inter-pixel filter results 503 a filtering for the decoded image data Recon to remove noise and stops the processing. The inter-pixel filter repeats the above-mentioned processing for each pixel matrix in the horizontal scanning line direction and vertical scanning line direction, respectively, over the block boundaries. In this way, block distortion is eliminated by performing the deblocking filter processing when one of the differences of the values of the three sets of adjacent pixels is less than a fixed value.

It is to be noted that in the deblocking filter processing in the above step S704, smoothing filtering (filtering to suppress high frequency components) is performed for the pixels near the boundary. For example, a new pixel value P0 of the pixel 604 are generated by smoothing using a low-pass filter to suppress high-frequency components for the pixel value p0 of the pixel 604 , the pixel value q0 of the pixel 605 , the pixel value p1 of the pixel 603 and the pixel value q1 of the pixel 606 ,

First Embodiment

8th Fig. 10 is a block diagram showing the construction of an image coding apparatus 500 according to the first embodiment shows. The picture coding device 500 differs from the picture coding device 300 in that, as inter-pixel filtering processing, the former can select whether the decoded image data Recon is as it is in the image memory 107 be saved as reference image data Ref or not. The picture coding device 500 includes the difference calculator 101 , the picture coding unit 102 , the picture decoding unit 104 , the adder 105 , the image memory 107 , the inter-picture prediction unit 108 , the inter-image prediction estimation unit 109 , a switch 501 , a switch 502 , an inter-pixel filter 503 , a lookup table storage unit 504 and a variable-length coding unit 505 ,

If the value of the filter type information FilterType2 is "0", the switch will change 501 and the switch 502 both connect to the side of port "0" to that of the adder 105 output decoded image data Recon in the image memory 107 to save as they are. When the value of the filter type information is "1", the switch changes 501 and the switch 502 both connect to the side of port "1" to that of the adder 105 output decoded image data Recon to a line to the inter-pixel filter 503 bring to. The inter-pixel filter 503 is a filter used to filter pixel values and, for example, a deblocking filter for suppressing coding noise in the higher frequency components at the block boundary. The filtered decoded image data FilteredImg2, which is the result of an inter-pixel filtering operation of the inter-pixel filter 503 are in the image memory 107 saved. The variable-length coding unit 505 performs a variable-length coding of this filter type information FilterType2, the coded difference image data CodedRes and the predictive parameter data PredParam, combines them into coded data Bitstream2, as in 5A and outputs it outside the picture coding device 500 out.

It should be noted that the picture coding device 500 has an inter-pixel filter, but it may have two or more inter-pixel filters. It can select any of the two or more inter-pixel filters or no inter-pixel filter operation to use them, and the filter type information indicating the type of inter-pixel filter used (including no inter-pixel filters -Operation) indicating insert into the encoded data. Furthermore, the image coding device 500 to the lookup table storage unit 504 and, instead, relinquish the function of the lookup table storage unit 504 in the inter-pixel filter 503 to have.

As explained above, in the picture coding apparatus 500 According to the present invention, it is possible to encode data using an inter-pixel filter depending on the processability of the image decoding apparatus to reproduce the image coding apparatus 500 output coded data Bitstream2 to produce. An inter-pixel filter may also be dependent on the processing capability of the image encoding device 500 to be selected. The filter type can also be switched in the unit of an image or the unit of a smaller image area than that of an image.

11 Fig. 10 is a block diagram showing the structure of a picture decoding apparatus 1200 according to the first embodiment of the invention, which can select whether to perform inter-pixel processing or not. The image decoding device 1200 differs from the picture decoding device 1000 in 6 in that the former performs an inter-pixel filtering operation not as inter-pixel filtering processing but storing the decoded image data Recon as they are in the image memory 205 as reference image data Ref can select. The image decoding device 1200 comprises a variable-length decoding unit 201 , the picture decoding unit 202 , the adder 203 , the image memory 205 , the inter-picture prediction unit 206 , a switch 1201 , a switch 1202 and an inter-pixel filter 1203 ,

The coded data Bitstream2, the header of which contains the filter type information FilterType2 indicating the inter-pixel filter in coding, for example the coded data in 9A generated by the image coding device 500 in 8th are encoded into the image decoding device 1200 entered. The filter type information FilterType2 includes the value indicating "no inter-pixel filter operation" as a filter type. The variable-length decoding unit 201 performs a variable-length decoding of the input coded data Bitstream2 and separates it into coded difference image data CodedRes, predictive parameter data PredParam and filter type information FilterType2. The separated coded difference image data CodedRes, the predictive parameter data PredParam and the filter type information FilterType2 become corresponding to the image decoding unit 202 , the inter-image prediction unit 206 and the switch 1201 and the switch 1202 output.

When the value "0" is input as the filter type information FilterType2, the switch changes 1201 and the switch 1202 both the connection to the side of port "0" and that of the adder 203 Recon output decoded image data as they are in the image memory 205 saved.

When the value "1" is input as the filter type information FilterType2, the switch changes 1201 and the switch 1202 both connect to the side of port "1" to filter through the inter-pixel filter 1203 to apply Recon to the decoded image data.

When it is judged by the filter type information that an inter-pixel filter is not used for a decoded image to be a reference image, the decoded image is not filtered by the inter-pixel filter for storing the image in the image memory as a reference image, but the inter-pixel filter can be used to output it only outside a picture decoding apparatus. 12 Fig. 10 is a block diagram showing the structure of a picture decoding apparatus 1300 including an image output unit equipped with selectable inter-pixel filters. As explained above, the image decoding apparatus is 1300 a picture decoding apparatus which, when the filter type information indicates FilterType2, indicates that the adder 203 outputted decoded image is not filtered by an inter-pixel filter that does not perform filtering of the decoded image stored in the image memory, but underfilters the decoded image output to the outside Using an out-of-range inter-pixel filter and includes the variable-length decoding unit 201 , the picture decoding unit 202 , the adder 203 , the image memory 205 , the inter-picture prediction unit 206 , the switch 1201 , the switch 1202 , the inter-pixel filter 1203 , a switch 1301 , a switch 1302 and an inter-pixel filter 1303 ,

If the value of the filter type information FilterType2 is "1", the switch will change 1201 , the desk 1202 , the desk 1301 and the switch 1302 all to the side of port "1". In this case connect the switch 1201 and the switch 1202 the output of the adder 203 , the inter-pixel filter 1203 and the image memory 205 , and the switch 1302 interrupts the connection to the output of the switch 1202 and the inter-pixel filter 1303 to him with the switch 1301 to bridge. Therefore, the inter-pixel filter performs 1203 performs a filtering operation on the decoded image data Recon and outputs the filtered decoded image data FilteredImg3. The filtered decoded image data FilteredImg3 is output as an output image Outlmg to a display device or the like outside the image decoding device as it is without passing through the inter-pixel filter 1303 to be filtered again. If the value of the filter type information FilterType2 is "0", the switch will change 1201 , the desk 1202 , the desk 1301 and the switch 1302 all the connection to the side of port "0". In this case, the switch interrupts 1201 the connection to the output of the adder 203 and the inter-pixel filter 1203 to do this with the switch 1202 to bridge. On the other hand, the switch connects 1302 the output of the switch 1202 , the inter-pixel filter 1303 and the external output terminal of the switch 1301 , Therefore, through the inter-pixel filter 1203 no inter-pixel filtering operation with those from the adder 203 but they become as they are as a reference image in the image memory 205 saved. For the decoded image data Recon, which from the output side of the switch 1202 That is, the filtered decoded image data FilteredImg3, which has not actually been filtered, becomes the inter-pixel filtering operation by the inter-pixel filter 1303 and executed as an output image OutImg to a display device or the like outside the image decoding device 1300 output.

It should be noted that the inter-pixel filter 1203 and the inter-pixel filter 1303 have been described herein as different elements for explanation, but an inter-pixel filter may be used for the implementation (it is no problem to use an inter-pixel filter because two inter-pixel filters are not operating simultaneously ). The inter-pixel filter 1203 and the inter-pixel filter 1303 can use the existing inter-pixel filter 106 be, or the inter-pixel filter 503 including the lookup table storage unit 504 , as in 8th shown. Furthermore, it may be that the inter-pixel filter 503 the lookup table storage unit 504 for holding a plurality of parameter tables 620 includes. In this case, however, the filter type information FilterType2 must also be in the inter-pixel filter 503 be entered.

As explained above, according to the image decoding apparatus 1300 Also, in the case where filtering for a decoded image is not performed, which is to be a reference image, filtering by an inter-pixel filter for the decoded image is performed by an inter-pixel filter, so that a display device for displaying one of the image decoding device 1300 output image OutImg can display moving images with a higher image quality. This is particularly effective for devices with sufficient processing capability when filtering for a decoded image, which is to be a reference image, is not performed.

It It should be noted that, as shown in the first example, when the Type of one indicated by the filter type information FilterType2 Inter-pixel filter in the unit of an image or in the unit a smaller image area than an image is switched, the inter-pixel filter at the time of changing the filter type is switched.

13 Fig. 10 is a block diagram showing the structure of a picture decoding apparatus 1400 shows which is capable of the inter-pixel filter 1203 depending on a picture type of each picture. The image decoding device 1400 is an image decoding device that decodes information about whether a decoded image is used as a reference image or not, for example, encoded data including an image type for each image and the like, and includes the image decoding unit 202 , the adder 203 , the image memory 205 , the inter-picture prediction unit 206 , the inter-pixel filter 1203 , a variable-length decoding unit 1401 , a switch 1402 , a switch 1403 and an image information conversion unit 1404 ,

The variable-length decoding unit 1401 performs variable length decoding of the externally input coded data Bitstream4 and separates it into picture type information PType, coded difference picture data CodedRes and predictive parameter data PredParam. The separated image type pinformation PType, the coded difference image data CodedRes and the predictive parameter data PredParam become corresponding to the image type conversion unit 1404 , the picture decoding unit 202 and the inter-image prediction unit 206 output. The image type information PType is the information indicating whether or not a current image is used as a reference image. For example, according to international standards, MPEG-1 and 2 are information in a coded data of each picture called an image type, and an image called a B-picture is not used as a reference picture. Therefore, this image type included in the coded data can be used as the image type information PType of the present invention. Even if the filtering by an inter-pixel filter is not performed for an image which is not used as a reference image, it does not seriously affect the decoding of other images.

Here leads the picture decoding device 1400 Inter-pixel filtering is not sufficient if the current image is not used as a reference image. For example, if the processing capability of the image decoding device 1400 is too low to perform the decoding in time for the reproduction time, the processing load of the image decoding apparatus 1400 by not performing the filtering by an inter-pixel filter for images which are not used as reference images. It is done using the block diagram in 13 explained. Change first when entering the image type information conversion circuit 1404 inputted picture type information PType indicates other than a B-picture, that is, when it indicates that the current picture is used as a reference picture, the switch 1402 and the switch 1403 both connect to port "1." This will cause the image decoding device 1400 an inter-pixel filtering operation for the decoded image data Recon using the inter-pixel filter 1203 and stores the operation result in the image memory 205 as filtered decoded image data FilteredImg5 and outputs it to a display device or the like outside the image decoding device 1400 out. On the other hand, when the picture type information PType indicates that the current picture is a B-picture, that is, it indicates that the current picture is not used as a reference picture, the switch changes 1402 and the switch 1403 the connection to port "0" and that from the adder 105 output decoded image data Recon without using the inter-pixel filter 1203 issued directly to the outside.

As mentioned above, the image decoding apparatus 1400 filtering by the inter-pixel filter 1203 For a B-picture to which other pictures hardly refer, the processing load required to decode the bitstream of coded data can be reduced without having much influence on the decoding of other pictures. As the image decoding device 1400 For example, if an inter-pixel filter is selected in accordance with an image type of coded data in this way, a filter processing load can also be reduced for coded data output from a conventional image coding apparatus, and its header information such as an image header will not be selection information of an inter-pixel filter because filtering processing is omitted from images that are not referenced.

It should be noted that it is not necessary to refer to images not referenced in the image memory 205 , for example in 13 regardless of whether filter processing is performed on the images or not. Therefore, it is necessary in the image memory 205 to store only data obtained by performing filter processing on images referenced.

It It should be noted in a stricter sense that a B-picture is not an image means which is not referred to, but an image coding method in which reference is made to a B-picture is, is conceivable. Therefore, if an inter-pixel filter not only dependent on a picture type selected but judges whether the picture is actually related is taken or not, a more suitable processing is carried out even if reference is made to a B-picture. Even if on a B-image, however, an inter-pixel filter may be used for Simplifying the implementation only switched depending on a picture type become.

Also, whether or not inter-pixel filtering is to be performed is not switched, but two filters, the inter-pixel filter 1003 and the inter-pixel filter 1004 , can be switched depending on a picture type or whether an image is referenced or not, as in 6 or 7 shown.

Farther An example of a picture decoding apparatus has been explained which an inter-pixel filter dependent of a picture type or it switches, whether referring to a picture or not, but a picture coding device can do this Switching also in the same way make.

Second Embodiment

14 Fig. 4 is a block diagram showing the functional structure of a picture coding apparatus 1500 according to the second embodiment of the present invention. The picture coding device 1500 is realized by a computer having a CPU, a memory, a hard disk (HD) on which an image coding program and others are installed, and functions as an operation console unit 1505 on, a preprocessing unit 1510 , a subtraction unit 1512 , an orthogonal transformation unit 1513 , a quantization unit 1514 , a variable-length coding unit 1517 , a post-processing unit 1520 , an inverse quantization unit 1521 , an inverse orthogonal transformation unit 1522 , an adding unit 1524 , a switching unit 1530 , an inter-pixel filter 1540 , a frame store 1541 , a motion estimation unit 1542 , a motion compensation unit 1543 , a priority determination unit 1550 and a filter processing control unit 1560 ,

The control panel unit 1505 accepts an input operation of an operator. The preprocessing unit 1510 is equipped with a format conversion unit for converting a format of an inputted image signal into one by the operation of the operation panel unit 1505 designated spatial resolution, a rearrangement unit for rearranging images according to image types and others, and outputs images or the like in an order.

It should be noted that there are the following kinds of pictures: an I-picture (intra-picture: intra-coded picture) which is generated in the intra-picture coding mode; a P-picture (predictive picture: predictively coded picture) which is generated in the inter-picture coding mode and refers to only one picture; and a B-picture (bipedal picture: multi-predictive picture) which may also refer to a backward picture and at the time of motion estimation in the inter-picture coding mode is the number of times in the picture memory 1541 stored decoded images, which can be referenced by the motion estimation unit at the same time.

Also when coding an image, there is a mode for encoding it using three kinds of pictures (hereinafter referred to as an "IPB coding mode") and a mode of encoding it using only two Types of pictures, an I-picture and a P-picture. As the mode to encode them using only two types of images, an I-picture and a P-picture, there is a mode for encoding a P-picture, which is a possibility has to be referenced, and a P-picture which has no possibility has to be referenced (hereinafter also referred to as a "first IP coding mode "), and a mode for coding a P-picture in the base layer in layer coding, a P-picture, which is a possibility has to be referenced, and a P-picture which has no possibility has to be referenced in the enhancement layer (hereafter as a "second IP Encoding Mode ") layer coding, images are classified into two groups, a base layer and an extension layer, and the base layer is a group of images that are rendered by themselves can be, and the extension layer is a group of images, which the group of pictures in the base line for coding and decoding needed. The layer coding is characterized in that, since the number of bits only for the base layer is low, but the number of bits for the base layer and for the extension layer is large and the number of pictures is big, two types of uses by recording and transmitting the base line in all cases and recording and transmitting the expansion layer only when it is required for high image quality is, can be realized.

In In the case of the first IP coding mode, the information "possibility" or "no Possibility "added to a picture and information" possibility "or" no Possibility "also becomes that Added image type. Also in the case of the second IP coding mode, information "base", "possibility" or "no Possibility "added to a picture and information" base "," possibility "or" no Possibility "becomes one too Added image type.

The subtraction unit 1512 gives that from the preprocessing unit 1510 outputted image in the intra-picture coding mode as it is, and calculates a motion compensation error (residual image) which is a difference value between the image and that of the motion compensation unit 1543 is motion compensation image output in the inter-picture coding mode.

The orthogonal transformation unit 1513 outputs frequency components in the frequency domain resulting from execution of the orthogonal transformation such as a discrete cosine transform for the image in the intra-image coding mode and the motion compensation error in the inter-image encoding mode, respectively output from the subtraction unit. The quantization unit 1514 outputs a quantized value by quantizing the from the orthogonal transformation unit 1513 output frequency components. The variable-length coding unit 1517 Outputs a coded signal for which further information compression is performed using a Huffman code corresponding to that of the quantization unit 1514 output quantized value assigns a code length depending on the frequency of its occurrence. The post-processing unit 1520 is provided with a buffer for temporarily storing the coded signal or the like, a frequency control unit for controlling a quantization area in the quantization unit 1514 , and others, and transforms the above-mentioned motion vector, picture type or the like and that of the variable-length coding unit 1517 output signal into a coded signal as a bit stream and outputs it.

The inverse quantization unit 1521 decodes the frequency components by inverse quantizing the by the quantization unit 1514 generated quantized value. The inverse orthogonal transformation unit 1522 decodes the picture in the intra-picture coding mode and the motion compensation error (residual picture), that is, the difference value of pixels in the inter-picture coding mode, by performing the inverse orthogonal transform of that of the inverse-quantizing unit 1521 decoded frequency components. The addition unit 1524 decodes the image by outputting the image (decoded image) transmitted by the inverse orthogonal transformation unit 1522 is decoded as it is in the intra-picture coding mode, and by adding the still picture formed by the inverse orthogonal transform unit 1522 is decoded, and the motion compensation image generated by the motion compensation unit 1543 is generated in the inter-picture coding mode.

The switching unit 1530 includes a pair of switches 1531 and 1532 for synchronously switching the switching state under the switch ON / OFF control of the filter processing control unit 1560 for each picture and contains the inter-pixel filter 1540 in a loop or omits it at the loop, that is, leaves the inter-pixel filter 1540 skip its processing. The inter-pixel filter 1540 performs a spatial low-pass filter processing for that of the addition unit 1524 decoded image on a block-by-block basis when the switches 1531 and 1532 ON to produce the decoded image without block distortion or the like. For example, it calculates an average between a pixel and adjacent pixels, and when the difference between the pixel and an adjacent pixel is within a predetermined range, it performs the processing of replacing each pixel around the block boundary with the calculated average.

The image memory stores a plurality of the switching unit 1530 output, decoded pictures. Thereby, it becomes possible to monitor the decoded image in the same state as the image decoding device which is that of the post-processing unit 1520 output coded signal or the decoded picture is used as a reference picture in the inter-picture coding mode. It is to be noted that, in the first IP coding mode and the second IP coding mode, a decoded P picture to which the reference possibility indicating information is added always remains in the picture memory 1541 and a decoded P-picture to which the non-reference-indicative information is added need not be stored in the image memory 1541 get saved. Therefore, the information means whether possibility or no possibility, whether the decoded image in the image memory 1541 is saved or not.

In the image decoding mode, the motion estimation unit searches 1542 among those in the image memory 1541 stored decoded images after a reference image whose difference from that of the preprocessing unit 1510 output image is the smallest, and outputs a motion vector, that is, the amount of movement of a differential pixel. It is to be noted that when the motion vector is output, a block prediction type indicating whether the reference picture is a forward picture, a backward picture or the average of both pictures is output. The motion compensation unit 1543 executes the operation indicated by the motion vector and the block prediction type and generates a motion compensation image. The priority determination unit 1550 outputs the priority of an image depending on the image type and the base layer or the enhancement layer. The filter processing control unit 1560 controls ON / OFF the switch 1531 and 1532 depending on the priority determination unit 1550 output priority or CPU utilization.

15 is a block diagram showing the detailed functional structure of the priority determination unit 1550 shows how in 14 shown. As shown in this figure, the priority determination unit gives 1550 The priority of an image depends on the image type and the base layer or enhancement layer and includes three tables 1551 ~ 1553 , a selector 1554 and a determination processing unit 1555 , as in 15 shown. It is to be noted that in the case of a P-picture in the second IP encoding mode, the "base", "possibility to refer to" or "no way to refer to" information is added to its picture type.

The table 1551 is a table which is selected when the IPB coding mode by the operation on the operation panel unit 1505 and in which the image types of images are assigned priorities thereof and the priorities are set to "0" for I-pictures, for "P-pictures" to "1" and for B-pictures to "2." It should be noted that the priorities are set such that they get lower as the numbers get larger.

The table 1552 is a table which is selected when the first IP coding mode by the operation on the operation panel unit 1505 in which the image types of images are assigned priorities thereof and the priorities are set to "0" for I-pictures, to "1" for P-pictures (which have a possibility to refer to) and to "1" for P-pictures. Images (which have no possibility to refer to) to "2".

The table 1553 is a table which is selected when the second IP coding mode (base, reference and no reference) is designated, and the priorities are set to "0" for I pictures, to "1" for P pictures (base) for P-pictures (which have a possibility to refer to them) to "2", and for P-pictures (which have no way of referring to them) to "3".

The selector 1554 select one of the tables 1551 ~ 1553 based on the coding mode (the IPB coding mode or the first IP coding mode) designated by the operation panel unit 1505 , The determination processing unit 1555 determines the priority depending on the image type and the base layer or enhancement layer output from the preprocessing unit 1510 with reference to that of the selector 1554 selected table, and outputs the given priority. In particular, when the IPB coding mode is designated, the selector selects 1554 the table 1551 off and the destination processing unit 1555 outputs the priority assigned to an I-picture, a P-picture or a B-picture every time the picture type is from the pre-processing unit 1510 is issued. Further, when the first IP coding mode is designated, the selector selects 1554 the table 1552 off and the destination processing unit 1555 outputs the priority based on the image type and the data added to a P-picture ("Possibility" or "Possibility"). Further, when the second IP coding mode is designated, the selector selects 1554 the table 1553 off and the destination processing unit 1555 outputs the priority based on the image type and the data added to a P-picture ("base", "possibility" or "no possibility").

16 Fig. 10 is a block diagram showing a detailed functional configuration of the filter processing control unit 1560 shows how in 14 shown. As shown in this figure, the filter processing control unit controls 1560 ON / OFF the switch 1531 and 1532 according to that of the priority determination unit 1550 output priority and CPU utilization and includes three tables 1561 ~ 1563 , a selector 1564 and a switching processing unit 1565 , as in 16 shown. The table 1561 is a table which is selected when the IPB coding mode is designated and indicates combinations of the priorities and CPU loads to perform the filtering processing, and is set to switch ON for the priorities 0 ~ 2 when the CPU load is less than 70 % is, switch ON only for priorities 0 and 1 when CPU utilization is 70% or more and less than 80%, and switch ON for priority 0 only when CPU utilization is 80% or more.

The table 1562 is a table which is selected when the first IP encoding mode is designated, and indicates combinations of the priorities and CPU loads to perform the filter processing, and is set to switch ON for the priorities 0 ~ 2 when the CPU utilization is less than 70%, switch ON is for priorities 0 and 1 only when CPU utilization is 70% or more and less than 80%, and switch ON is for priority 0 only when CPU utilization is 80% or more.

The table 1563 is a table which is selected when the first IP encoding mode is designated, and indicates combinations of priorities and CPU loads to perform the filter processing, and is set to switch ON for the priorities 0 ~ 3 when the CPU utilization is less than Is 70%, switch ON only for priorities 0, 1 and 2 when the CPU utilization is 70% or more and less than 80%, and switch ON only for the priorities 0 and 1 when the CPU utilization is 80% or more.

The selector 1564 select one of the tables 1561 ~ 1563 based on the coding mode (the IPB coding mode, the first IP coding mode or the second IP coding mode) designated by the operation panel unit 1505 , The switching processing unit 1565 outputs a signal to switch ON or OFF to the switches 1531 and 1532 the switching unit 1530 based on the image reached by each priority image unit 1550 and the CPU utilization priority issued by the selector 1564 control selected table.

In particular, when the IPB coding mode is designated, the selector selects 1564 the table 1561 off and the switch switch processing unit 1565 gives a signal to turn on I-picture, P-picture and B-picture when the CPU load is less than 70%. When the CPU utilization is 70% or more and less than 80%, the switching processing unit gives 1565 a signal for turning ON only for an I-picture and a P-picture. When the CPU utilization is 80% or more, the switching processing unit gives 1565 a signal to turn ON only for one I-picture.

When the first IP coding mode is designated, the selector selects 1564 the table 1562 off and the switching processing unit 1565 outputs a signal for turning on I-picture, P-picture (possibility) and P-picture (no possibility) when the CPU load is below 70%. When the CPU utilization is 70% or higher and under 80%, the switching processing circuit gives 1565 a signal for turning ON only for an I-picture and a P-picture (possibility). When the CPU utilization is 80% or more, the switching processing circuit gives 1565 a signal to turn on for an I-picture.

Furthermore, the selector selects 1564 if the second IP encoding mode is called, the table 1563 off and the switching processing unit 1565 outputs a signal for turning on I-picture, P-picture (base), P-picture (possibility) and P-picture (no possibility). When the CPU utilization is 70% or more and less than 80%, the switching processing unit gives 1565 a signal for turning ON only for an I-picture, a P-picture (base) and a P-picture (possibility). When the CPU utilization is 80% or more, the switching processing circuit gives 1565 a signal for turning ON only for an I-picture and a P-picture (base).

Next, the operation of the image coding apparatus constructed as explained above will be described 1500 explained.

In the intra-picture coding mode for coding an image such as an I-picture, one of the pre-processing unit becomes 1510 output image by an orthogonal transformation by the orthogonal transformation unit 1513 compressed and encoded into a frequency component and by the quantization unit 1514 compressed and encoded into a quantized value. This quantized value is obtained by variable-length coding by the variable-length coding unit 1517 compressed and encoded into a variable length, into a coded signal of a bit stream of an I-picture by the post-processing unit 1520 converted and stored on a storage medium such as a hard disk.

On the other hand, that of the quantization unit 1514 output quantized value by inverse quantization by the inverse quantization unit 1521 converted into a frequency component and by inverse orthogonal transformation by the inverse orthogonal transformation unit 1522 decoded into a picture. When the switches 1531 and 1532 under the control of the filter processing control unit 1560 ON, this decoded image will be in the image memory 1541 after it has been cleared for eliminating the block distortion by the inter-pixel filter 1540 was processed filter, and if the switches 1531 and 1532 OFF, it will be in the image memory 1541 stored without being filter processed.

In the inter-picture coding mode for coding an image as a P-picture and a B-picture, a motion vector is detected by the motion estimating unit 1542 generates a motion compensation image (predictive image) by the motion compensation unit 1543 and a motion compensation error (difference image) is generated by the subtraction unit 1512 generated. It should be noted that the motion estimation unit 1542 among those in the image memory 1541 stored decoded pictures searches for a predictive picture as one of a plurality of forward or backward reference pictures, the difference thereof from that of the pre-processing unit 1510 output image is the lowest.

17 FIG. 13 is a diagram showing reference relationships between in the image memory. FIG 1541 stored pictures shows. In particular 17A a diagram showing reference images for prediction according to the IPB method, 17B Fig. 13 is a diagram showing reference pictures for prediction after the first IP method, and 17C Fig. 13 is a diagram showing reference pictures for prediction according to the second IP method. It should be noted that under each picture of each method the priority (priority level) of the picture is indicated.

For predicting a P-picture in the case of the IPB method in 17A may be referred to a forward I-picture and P-picture. For predicting a B-picture, a forward I-picture or P-picture may be referred to, and a backward and temporally next I-picture or P-picture may be referred to.

It should be noted that in order to predict a B-picture in H.26L, a B-picture may be referred to as a forward picture in addition to an I-picture and a P-picture. In the mode using this B-picture as a reference picture, there is also information "There is a possibility" or "There is no possibility" to the picture type added. And in this mode, the decoded B-picture is always in the frame memory with the information that there is a possibility to refer to it 1541 and the decoded B-picture with the information that there is no possibility to refer to it need not be stored in the image memory 1541 get saved.

For predicting a P-picture (reference possibility) after the first IP-method in 17B may be referred to a forward I-picture or P-picture (reference possibility). For predicting a P-picture (no reference possibility), a forward I-picture or P-picture (reference possibility) can be referred to.

For predicting a P-picture (base) after the second IP method in 17C may be referred to a forward I-picture or P-picture (base). For predicting a P-picture (reference possibility), a forward I-picture and P-picture (base) may be referred to. For predicting a P-picture (no reference possibility), a plurality of forward I-pictures, P-pictures (base) or P-pictures (reference possibility) can be referred to.

It It should be noted that to simplify the explanation, the case in which IPB coding mode is explained.

Under such a restriction, the motion estimation unit gives 1542 the amount of movement of the difference pixels between the searched reference image and that of the preprocessing unit 1510 output image as a motion vector and also outputs a block prediction type which indicates whether a reference picture is a forward picture, a backward picture or an average of bi-predictive pictures. The motion compensation unit 1543 For the differential image point, that also leads through the motion estimation unit 1542 output motion vector and block prediction type indicated operation to generate a motion compensation image. And the subtraction unit 1512 generates a motion compensation error (difference image) by subtracting it by the motion compensation unit 1543 generated compensation image of that of the preprocessing unit 1510 output image.

The one from the subtraction unit 1512 output motion compensation error (difference image) is obtained by orthogonal transformation by the orthogonal transformation unit 1513 is compressed into a frequency component and encoded, and quantized by the quantization unit 1514 compressed and encoded into a quantized value. This quantized value is obtained by variable-length coding by the variable-length coding unit 1517 compressed and coded into a coded signal in a bit stream of a P-picture or a B-picture together with the motion vector and others by the post-processing unit 1520 converted and stored on a storage medium such as a hard disk.

On the other hand, the quantized value of a P-picture or a B-picture with a possibility to refer to it, that of the quantization unit 1514 is output by an inverse quantization by the inverse quantization unit 1521 converted into a frequency component and by inverse orthogonal transformation by the inverse orthogonal transformation unit 1522 decoded into a motion compensation error (difference image). Then the addition unit adds 1524 the motion compensation error (difference image) and the motion compensation image, and thus it is decoded into an image. This decoded image is stored in the image memory 1541 after it has been cleared for eliminating block distortion by the inter-pixel filter 1540 Filtered is when the switches 1531 and 1532 under the control of the filter processing control unit 1560 ON, and is in the image memory 1541 stored without being filter processed when the switches 1531 and 1532 OFF are switched.

The ON / OFF control of the switches 1531 and 1532 through the filter processing control unit 1560 will be explained in more detail here.

18 Fig. 10 is a flowchart showing one by the switching processing unit 1565 in the filter processing control unit 1560 executed switch activation processing.

The determination processing unit 1555 in the priority determination unit 1550 determines the priority of each of the preprocessing unit 1510 output image depending on the image type with reference to that of the selector 1554 selected table 1551 and outputs the given priority. In particular, when the IPB coding mode is designated, the selector selects 1554 the table 1551 off and the destination processing unit 1555 outputs the priority "0" for an I-picture, the priority "1" for a P-picture and the priority "2" for a B-picture every time the picture type is from the pre-processing unit 1510 is issued.

Each time an image is encoded, the switching processing unit detects 1565 in the filter processing control unit 1560 the priority of the image and the utilization of the image encoding device 1500 contained CPU (S21) and determines the entry on which in the table Be is to take (the table 1561 in the example in 16 ) (S22).

Especially, if the CPU usage is below 70%, it determines that the entry, To refer to, the first line is when the CPU usage is 70% or higher is more and less than 80%, it determines that entry, on which reference should be made, the second line is, and if the CPU utilization 80% or more, Does it specify that the entry to which reference should be made the third line is.

After determining the entry to which reference should be made, the switching processing unit reads 1565 the right column of the entry (S23) and judges whether or not the priority set for the picture type of the decoded picture is included in the right column (S24). If it is included in the right column (Yes in S24), the switching processing unit gives 1565 a signal to turn on to the switches 1531 and 1532 off (S25). Thereby, a filter processing for the decoded image is performed, and the filter-processed decoded image is stored in the image memory 1541 saved.

On the contrary, if it is not included in the right column (No in S24), the switching processing unit returns 1565 a signal to turn OFF the switches 1531 and 1532 off (S26). This skips the filter processing for the decoded picture and the decoded picture is stored in the picture memory 1541 stored without being filter processed.

Such control is carried out for each picture, and decoded pictures which have been filter-processed and which have not been filter-processed are stored in the order in the picture memory 1541 saved. Accordingly, in the image coding, inter-pixel filtering for noise removal or the like is not always carried out, but the inter-pixel filtering is carried out as needed, therefore, when inter-pixel filtering becomes only for images having a serious influence image quality is performed, for example, even in a picture decoding apparatus having a low processing capability, it is possible to maintain the picture quality of the important pictures stored in the picture memory so as to prevent accumulation of the block distortion in the decoded pictures stored in the picture memory have been made to improve the predictive efficiency by the motion compensation unit and to reduce the deterioration of the picture quality more than the MPEG technique, and thus the large effect of improving the picture quality at a low bit rate can be achieved.

Especially when running the inter-pixel filtering is given a priority to an image which a big Has an impact on other images, that is, an intra-coded image, a forward-predictively coded Image, a base layer image or the like, so the effect of improving image quality, such as noise elimination, by an inter-pixel filter even at the same increase in the Processing load can be realized more effectively.

Farther the filter processing on / off can be controlled to full Use of the processing ability the picture coding device, so that the CPU with high Efficiency is used, and thus coding for higher image quality can also be used the same hardware resources be realized.

Third Embodiment

Next, an image decoding apparatus according to an embodiment of the present invention will be explained. 19 FIG. 10 is a block diagram showing the functional structure of a picture decoding apparatus. FIG 1600 according to a third embodiment of the present invention.

This image decoding device 1600 is an apparatus for decoding the encoded signal encoded by image encoding apparatus 1500 , as in 14 and is realized by a computer apparatus equipped with a CPU, a memory, a hard disk (HD) on which a picture decoding program or the like is installed, and includes as functions a pre-processing unit 1610 , a variable-length decoding unit 1617 , an inverse quantization unit 1621 , an inverse orthogonal transformation unit 1622 , an addition unit 1624 , a switching unit 1630 , an inter-pixel filter 1640 , a post-processing unit 1670 , a frame store 1641 , a motion compensation unit 1643 , a priority determination unit 1650 and a filter processing control unit 1660 ,

The preprocessing unit 1610 is provided with a buffer or the like for temporarily storing a coded signal, and separates it into the picture type, the motion vector and the coded signal of an image itself contained in the coded signal to output them. It should be noted that when the coded signal of the picture is in the first IP coding mode, the information about "possibility" and "no possibility" is added to the picture and the information about "possibility" and "no possibility" is also added added to the image type. Although it is in the second IP encoding mode, the information is "base", "Possibility" and "no possibility" are added to the image and the information "base", "possibility" and "no possibility" is also added to the image type.

The variable-length decoding unit 1617 outputs a fixed-length quantized value by decoding (Huffman decoding) of the pre-processing unit 1610 output coded signal. The inverse quantization unit 1621 decodes the frequency component by inverse quantization of that from the variable length decoding unit 1617 output quantized value. The inverse orthogonal transformation unit 1622 decodes the image in the intra-image coding mode and the motion compensation error (residual image), that is, a difference value of pixels in the inter-image coding mode, by inverse orthogonal transforming by the inverse quantizing unit 1621 decoded frequency component.

The addition unit 1624 gives that through the inverse orthogonal transformation unit 1622 decoded image as it is in the intra-picture coding mode and decodes the picture by adding the by the inverse orthogonal transform unit 1622 decoded motion compensation error (residual image) and by the motion compensation unit 1643 motion compensation image generated in the inter-picture coding mode. The switching unit 1630 includes a pair of switches 1631 and 1632 for synchronously switching its switching state under the switch ON / OFF control of the filter processing control unit 1660 for each picture and includes the inter-pixel filter 1640 in a loop or omits it at the loop, that is, leaves the inter-pixel filter 1640 skip its processing.

The inter-pixel filter 1640 performs a spatial low-pass filter processing for that of the addition unit 1624 output decoded image on a block-by-block basis when the switches 1631 and 1632 ON to produce the decoded image without block distortion or the like. For example, it calculates an average of one pixel and adjacent pixels, and if the difference between the pixel and the adjacent pixels is within a predetermined range, it performs the processing of replacing each pixel at the block boundary with the calculated average.

The post-processing unit 1670 is provided with a format conversion unit for converting a format into a predetermined spatial resolution, an image order restoration unit for restoring the order of the images which have been rearranged depending on their image types in their original order and the like, and outputs the decoded images to a monitor or similar. The image memory 1641 stores a plurality of the switching unit 1630 output decoded images with a possibility to refer to. The motion compensation unit 1643 passes through the motion vector and the block prediction type for the in-frame memory 1641 stored decoded images output from the preprocessing unit 1610 , to generate motion compensation images. The priority determination unit 1650 has the same structure as the one in 15 shown priority determination unit 1550 and gives the priority of an image depending on that of the preprocessing unit 1610 output image type and the base layer or enhancement layer. The filter processing control unit 1660 has the same structure as the filter processing control unit 1560 , as in 16 shown, and controls on / off the switch 1631 and 1632 the switching unit 1630 depending on the priority determination device 1650 issued priority and the CPU utilization obtained by monitoring.

Next, the operation of the image decoding apparatus constructed as explained above will be described 1660 explained. It should be noted as to the appropriateness of the explanation thereof together with the picture coding apparatus 1500 the case in which the IPB coding mode is determined will be explained.

In the intra-picture decoding mode for decoding an encoded signal of an I-picture into an image, that of the preprocessing unit 1610 output coded signal by variable-length decoding by the variable-length decoding unit 1617 is decoded into a quantized value, expanded and by inverse quantization by the inverse quantization unit 1621 is decoded into a frequency component and by inverse orthogonal transformation by the inverse orthogonal transformation unit 1622 decoded into an image (decoded image). When the switches 1631 and 1632 under the control of the filter processing control unit 1660 ON, this decoded image will be in the image memory 1641 after it has been cleared for eliminating the block distortion by the inter-pixel filter 1640 was processed, and the order of the images is in the post-processing unit 1670 restored as the original, whose format is converted, and then output to a monitor or the like. In contrast, when the switches 1631 and 1632 OFF, the decoded image in the image memory 1641 stored without being filter-processed, and the order of the images is in the post-processing unit 1670 restored as the original, whose format is converted, and then output to a monitor or the like.

In the inter-picture decoding mode for decoding coded signals of a P-picture and a B-picture into pictures, the picture is reproduced by the preprocessing unit 1610 output coded signal by the variable-length decoding by the variable-length decoding unit 1617 is decoded into a quantized value, expanded and by inverse quantization by the inverse quantization unit 1621 is decoded into a frequency component and by inverse orthogonal transformation by the inverse orthogonal transformation unit 1622 decoded into a motion compensation error (difference image).

On the other hand, the motion compensation image (predictive image) by the motion compensation unit 1643 generated. It should be noted that the motion compensation unit 1643 the motion vector and block prediction type output by the preprocessing unit 1610 , for the difference pixels in the image memory 1641 is executed to generate a motion compensation image.

The addition unit 1624 adds the motion compensation error (difference image) and the motion compensation image to decode the result into an image. When the switches 1631 and 1632 under the control of the filter processing control unit 1660 After this decoded picture to turn off the block distortion by the inter-pixel filter 1640 and then the order of the images in the post-processing unit 1670 is restored, its format is converted and the image is output to a monitor or the like, and the decoded image with a possibility to refer to it is stored in the image memory 1641 saved. If, on the other hand, the switches 1631 and 1632 OFF, filter processing is not performed, but the order of images is in the post-processing unit 1670 is restored as the original order, its format is converted, and the image is output to a monitor or the like, and the decoded image with a possibility to refer to it is stored in the image memory 1641 saved. Here, as in the case of the switch 1531 and 1532 the picture coding device 1500 the on / off of the switches 1631 and 1632 through the filter processing control unit 1660 controlled.

In particular, the switching processing unit in the filter processing control unit 1660 detects the priority of an image and the occupancy of one in this image decoding device 1600 CPU for each picture coding, determines the entry to be referenced in the IPB coding mode table, reads out the right column of the entry and judges whether the priority set for the picture type of the decoded picture is in the right column is or not. If included in the right column, the switching processing unit gives in the filter processing control unit 1660 a signal to turn on the switch 1631 and 1632 out. Thereby, the filter processing for the decoded image is executed and the filter processed decoded image is stored in the image memory 1641 saved. On the other hand, if it is not included in the right column, the switching processing unit gives in the filter processing control unit 1660 a signal to turn OFF the switches 1631 and 1632 out. Therefore, the filter processing for the decoded picture is skipped and the decoded picture is stored in the picture memory 1641 stored without being filter processed.

Such control is carried out for each picture, and decoded pictures which have been filter-processed and which have not been filter-processed are stored in the order in the picture memory 1641 saved.

Corresponding For image decoding, inter pixel filtering is used Noise removal or the like not executed every time but the inter-pixel filtering is selectively executed as needed, therefore it will, if the inter-pixel filtering only for images with a serious impact on image quality is executed, for example, too in a picture decoding apparatus having a low processability, possible, the picture quality the important images stored in the image memory to maintain an accumulation of the block distortion in the decoded ones To reduce images stored in the image memory, a prediction efficiency by the motion compensation unit to improve and an impairment the picture quality more than the MPEG technique and thus can reduce the big impact the picture quality improvement be achieved at a low bit rate.

Especially when running Inter-pixel filtering becomes an image of great influence other pictures priority given, that is, an intra-coded picture, a forward-predictively coded picture, a picture Basic layer image or the like, so that the effect of improving image quality, such as noise elimination, through an inter-pixel filter can be achieved more effectively, too with the same increase in processing load.

Also On / off filtering processing can be controlled to full use from the processing ability the image decoding device, so that the CPU with a higher Efficiency is used and thus decoding with higher image quality can also be used the same hardware resources be realized.

It It should be noted that the present invention not only as a such picture coding device or picture decoding device can be realized, but also as a picture coding method or an image decoding method for performing in these devices feature units included as steps or as a program for one Computer that performs these steps. And it does not require to say that the program over a recording medium such as a CD-ROM or a distribution medium how the internet can be distributed.

(Further examples)

In addition will it is possible if a program for realizing the structure of the picture coding method or the image decoding method in each of those described above embodiments recorded on a storage medium such as a floppy disk Processing as in any of the above embodiments easily in one independent Computer system.

20 Fig. 14 is a diagram showing a case where the processing is performed in a computer system using a floppy disk storing the image coding method or the picture decoding method in the above-mentioned embodiments.

20B shows the front view and the cross-sectional view of the appearance of a floppy disk and the floppy disk and 20A shows an example of a physical format of a floppy itself as a storage medium. A floppy disk FD is contained in a case F, a plurality of tracks Tr are formed concentrically on the surface of the disk in radial circumferentially, and each track is divided in the direction of circulation into 16 sectors. Therefore, for storing the above-mentioned program on the floppy disk, the picture coding method is recorded as the program in an area dedicated thereto for the floppy disk FD.

20C shows the structure for recording and reproducing the program on and from a floppy disk FD. To record the program on the floppy disk, the computer system Cs writes the picture coding method or the picture decoding method as a program to the floppy disk FD through a floppy disk drive. To create the above image coding method in the computer system by the program recorded on the floppy disk, the program is read from the floppy disk by the floppy disk drive and transmitted to the computer system.

It It should be noted that the above explanation is given under the assumption is that a recording medium is a floppy disk, but the same processing can also be performed using an optical disk. additionally the recording medium is not limited to these, but any other medium such as an IC card or a ROM cassette can used in the same way when a program is on it can be recorded.

21 ~ 24 Fig. 15 are diagrams of the apparatuses for performing the encoding or decoding processing as shown in the above-mentioned embodiments and the system using the same.

21 Fig. 10 is a block diagram showing the overall construction of a content providing system ex100 for realizing the content distribution service. The area for providing a communication service is divided into cells of a desired size, and base stations ex107~ex110, which are fixed wireless stations, are placed in the respective cells. This content providing system ex100 is, for example, connected through the Internet ex101, an Internet service provider ex102 and a telephone network ex104 connected to a computer ex111, a PDA (Personal Digital Assistant) ex112, a camera ex113 and a mobile phone ex114. However, the content delivery system ex100 is not based on the combination, as in 21 shown, limited and can be combined with a combination of each of them. It may also be connected directly to the telephone network ex104, not through the base stations ex107 ~ ex110, which are fixed wireless stations.

The Camera ex113 is a device like a digital video camera that is capable of moving pictures record. The mobile phone can be any PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband Code Division Multiple Access) system or a GSM (Global System for Mobile Communications) system, a PHS (Personal Handyphone System) and the like.

A streaming server ex103 is also connected to the camera ex113 through the base station ex109 and the telephone network ex104, which has a live distribution or the like using the camera ex113 based on that of the user transmitted coded data allows. Either the camera ex113 or the server for transmitting the data can encode the data recorded by the camera. Also, moving picture data taken by a camera ex116 may be transmitted to the streaming server ex103 by the computer ex111. The camera ex116 is a device like a digital camera, which is able to take still and moving pictures. In this case, either the camera ex116 or the computer ex111 can encode the data of the moving pictures. An LSI ex117 included in the computer ex111 or the camera ex116 performs the encoding processing. It should be noted that image encoding and decoding software may be integrated in any type of storage medium (such as a CD-ROM, a floppy disk and a hard disk), that is, a recording medium that can be read by the computer ex111 or the like , Furthermore, the camera-equipped mobile phone ex115 can transmit the motion picture data. These moving picture data are the data encoded by the LSI included in the mobile phone ex115.

22 Fig. 13 is a diagram showing an example of a cellular phone ex115. The mobile phone ex115 has an antenna ex201 for transmitting and receiving radio waves between the base station ex110, a camera unit ex203 such as a CCD camera capable of recording video and still images, a display unit ex202 such as a liquid crystal display for displaying the Decoding the video obtained by the camera ex203, video received by the antenna ex201 or the like, a main body ex204 having a set of operation keys, a voice output unit ex208 such as a speaker for outputting voice, a voice input unit ex205 such as a microphone for inputting voice Language, a storage medium ex207 for storing encoded or decoded data, such as data of camera-acquired, moving or still images, of received emails, and a chute unit ex206 for inserting the storage medium ex207 into the mobile phone ex115. The storage medium ex207 includes a flash memory element, a type of EEPROM (Electrically Erasable and Programmable Read Only Memory), that is, an electrically erasable and rewritable nonvolatile memory in a plastic housing such as an SD card.

In This content delivery system ex100 will be used by the users using the camera ex113, the camera ex116 or the like recorded content (like a live music video) in the same way as in the above embodiments encoded and transmitted to the streaming server ex103 while the Streaming Server ex103 a stream distribution of content data to the Execute clients on their request. The clients include the computer ex111, the PDA ex112, the camera ex113, the mobile phone ex114 and so on, which are capable of the above to decode encoded data. The content delivery system ex100 is a system in which the clients are thus the encoded ones Can receive and play data and continue to receive the data receive, decode and play back in real time a personal one To realize broadcasting.

Furthermore, the mobile phone ex115 is based on 23 explained. In the mobile phone ex115, a main control unit ex311 for overall control of each unit of the display unit ex202 and the main body ex204 to a power supply circuit unit ex310, an operation input control unit ex304, an image coding unit ex312, a camera interface unit ex303, an LCD (liquid crystal display) control unit ex302, a picture decoding unit ex309, a multiplexing / demultiplexing unit ex308, a recording / reproducing unit ex307, a modem switching unit ex306, and a voice processing unit ex305 connected via a synchronous bus ex313. When an end-of-conversation key or a power-on key is turned ON by the operation of a user, the power supply circuit unit ex310 supplies power to corresponding units from a battery to put the camera-equipped digital mobile telephone ex115 in a standby state. In the mobile phone ex115, the voice processing unit ex305, under control of the main control unit ex311, converts the voice signals received from the voice input unit ex205 into digital voice data with a CPU, a ROM and RAM, the modem switching unit ex306 performs spread spectrum processing of the digital voice data and the transmission / reception circuit unit ex301 performs digital-to-analog conversion and frequency transformation of the data to be transmitted through the antenna ex201. In the mobile phone ex115, the data received by the antenna ex201 in the conversation mode is also amplified, and frequency transformation and analog-to-digital conversion are performed, the modem circuit ex306 performs inverse spread spectrum processing of the data, and the voice processing unit ex305 converts the analog voice data to output the result through the speech output unit ex208. Further, when transmitting an e-mail in the data communication mode, the text data of the e-mail inputted by operating the operation keys of the main body ex204 by the operation input control unit ex304 becomes the Main control unit ex311 sent. In the main control unit ex311, after the modem circuit unit ex306 has performed the spread spectrum processing of the text data and the transmission / reception circuit unit ex301 has performed the digital / analog conversion and the frequency transformation thereof, the result is sent to the base station by the antenna ex201 transmitted ex110.

If the image data is transmitted in the data communication mode, the from the camera unit ex203 recorded image data of the image encoding unit ex312 provided by the camera interface unit ex303. If the image data is not transferred can, can the image data taken by the camera unit ex203 by the Camera interface unit ex303 and the LCD control unit ex 302 are displayed directly on the display unit ex202.

The Image encoding unit ex312 compresses and encodes that of the Camera unit ex203 provided image data by the encoding method, such as in the above mentioned embodiments shown to transform into encoded image data, and sends to the multiplex / demultiplex unit ex308. At this time the mobile phone ex115 transmits during the image acquisition the camera unit ex203 received from the voice input unit ex205 Speech by the language processing unit ex305 as digital Voice data to the multiplexing / demultiplexing unit ex308.

The Multiplex / Demultiplex unit ex308 multiplexes those from the image coding unit ex312 provided coded image data and that of the language processing unit ex305 provided voice data according to a predetermined method, the Modem circuit unit ex306 leads a spread spectrum processing the resulting multiplex data and the transmission / reception circuit unit ex301 leads a digital-to-analog conversion and frequency transformation of the result for transmission through the antenna ex201.

As for receiving data of a moving picture file which with a website or similar connected is, in data communication mode, performs the modem circuit unit ex306 a spread spectrum processing of the base station ex110 data received by the antenna ex201 and transmits the resulting multiplexed data to the multiplex / demultiplex unit ex308.

Around the multiplexed data received by antenna ex201 The multiplexed / demultiplexed unit ex308 demultiplexes the multiplexed data into coded image data and speech data and represents the coded image data for the image decoding unit ex309 and the speech data for the speech processing unit ex305 through the synchronous bus ex313 ready.

When next The image decoding unit ex309 decodes the encoded image data by the decoding method paired with the coding method, as in the above mentioned embodiments to produce and display reproduced image data Data by the LCD control unit ex302 for the display unit ex202 ready and thus become in one example with a Website linked Motion picture data included in the motion picture file. At the same time converts the voice processing unit ex305 converts the voice data into analog Voice data and provides this data for the voice output unit ex208 ready and thus be in one for example with a website linked Play back voice data contained in the motion picture file.

The present invention is not limited to the above-mentioned system. Ground-based or satellite digital broadcasting has recently been on the news, and at least one of the image coding method and the picture decoding method in the above-mentioned embodiments may be included in such a digital broadcasting system as shown in FIG 24 shown. More specifically, an encoded bit stream of video information from a broadcasting station ex409 is transmitted by radio waves or communicated with a broadcasting satellite ex410. Upon receiving it, the broadcasting satellite ex410 transmits radio waves for broadcasting, a house antenna ex406 having a satellite broadcast receiving function receives the radio waves, and a device such as a television receiver ex401 or a set-top box (STB) ex407 decodes the coded bit stream for playback , The image decoding apparatus as shown in the above-mentioned embodiments may be implemented in the reproducing apparatus ex403 for reading out a bit stream recorded on a storage medium ex402, that is, a recording medium such as a CD or DVD, and decoding it. In this case, the reproduced video signals are displayed on a monitor ex404. It is also conceivable to implement the image decoding apparatus in the set-top box ex407 connected to an cable ex405 for a cable television or the antenna ex406 for satellite and / or land-based broadcasting to display it on a monitor ex408 of the television ex401. The image decoding apparatus may be included in the television, not in the set-top box. Or a car ex412 with an antenna ex411 can send signals from the ex410 satellite, the base station ex107 or the like for reproducing moving pictures on one Display device such as a car navigation device ex413 or the like in the car ex412 received.

Farther For example, as shown in the above-mentioned embodiments, the picture coding apparatus can Encoding image signals for recording on a recording medium. As a concrete example, there is a recorder ex420, such as a DVD recorder for recording image signals on a DVD disc ex421 and a disk recorder to record them on one Hard disk. You can as well be recorded on an SD card ex422. If the recorder ex420 the image decoding apparatus as in the above-mentioned embodiments shown, contains, can those recorded on the ex421 dvd disk or the ex422 sd-card Image signals to be displayed on the monitor ex408.

It should be noted, however, that the structure of the car navigation device ex413 is the same as that of the mobile phone ex115, such as in FIG 23 the structure without the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 is shown as the units as shown in FIG 23 shown, is conceivable. The same applies to the computer ex111, the TV (receiver) ex401 and others.

In addition are three types of implementations for a port like the one above mentioned Mobile phone ex114 imaginable; one with a coder and one Decoder equipped send / receive port, one with only an encoder equipped transmitting connection and one with only one Decoder equipped receiving terminal.

As described above it becomes possible any type of device or system as in the present embodiment shown by implementing the coding method and decoding method, as shown in the present specification.

industrial applicability

The Image decoding and encoding method according to the present invention can be in a personal computer, a PDA and a mobile phone be included with a communication function.

Claims (1)

Method for decoding encoded data, wherein the method comprises: Decoding coded data (Bitstream4) to obtain a decoded image (Recon) with or without use a predictive Picture (Pred), being the predictive Image (Pred) is generated with reference to a reference image (Ref); Filter the decoded picture (Recon); and Save the decoded Image (FilteredImg5), which is filtered for use as a reference image (Ref) to the following encoded data (Bitstream4) to decode, characterized by Perform the filtering of a decoded one Picture with a higher Smoothing, if the decoded image (Recon) in a decoding process without Use of the predictive Picture (Pred) is decoded when filtering a decoded Image (Recon), which is used in a decoding process of the predictive Picture (Pred) is decoded.