JP2003289544A - Equipment and method for coding image information, equipment and method for decoding image information, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a memory size required for coding and decoding processing while minimizing deterioration in coding efficiency and image quality in multiple frame estimation. <P>SOLUTION: In the image information coding equipment 10, reference image information stored in a short-term frame memory 22 is stored in a long-term frame memory 22 by LPIR and LPIN fields in image compression information. At this time, at least one portion of the image information to be stored in the long-term frame memory 22, for example down sampling is performed at an information reduction section 21, and the amount of information is reduced for storing. The reference image information where the down sampling is carried out by the information reduction section 21 is subjected to, for example, up sampling at an information development section 23, and then movement estimation/compensation processing are carried out at a movement estimation/ compensation section 24. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to H.264. 26L or the like, image compression information (bit stream) compressed by orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform and motion compensation using multiple frame prediction is transmitted to satellite broadcasting, cable TV or the Internet. The present invention relates to an image information encoding device and method, an image information decoding device and method, and a program used when receiving via a network medium or when processing on a storage medium such as an optical disk, a magnetic disk, or a flash memory. It is a thing.

[0002]

2. Description of the Related Art In recent years, image information has been treated digitally, with the purpose of transmitting and storing highly efficient information.
MPEG that compresses by orthogonal transform such as discrete cosine transform and motion prediction / compensation using redundancy peculiar to image information
A device that conforms to the method such as
It is becoming widespread both for receiving information and for general households.

In particular, MPEG2 (ISO / IEC 13818-2)
Is defined as a general-purpose image coding method, and is a standard that covers both interlaced scan images and progressive scan images, as well as standard resolution images and high-definition images, and is currently widely used in a wide range of professional and consumer applications. Has been. By using the MPEG2 compression method, for example, a standard resolution interlaced scan image having 720 × 480 pixels has 4 to 8 Mbps,
For a high-resolution interlaced scanning image having 1920 × 1088 pixels, a high compression rate and good image quality can be realized by assigning a code amount (bit rate) of 18 to 22 Mbps.

Although MPEG2 is mainly intended for high image quality coding suitable for broadcasting, it does not support a coding amount (bit rate) lower than that of MPEG1, that is, a coding system having a higher compression rate. However, with the widespread use of mobile terminals, the need for such an encoding method is expected to increase in the future, and the MPEG4 encoding method has been standardized in response to this. Regarding the image encoding method, 1998 1
In February, the standard was approved as an international standard as ISO / IEC 14496-2.

Furthermore, in recent years, H.26L (ITU-T Q6 / 16 VCEG) was originally designed for the purpose of image coding for video conferences.
That standard is being standardized. H. 26L is MPE
It is known that higher encoding efficiency can be realized, though a larger amount of calculation is required for encoding and decoding compared with conventional encoding methods such as G2 and MPEG4. Also, as a part of the activities of MPEG4, this H.264 standard is currently being used. H.26L as a base. The Joint Model of Enhanced-Compression Video Co is standardization that incorporates functions not supported by 26L and achieves higher coding efficiency.
It is done as ding.

Here, a schematic configuration of an image information coding apparatus for realizing image compression by orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform and motion prediction / compensation is shown in FIG.
It shows in 0. As shown in FIG. 10, the image information encoding device 1
00 is an A / D converter 101, a screen rearrangement buffer 102, an adder 103, an orthogonal transformer 104, a quantizer 105, a lossless encoder 106, and a storage buffer 1.
07, an inverse quantization unit 108, an inverse orthogonal transform unit 109,
A frame memory 110, a motion prediction / compensation unit 111,
The rate control unit 112 is included.

In FIG. 10, the A / D converter 101 is
The input image signal is converted into a digital signal. Then, the screen rearrangement buffer 102 receives the GOP (Gr (Gr) of the image compression information output from the image information encoding device 100).
Frames are rearranged according to their structure. Here, the screen rearrangement buffer 102 supplies the image information of the entire frame to the orthogonal transformation unit 104 for the image to be intra-coded. Orthogonal transformation unit 1
Reference numeral 04 performs orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform on the image information, and supplies the transform coefficient to the quantization unit 105. The quantization unit 105 performs a quantization process on the transform coefficient supplied from the orthogonal transform unit 104.

The lossless coding unit 106 performs lossless coding such as variable length coding and arithmetic coding on the quantized transform coefficient, and supplies the coded transform coefficient to the accumulation buffer 107 for accumulation. Let The encoded transform coefficient is output as image compression information.

The behavior of the quantizer 105 is as follows:
Controlled by 12. Further, the quantization unit 105 supplies the quantized transform coefficient to the inverse quantization unit 108, and the inverse quantization unit 108 inversely quantizes the transform coefficient. The inverse orthogonal transform unit 109 performs inverse orthogonal transform processing on the inversely quantized transform coefficient to generate decoded image information, and supplies the information to the frame memory 110 for storage.

On the other hand, the screen rearrangement buffer 102 supplies the image information to the motion prediction / compensation unit 111 for the image to be inter-coded. Motion prediction / compensation unit 1
Reference numeral 11 denotes the image information that is simultaneously referred to by the frame memory 1.
Then, reference image information is generated by performing motion prediction / compensation processing. The motion prediction / compensation unit 111 supplies this reference image information to the adder 103, and the adder 103
The reference image information is converted into a difference signal with the image information.
Further, the motion compensation / prediction unit 111 simultaneously supplies the motion vector information to the lossless encoding unit 106.

The lossless coding unit 106 performs lossless coding processing such as variable length coding or arithmetic coding on the motion vector information to form information to be inserted in the header section of the image compression information. Note that the other processing is the same as that of the image compression information that is subjected to intra coding, and thus the description thereof is omitted.

Subsequently, the above-mentioned image information coding apparatus 10
FIG. 11 shows a schematic configuration of the image information decoding device corresponding to 0. As shown in FIG. 11, the image information decoding device 120
A storage buffer 121, a lossless decoding unit 122, an inverse quantization unit 123, an inverse orthogonal transformation unit 124, an adder 125,
Screen rearrangement buffer 126 and D / A conversion unit 127
, Motion prediction / compensation unit 128, and frame memory 129
It is composed of and.

In FIG. 11, the accumulation buffer 121 is
The input image compression information is temporarily stored and then transferred to the lossless decoding unit 122. The lossless decoding unit 122 performs variable length decoding or arithmetic decoding processing on the image compression information based on the determined format of the image compression information, and supplies the quantized transform coefficient to the inverse quantization unit 123. . Further, when the frame is inter-coded, the lossless decoding unit 122 also decodes the motion vector information stored in the header part of the image compression information, and the information is motion prediction / compensation unit 128. Supply to.

The inverse quantization unit 123 inversely quantizes the quantized transform coefficient supplied from the lossless decoding unit 122 and supplies the transform coefficient to the inverse orthogonal transform unit 124. Inverse orthogonal transform unit 12
Reference numeral 4 applies inverse orthogonal transform such as inverse discrete cosine transform or inverse Karhunen-Loeve transform to the transform coefficient based on the determined image compression information format.

Here, when the frame is intra-coded, the image information subjected to the inverse orthogonal transform process is stored in the screen rearrangement buffer 126 and D
It is output after the D / A conversion processing in the / A conversion unit 127.

On the other hand, when the frame is inter-coded, the motion prediction / compensation unit 128 uses the motion vector information that has undergone the lossless decoding process and the image information stored in the frame memory 129. A reference image is generated based on the generated reference image and is supplied to the adder 125. The adder 125 synthesizes this reference image and the output of the inverse orthogonal transform unit 124.
Note that the other processing is the same as that of the intra-coded frame, and therefore the description thereof is omitted.

H. One of the elemental technologies for achieving high coding efficiency in 26L is motion prediction compensation based on a variable block size. Currently, seven types of motion prediction compensation block sizes as shown in FIG. 12 are defined. ing.

In addition, the H. In 26L, high-precision motion prediction compensation processing such as 1/4 pixel accuracy and 1/8 pixel accuracy is defined. In the following, first, the motion prediction / compensation process with quarter-pixel accuracy will be described.

H. FIG. 13 shows the motion prediction / compensation process with the 1/4 pixel precision defined in 26L. When generating a predicted image with 1/4 pixel accuracy, first, based on the pixel value stored in the frame memory, a pixel value with 1/2 pixel accuracy is obtained by using an FIR filter with 6 taps in each of the horizontal and vertical directions. To generate. Here, as the coefficients of the FIR filter, those shown in the following equation (1) are defined.

[0020]

[Equation 1]

Then, based on the generated prediction image of 1/2 pixel precision, a prediction image of 1/4 pixel precision is generated by linear interpolation.

In addition, the H. In 26L, in order to perform motion prediction compensation with 1/8 pixel accuracy, the filter bank shown in the following Expression (2) is defined.

[0023]

[Equation 2]

In the image compression information, regarding the accuracy of the motion vector, RTP (Real-time Transfer Protocol) is used.
col) is specified by the Motion Resolution field in the layer.

[0025]

[Problems to be Solved by the Invention] 26L
In the same manner as in MPEG2, the above-mentioned includes a regulation regarding B picture. H. FIG. 14 shows a bidirectional prediction method using B pictures in 26L. As shown in FIG. 14, the B ₂ picture and the B ₃ picture use the I ₁ picture and the P ₄ picture as reference images, and the B ₅ picture and the B ₆ picture use P ₄ picture and P ₇ picture as reference images. I am trying.

Further, in the image compression information, the use of each picture is designated by PTYPE in the picture header as shown in FIG. As shown in FIG.
When the value of Code number is 0 or 1, the use of P picture is designated, when the value of Code number is 2, the use of I picture is designated and the value of Code number is 3 or 4.
When, the use of B picture is specified. On this occasion,
When the Code number value is 0, only the immediately preceding picture is used for prediction, whereas the Code number value is 1.
When, a plurality of past pictures are used for prediction. When the Code number is 3, the pictures immediately before and immediately after are used for prediction, while Code n
When the value of umber is 4, a plurality of past and future pictures are used for prediction. In this way, multiple frame prediction can be used for B pictures as well as P pictures.

In the multiple frame prediction, as shown in FIG. 16, the motion prediction / compensation processing is performed in block units using a plurality of reference frames for one frame, thereby improving the coding efficiency. This is a technique in which the immediately preceding and / or the immediately following picture is stored in the short term frame memory, and the other past and / or future pictures are stored in the long term frame memory. Regarding this multiple frame prediction, for example, the document “Long
-Term Memory Motion -Compensated Prediction ”(T.Wi
egand et al., IEEE Trans. on Circuit and Systems f
or Video Technology, Vol.9, No.1, Feb.1999, pp.70-
83) ”and the like.

Hereinafter, H.26L Annex U (ITU-T SC16 VCEG-
O10.doc) describes the buffer management and decoding processing for multiple frame prediction specified in O10.doc).

First, the syntax of information embedded in image compression information for buffer management in multiple frame prediction will be described. Note that the fields in the following syntax exist in the slice layer, and UVLC (Univ
ersal Variable Length Code) is encoded by a variable length coding method called.

PN (Picture Number) is from 0 to ^2x-
It is a unique value for each picture that takes a value up to 1, and that value increases by 1 in time series. Therefore, in the multiple frame buffer, 2 ^x -1
It is not possible to secure more than one image as a reference image.

RPSL (Reference Picture Selection)
The presence or absence of Layer) is specified by the following values. That is, RPSL is not transmitted when the Code Number value is 0, and is transmitted when the Code Number value is 1. Here, when the RPSL is not transmitted, the settings before the slice are also applied to the slice. Further, when RPSL is transmitted, buffer management for multiple frame prediction regarding the slice is performed using the fields described below.

RMPNI (Re-Mapping of Picture) is applied to slices included in P pictures or B pictures.
There is a field called Numbers Indicator). Here, as shown in FIG. 17, when the code number values are 0 and 1, ADPN (Absolute Difference of
Picture Numbers) field exists, and its value represents a negative value and a positive value, respectively. Also, the value of Code number is 3
, The LPIR (Long-term Picture Index for
The Re-Mapping field specifies the long-term index for the predicted frame and Cod
When the value of e number is 4, the remapping (Re-Mappi
ng) The loop ends. The ADPN field and LPIR field will be described below.

Code Number in ADPN field
Represents the value of ADPN-1. ADPN indicates the absolute value of the difference between the PN of the predicted image and the PN of the image. If the ADPN field has not been transmitted before that, the value indicates the PN of the image.

If the PN of the reference image is PNP and the PN of the image in question is PNQ, the ADPN field decoding process can be performed using the technique shown in the following pseudo code. It is possible. if (RMPNI == '1') {// a negative difference if (PNP-ADPN <0) PNQ = PNP-ADPN +1024; else PNQ = PNP-ADPN;} else {// a positive difference if (PNP + ADPN> 1023) PNQ = PNP + ADPN-1024; else PNQ = PNP + ADPN;} In addition, as an example of the processing on the encoder side, the following pseudo code can be used. Here, abs () is a function whose return value is the absolute value of (). For the ADPN obtained in this way, ADP
Information to be embedded in the image compression information can be generated by performing variable length coding on N-1 by UVLC.

The remapping of indexes in long term memory is specified by a field called LPIR.

Regarding how to store the image currently being decoded in the frame memory, see R
It is specified in a field called PBT (Reference Picture Buffering Type). That is, its Code
If the value of Number is 0, First-In-First-Out, that is, the oldest reference image in the frame memory is discarded, and the image currently being decoded is stored in the frame as a new reference image. To be done. When the value of the Code Number is 0, the adaptive memory control (Adaptive M
Since the emory control) process is performed, the image is handled by the MMCO (Memory Managem) shown in FIG.
The operation is defined by a field called ent control operation). Note that a plurality of MMCO fields may be present following the RPBT field.

As shown in FIG. 18, when the value of Code Number is 1, one of the short-term pictures is regarded as "Unused" and discarded, and when the value of Code Number is 2, One of the long-term pictures is "Unus
ed ”and discarded. Also, the Code Number value is 3
In the case of, a long-term index (Long-Term Index-Decimated) is assigned to the picture. If the Code Number value is 4, MLIP
1 (Maximum Long-Term Picture Index Plus 1) defines the maximum value of the index for the downsampled long-term picture. In addition, MLI
P1 is a field that defines the maximum value of the index for a long-term picture, and the initial value is
"0" is set.

Further, in FIG. 18, DPN (Differen
ce of Picture Number) is the PN for MMCO operation
Is a field for calculating. On the decoding side, the PN (PNQ) of the image in question can be decoded using the DPN field by the method as shown in the pseudo code below. The processing on the encoding device side can be described by the following pseudo code.

[0039] The index for long-term pictures is LPIN.
(Long-term Picture Index).

Next, the decoding process of multiple frame prediction will be described. The decoding process for the multiple frame prediction is performed based on the information described in the above syntax and embedded in the image compression information for buffer management in the multiple frame prediction. The multi-frame buffer is composed of a short term buffer and a long term buffer. The decoded image is stored in the short term buffer by default, and is stored in the long term buffer by the MMCO command described above. As a result of the decoding process, when the frame is lost due to a packet loss or the like, information about the immediately preceding frame stored in the buffer is copied to the buffer for the frame. Information about the reference frame for each macroblock is described in the macroblock layer in the image compression information.

By the way, the multiple frame prediction whose concept is shown in FIG. 16 has an advantage of improving the coding efficiency, but has a problem that a large capacity frame memory is required for the coding process and the decoding process. . Therefore, there is a problem that the cost is increased on the encoding side.

In addition, the H. When the terminal that receives the image compression information encoded by the 26L is a personal computer or the like, since the specifications of the terminal are different, there is a possibility that a frame memory for multiple frame prediction cannot be secured. There is.

The present invention has been proposed in view of such a conventional situation, and when performing multiple frame prediction, encoding processing and decoding are performed while minimizing deterioration of encoding efficiency or image quality. An object of the present invention is to provide an image information encoding apparatus and method, an image information decoding apparatus and method, and a program that reduce the memory size required for processing.

[0044]

In order to achieve the above-mentioned object, an image information coding apparatus according to the present invention uses orthogonal transformation of input image information and prediction of past plural frames and / or future plural frames. An image information coding apparatus that generates image compression information by performing compression coding by use of motion prediction / compensation that can be used, and stores at least a part of the reference images before storing the reference images in the frame memory. An information reducing means for reducing the information amount, an information extending means for extending the reference image of which the information amount has been reduced by the information reducing means, and an extension by the reference image and / or the information extending means in the frame memory. Further, it is characterized by comprising a motion prediction / compensation means for performing motion prediction / compensation processing using the reference image.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and the information reducing means is at least a part of the reference image stored in the long-term frame memory or Reduce the total amount of information.

The information reducing means may reduce the information amount of the reference image by downsampling, and the information extending means may extend the reference image having the reduced information amount by upsampling. The information reducing unit may use a lossless encoding method and / or an orthogonal transform method as the information reducing method, and the information extending section may use a lossless decoding method and / or an inverse orthogonal transform method as the information extending method. .

Such an image information coding apparatus uses downsampling or the like to perform at least a long-term frame when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. The amount of information of the reference image stored in the memory is reduced, and when the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

In order to achieve the above-mentioned object, the image information coding method according to the present invention can use the input image information for orthogonal transformation and prediction of past plural frames and / or future plural frames. It is an image information encoding method that generates image compression information by performing compression encoding with various motion prediction / compensation, and reduces the information amount of at least a part of the reference images before storing the reference images in the frame memory. Information reducing step, an information extending step of extending the reference image whose information amount is reduced in the information reducing step, and the reference image in the frame memory and / or the information extending in the information extending step. It is characterized by having a motion prediction / compensation step of performing motion prediction / compensation processing using a reference image.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reduction step, and the reference image having the reduced information amount may be extended by upsampling in the information extension step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

In such an image information coding method, when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction, at least a long term frame is obtained by downsampling or the like. The information amount of the reference image stored in the memory is reduced, and when the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

In addition, in order to achieve the above-mentioned object, the program according to the present invention is a motion estimation / prediction that can use orthogonal transformation of input image information and prediction of past plural frames and / or future plural frames. A program that causes a computer to execute a process of compression-encoding by compensation and generating image compression information, and information for reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory. A reduction step, an information extension step of extending the reference image whose information amount has been reduced in the information reduction step, the image information in the frame memory and / or the reference image extended in the information extension step It has a motion prediction / compensation process for performing motion prediction / compensation processing using.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reduction step, and the reference image having the reduced information amount may be extended by upsampling in the information extension step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

Such a program is stored in at least a long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. When the predicted image is generated by reducing the amount of information of the reference image, the computer is caused to execute a process of extending the reduced reference image by upsampling or the like.

Further, in order to achieve the above-mentioned object, the image information decoding apparatus according to the present invention uses the image compression information generated in the image information encoding apparatus for motion estimation / inversion using inverse orthogonal transform and multiple frame prediction. An image information decoding device for decoding by compensation, wherein an information reducing means for reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory, and the information amount by the information reducing means. Information extension means for extending the reduced reference image, and motion prediction / compensation processing for performing motion estimation / compensation processing using the reference image in the frame memory and / or the reference image extended by the information extension means. Compensation means are provided. here,
The frame memory is composed of a short-term frame memory and a long-term frame memory,
The information reducing unit reduces at least the information amount of a part or all of the reference image stored in the long-term frame memory.

The information reducing means may reduce the information amount of the reference image by downsampling, and the information extending means may extend the reference image having the reduced information amount by upsampling. The information reducing unit may use a lossless encoding method and / or an orthogonal transform method as the information reducing method, and the information extending section may use a lossless decoding method and / or an inverse orthogonal transform method as the information extending method. .

Such an image information decoding device uses at least a long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. By reducing the amount of information in the reference image stored in the
Extend the reduced reference image.

Further, in order to achieve the above-mentioned object, the image information decoding method according to the present invention is such that the image compression information generated in the image information encoding device is subjected to inverse orthogonal transform and past plural frames and / or future frames. An image information decoding method for decoding by motion prediction / compensation capable of using a plurality of frames for prediction, and reducing the amount of information of at least a part of the reference images before storing the reference images in the frame memory. Information reducing step, an information extending step of extending the reference image whose information amount is reduced by the information reducing means, the reference image in the frame memory and / or the reference image extended by the information extending means It has a motion prediction / compensation process for performing motion prediction / compensation processing using.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reduction step, and the reference image having the reduced information amount may be extended by upsampling in the information extension step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

In such an image information decoding method, when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction, at least a long-term frame memory is downsampled. The amount of information in the reference image stored in
When the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

Further, in order to achieve the above-mentioned object, the program according to the present invention performs an inverse orthogonal transformation on the image compression information generated in the image information encoding device and performs a plurality of past frames and / or a plurality of future frames. A program that causes a computer to execute a process of decoding by motion prediction / compensation that can be used for prediction, and reduces the amount of information of at least a part of the reference images before storing the reference images in the frame memory. Information reduction process,
An information stretching step of stretching the reference image of which the amount of information has been reduced by the information reducing means, and a motion prediction using the reference image in the frame memory and / or the reference image extended by the information extending means. It is characterized by having a motion prediction / compensation step of performing compensation processing.

Here, the frame memory is
It consists of short-term frame memory and long-term frame memory.
At least part or all of the information amount of the reference image stored in the long-term frame memory is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reducing step, and the reference image having the reduced information amount may be extended by upsampling in the information extending step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

Such a program is stored in at least a long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. When the predicted image is generated by reducing the amount of information of the reference image, the computer is caused to execute a process of extending the reduced reference image by upsampling or the like.

[0067]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Specific embodiments to which the present invention is applied will be described below in detail with reference to the drawings. The first embodiment described below is a method according to the present invention. Two
An image information coding apparatus for generating image compression information by orthogonal transformation such as discrete cosine transformation or Karhunen-Loeve transformation and motion compensation using multiple frame prediction, such as 6L, and an image for decoding the image compression information It is applied to an information decoding device.

In the image information coding apparatus and the image information decoding apparatus according to the first embodiment, as will be described later,
Of the short-term frame memory and the long-term frame memory used for multiple frame prediction, at least a part of the image information stored in the long-term frame memory is encoded by reducing the amount of information stored. The memory size required for the processing and the decoding processing can be reduced.

First, FIG. 1 shows a schematic configuration of an image information coding apparatus according to the first embodiment. As shown in FIG. 1, the image information encoding device 10 according to the first embodiment.
Is an A / D converter 11 and a screen rearrangement buffer 12
, Adder 13, orthogonal transformation unit 14, and quantization unit 15
A lossless encoding unit 16, a storage buffer 17, an inverse quantization unit 18, an inverse orthogonal transformation unit 19, and a short term (Sh
ort-Term) frame memory 20, information reduction unit 21,
It is composed of a long-term frame memory 22, an information extension unit 23, a motion prediction / compensation unit 24, and a rate control unit 25.

In FIG. 1, the A / D converter 11 converts the input image signal into a digital signal. Then, the screen rearrangement buffer 12 uses the image information encoding device 1
GOP (Group of Pic) of image compression information output from 0
tures) Frames are rearranged according to their structure. Here, the screen rearrangement buffer 12 supplies the image information of the entire frame to the orthogonal transformation unit 14 for the intra-frame coded image (hereinafter referred to as I picture) on which intra coding is performed. The orthogonal transform unit 14 performs orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform on the image information, and supplies the transform coefficient to the quantizer 15.

The quantizing unit 15 performs a quantizing process on the transform coefficient supplied from the orthogonal transforming unit 14.

The lossless coding unit 16 performs lossless coding such as variable length coding or arithmetic coding on the quantized transform coefficient, and supplies the coded transform coefficient to the accumulation buffer 17 for storage. Let The encoded transform coefficient is output as image compression information.

The behavior of the quantizer 15 is the rate controller 25.
Controlled by. Further, the quantization unit 15 supplies the quantized transform coefficient to the inverse quantization unit 18, and the inverse quantization unit 18
Dequantizes the transform coefficient.

The inverse orthogonal transform section 19 performs inverse orthogonal transform processing on the inversely quantized transform coefficient to generate decoded image information, and supplies the information to the short-term frame memory 20 for storage. This decoded image information is the L
It is transferred to the long term frame memory 22 by the PIR field and the LPIN field. At this time,
At least a part of the information is transferred after the information amount is reduced by the information reduction unit 21. As the information reduction unit 21, for example, a down sampler can be used.

On the other hand, the screen rearrangement buffer 12 has an interframe forward predictive coded image (hereinafter referred to as P picture) and an interframe bidirectional predictive coded image (hereinafter referred to as B picture) which are inter-coded. 2), the image information is supplied to the motion prediction / compensation unit 24. At the same time, the motion prediction / compensation unit 24 takes out the referred image information from the short-term frame memory 20 or the long-term frame memory 22, performs motion estimation / compensation processing, and generates reference image information. At this time, the information reduction unit 21
The decoded image information stored in the long-term frame memory 22 with the reduced amount of information is subjected to motion prediction / compensation processing by the motion prediction / compensation unit 24 after the information amount is expanded by the information expansion unit 23. Then, the reference image information is generated. As the information extension unit 23, for example, an upsampler can be used.

The motion prediction / compensation unit 24 supplies the reference image information to the adder 13, and the adder 13 converts the reference image information into a difference signal with the image information. Further, the motion compensation / prediction unit 24 simultaneously supplies the motion vector information to the lossless encoding unit 16.

The lossless coding unit 16 performs lossless coding processing such as variable length coding or arithmetic coding on the motion vector information to form information to be inserted in the header section of the image compression information. Note that the other processing is the same as the image compression information to which intra coding is applied, and therefore the description thereof is omitted.

Next, FIG. 2 shows a schematic configuration of the image information decoding apparatus according to the present embodiment. As shown in FIG. 2, the image information decoding device 30 according to the present embodiment has a storage buffer 31, a lossless decoding unit 32, an inverse quantization unit 33, an inverse orthogonal transformation unit 34, an adder 35, and a screen. Rearrangement buffer 36, D / A converter 37, motion prediction / compensation unit 38
The short term frame memory 39, the information reduction unit 40, the long term frame memory 41, and the information extension unit 42.

In FIG. 2, the storage buffer 31 temporarily stores the input image compression information and then transfers it to the lossless decoding unit 32.

The lossless decoding unit 32 performs variable length decoding or arithmetic decoding on the image compression information based on the determined format of the image compression information, and dequantizes the quantized transform coefficient into the inverse quantization unit 33. Supply to. Also, the lossless decoding unit 3
When the frame is a P picture or a B picture, 2 also decodes the motion vector information stored in the header part of the image compression information and supplies the information to the motion prediction / compensation unit 38.

The inverse quantization unit 33 inversely quantizes the quantized transform coefficient supplied from the lossless decoding unit 32, and supplies the transform coefficient to the inverse orthogonal transform unit 34. The inverse orthogonal transform unit 34 performs inverse orthogonal transform such as inverse discrete cosine transform or inverse Karhunen-Loeve transform on the transform coefficient based on the determined format of the image compression information.

Here, when the frame is an I picture, the inverse orthogonal transform section 34 supplies the image information after the inverse orthogonal transform processing to the short-term frame memory 39 and stores it therein. This image information is transferred to the long-term frame memory 41 by the above-mentioned LPIR field and LPIN field. At this time, at least part of it is transferred after the information amount is reduced by the information reduction unit 40. .

The inverse orthogonal transform unit 34 also supplies the image information after the inverse orthogonal transform processing to the screen rearrangement buffer 36. The screen rearrangement buffer 36 temporarily stores this image information and then supplies it to the D / A conversion unit 37. D / A
The conversion unit 37 performs D / A conversion processing on this image information and outputs it.

On the other hand, when the frame is a P picture or a B picture, the motion prediction / compensation unit 38 extracts the referred image information from the short term frame memory 39 or the long term frame memory 41, and the lossless decoding process. Motion estimation / compensation processing is performed based on the motion vector information that has been subjected to the motion vector information to generate reference image information. At this time, the image information stored in the long-term frame memory 41 after the information amount is reduced by the information reduction unit 40 is expanded by the information expansion unit 42, and then is transmitted to the motion prediction / compensation unit 38. Motion prediction / compensation processing is performed,
Reference image information is generated. The adder 35 synthesizes this reference image and the output of the inverse orthogonal transform unit 34. Note that the other processing is the same as that of the intra-coded frame, and therefore the description thereof is omitted.

By the way, as described above, in the image information encoding apparatus 10 according to the first embodiment, the reference image information stored in the short-term frame memory 22 is the LPIR field and the LPIN in the image compression information.
Depending on the field, it is stored in the long term frame memory 22. At that time, the long term frame memory 2 is stored.
At least a part of the image information stored in No. 2 is subjected to thinning processing in the information reduction unit 21 and stored in a form in which the amount of information is reduced.

The information reduction unit 21 has, as a component, a circuit for performing a thinning filter process as shown in the following expression (3), for example, image information is transferred from the short term frame memory 20 to the long term frame memory 22. Is transmitted, for at least some image information,
The horizontal direction and the vertical direction are downsampled to ½, and the amount of information is reduced to ¼.

[0087]

[Equation 3]

When the image compression information is an interlaced scanning image, downsampling processing is performed for each field. At this time, for both the first field and the second field, a pixel value having a phase as shown by a shaded circle in FIG. 3A is generated using the filter defined by the equation (3). However, it may be stored in the long-term frame memory 22, but for example, it has a circuit for performing the filtering process for the second field as shown in the following formula (4) as a component, and the network of FIG. It is also possible to generate a pixel value having a phase as shown by a circle that has been multiplied and store it in the long-term frame memory 22.

[0089]

[Equation 4]

In the above example, 2: 1 down-sampling is performed in each of the horizontal direction and the vertical direction for both the case where the input image information is the progressive scanning image and the case where the image information is the interlaced scanning image. Although the information amount of the memory is reduced to 1/4, it is not limited to this. For example, the information amount of the frame memory is reduced to 1/2 by performing 2: 1 down sampling only in the horizontal direction.
May be reduced to. In particular, when the input image information is an interlaced scan image, it is possible to avoid the image quality deterioration due to the reduction of the amount of information by downsampling only in the horizontal direction.

Here, the short-term frame memory 2
When the thinning-out processing is performed only on a part of the image information of the predicted image information transferred from 0 to the long-term frame memory 22, the RMPNI described above with reference to FIG.
It is necessary to make the field and the MMCO field described with reference to FIG. 16 correspond to the image information in the down-sampled long-term frame memory 22.

Therefore, in the slice layer of the image compression information output from the image information encoding device 10, the LPIRD is set.
(Long-term Picture Index for Re-Mapping-Decimat
A new field called ed) is added. This LPIRD field represents the remapping of the index to the reference image subjected to the thinning processing in the long term frame memory 22. Code Number in the RMPNI field embedded in the image compression information
If the value of is 3, the LPIRD field specifies the long-term index for the downsampled prediction frame, as shown in FIG.

Information contained in the MMCO field embedded in the image compression information output from the image information encoding device 10 is shown in FIG. As shown in Figure 5, Code N
When the value of umber is 3, one of the thinned-out long-term pictures is set to "Unused" and discarded. When the value of Code Number is 6, a long-term index (decimated) for downsampling is assigned to the picture. If the Code Number value is 8,
MLIP1D (Maximum Long-Term Picture Index Plus
1-Decimated) specifies the maximum index value for downsampled long-term pictures. "0" is set as the initial value of the MLIP1D field.

In the image information encoding device 10, not only when the predicted image information in the short term frame memory 20 is transmitted to the long term frame memory 22, but also in the long term frame memory 22, a resolution equivalent to that of the input image information is obtained. It is possible to reduce the amount of information by performing a downsampling process also on the predicted image information having the. This operation is realized by an operation in which the value of Code Number is 4 in the MMCO field.

The reference image information thinned by the information reduction unit 21 is up-sampled by the information expansion unit 23, and then motion-predicted / compensated by the motion prediction / compensation unit 24. At this time, the interpolation processing in the information extension unit 23 and the motion prediction / compensation processing in the motion prediction / compensation unit 24 can be performed at the same time.
That is, when the motion vector information included in the image compression information has a 1/4 pixel precision, the upsampling and the motion prediction / compensation processing are performed by the 1/8 pixel precision filter expressed by the above-mentioned equation (2). When the motion vector information has 1/8 pixel precision, it is possible to predefine a filter with 1/16 pixel precision and perform upsampling and motion prediction / compensation processing at the same time. .

On the other hand, the image information decoding device 30 performs a decoding process of the image compression information output from the image information encoding device 10. At this time, the short term frame memory 39,
The operation principle in the information reduction unit 40 and the long term frame memory 41 is controlled by the information of the RMPNI field and the MMCO field included in the image compression information, as in the case of the image information encoding device 10.

When decoding the image compression information coded by the image information coding apparatus 100 which does not perform the multiple frame prediction as shown in FIG. 10, especially when the terminal is a personal computer or a PDA (Personal D).
In some cases, the image information decoding apparatus 120 shown in FIG. 11 may not be able to secure a frame memory for decoding. However, in this case, the image information decoding apparatus 30 shown in FIG. A decryption process can be performed. At this time, since the image compression information does not include the RMPNI field and the MMCO field described above, the short term frame memory 3
9, information reduction unit 40 and long-term frame memory 4
The operation in 1 is controlled by the resource manager in the terminal.

Next, the image information coding apparatus 50 shown in FIG. 6 as the second embodiment has the same basic structure as the image information coding apparatus 10 shown in FIG. 1, but a temporary memory 51.
And the information reduction unit 52, the amount of information is reduced not only in the predicted image information stored in the long-term frame memory 22 but also in at least a part of the predicted image information stored in the short-term frame memory 20. It is characterized by points.

Further, the image information decoding device 70 shown in FIG.
Has a basic configuration similar to that of the image information decoding apparatus 30 shown in FIG. 2, but has a temporary memory 71 and an information reduction unit 72, and not only the predicted image information stored in the long-term frame memory 41, Short term frame memory 3
9 is characterized in that the amount of information is reduced for at least a part of the predicted image information stored in 9.

Therefore, the same components as those of the image information coding apparatus 10 and the image information decoding apparatus 30 shown in FIGS. 1 and 2 are designated by the same reference numerals and their detailed description will be omitted.

In the image information coding apparatus 50 according to the second embodiment, the decoded image information which has been subjected to the inverse orthogonal transform process in the inverse orthogonal transform section 19 is temporarily stored in the temporary memory 51 and then short-circuited. The decoded image information is stored in the term frame memory 20. At that time, at least a part of the decoded image information is thinned by the information reduction unit 52, and the information amount is stored in a reduced form.

The reference image information stored in the short term frame memory 22 is stored in the long term frame memory 22 by the LPIR field and the LPIN field in the image compression information. At that time, the reference image information is stored in the long term frame memory 22. At least a part of the image information to be stored is subjected to thinning processing in the information reducing unit 21 and stored in a form in which the amount of information is reduced.

The reference image information subjected to the thinning processing by the information reducing units 21 and 52 is subjected to upsampling in the information expanding unit 23 and then subjected to motion prediction / compensation processing in the motion prediction / compensation unit 24. Be seen.

Here, in the image compression information output from the image information encoding device 50, the above-mentioned FIG. 4 and FIG.
Corresponding to, the RMPNI field and the MMCO field as shown in FIGS. 8 and 9 are included.

In FIG. 8, a new ADPND (Abso
lute Difference of Picture Numbers-Decimated) field has been added. The ADPND field is an index for the down-sampled predicted image information in the short-term frame memory 20, and the decoding process using the field conforms to the ADPN field.

Further, in FIG. 9, DPND (Differ
ence of Picture Number-Decimated) field is newly added. This field is DP except that it corresponds to the downsampled image in the short-term frame memory 20.
This is the same as the MMCO operation using the N field.

The short-term frame memory 20
When the predicted image information that has been down-sampled therein is transferred to the long-term frame memory 22, it is treated as down-sampled long-term predicted information. That is, when the data is transferred from the short-term frame memory 20 to the long-term frame memory 22, it is up-sampled and stored in the long-term frame memory 22 as predicted image information having a spatial resolution equivalent to that of the input image. Forbidden.

On the other hand, the image information decoding device 70 performs the decoding process of the image compression information output from the image information encoding device 50. At this time, the operation principle of the temporary memory 71, the short-term frame memory 39, the information reduction units 40 and 72, and the long-term frame memory 41 is the same as in the image information encoding device 50, that is, the RMPNI field included in the image compression information. And MMCO field information.

As described above, according to the image information coding devices 10 and 50 and the image information decoding devices 30 and 70 in this embodiment, when performing the motion prediction / compensation process using the multiple frame prediction, , Downsampling reduces the amount of information stored in the frame memory,
When generating a predicted image, the reduced image information is expanded by upsampling to reduce the memory size required for encoding and decoding while minimizing the deterioration of encoding efficiency and image quality. Can be reduced.

The present invention is not limited to the above-mentioned embodiments, and it goes without saying that various modifications can be made without departing from the gist of the present invention.

For example, in the above-described embodiment, the case where the down-sampling ratio of the reference image is 1/2 × 1/2 has been described, but the present invention is not limited to this, and the down-sampling ratio for the reference image is not limited to this. It is possible to define the sampling ratio. It is also possible to combine a plurality of values as the downsampling ratio on the encoding side and the decoding side.

Further, in the above-described embodiments, the description has been made assuming that downsampling and upsampling are used as the information reduction and information extension methods, but the invention is not limited to this, and for example, variable length coding method. The amount of information may be reduced by using a reversible coding method such as the above or an orthogonal transformation method such as a discrete cosine transformation. Further, the lossless information may be reduced by combining the lossless coding method and the orthogonal transformation method.

Further, in the above-mentioned embodiments, the hardware configuration has been described, but the present invention is not limited to this, and the processes in the image information encoding devices 10 and 50 and the image information decoding devices 30 and 70 are CPU (Cent
It can also be realized by causing a ral processing unit) to execute a computer program.

[0114]

As described in detail above, the image information coding apparatus according to the present invention is a motion capable of using orthogonal transformation of input image information and prediction of past plural frames and / or future plural frames. An image information encoding device that generates image compression information by compression encoding with prediction / compensation, and before storing a reference image in a frame memory,
An information reducing unit that reduces the information amount of at least a part of the reference image, an information extending unit that extends the reference image whose information amount has been reduced by the information reducing unit, the reference image in the frame memory, and And / or a motion prediction / compensation unit that performs motion prediction / compensation processing using the reference image expanded by the information expansion unit.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and the information reducing means is at least a part of the reference image stored in the long-term frame memory or Reduce the total amount of information.

The information reducing means may reduce the information amount of the reference image by downsampling, and the information extending means may extend the reference image with the reduced information amount by upsampling. The information reducing unit may use a lossless encoding method and / or an orthogonal transform method as the information reducing method, and the information extending section may use a lossless decoding method and / or an inverse orthogonal transform method as the information extending method. .

Such an image information coding apparatus uses downsampling or the like to perform at least a long term frame when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. The amount of information of the reference image stored in the memory is reduced, and when the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

As a result, the memory size required for the encoding process and the decoding process can be reduced while minimizing the deterioration of the encoding efficiency and the image quality.

Further, the image information coding method according to the present invention compresses the input image information by orthogonal transformation and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction. An image information encoding method for generating image compression information by converting the information into an information reduction step of reducing the information amount of at least a part of the reference images before storing the reference images in a frame memory, and the information reduction. An information extension step of extending the reference image whose information amount is reduced in the step, and motion prediction / compensation using the reference image in the frame memory and / or the reference image extended in the information extension step. It is characterized by having a motion prediction / compensation step of performing processing.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reducing step, and the reference image having the reduced information amount may be extended by upsampling in the information extending step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

In such an image information coding method, at the time of performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction, at least a long term frame is downsampled. The information amount of the reference image stored in the memory is reduced, and when the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

As a result, the memory size required for the encoding process and the decoding process can be reduced while minimizing the deterioration of the encoding efficiency and the image quality.

Further, the program according to the present invention is a motion prediction / input method capable of using orthogonal transformation of input image information and prediction of past plural frames and / or future plural frames.
A program that causes a computer to execute a process of compression-encoding by compensation and generating image compression information, and information for reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory. A reduction step, an information extension step of extending the reference image whose information amount has been reduced in the information reduction step, the image information in the frame memory and / or the reference image extended in the information extension step It has a motion prediction / compensation process for performing motion prediction / compensation processing using.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reducing step, and the reference image having the reduced information amount may be extended by upsampling in the information extending step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

Such a program is stored in at least a long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. When the predicted image is generated by reducing the amount of information of the reference image, the computer is caused to execute a process of extending the reduced reference image by upsampling or the like.

As a result, it is possible to reduce the memory size required for the encoding process and the decoding process while minimizing the deterioration of the encoding efficiency and the image quality.

The image information decoding apparatus according to the present invention is
An image information decoding device for decoding image compression information generated in an image information encoding device by inverse orthogonal transform and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction. Prior to storing the reference image in the frame memory, information reducing means for reducing the information amount of at least a part of the reference image, and information for extending the reference image having the information amount reduced by the information reducing means It is characterized in that it is provided with a stretching means and a motion prediction / compensation means for carrying out motion prediction / compensation processing using the reference image in the frame memory and / or the reference image expanded by the information expanding means. Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and the information reducing means is at least part or all of the information of the reference image stored in the long-term frame memory. Reduce the amount.

The information reducing means may reduce the information amount of the reference image by downsampling, and the information extending means may extend the reference image with the reduced information amount by upsampling. The information reducing unit may use a lossless encoding method and / or an orthogonal transform method as the information reducing method, and the information extending section may use a lossless decoding method and / or an inverse orthogonal transform method as the information extending method. .

Such an image information decoding apparatus is capable of performing at least long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using past plural frames and / or future plural frames for prediction. By reducing the amount of information in the reference image stored in the
Extend the reduced reference image.

As a result, the memory size required for the encoding process and the decoding process can be reduced while minimizing the deterioration of the encoding efficiency and the image quality.

Also, the image information decoding method according to the present invention is
An image information decoding method for decoding image compression information generated in an image information encoding device by inverse orthogonal transform and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction. Prior to storing the reference image in the frame memory, an information reducing step of reducing the information amount of at least a part of the reference image, and information for extending the reference image of which the information amount is reduced by the information reducing means. It is characterized by including a stretching step and a motion prediction / compensation step of performing a motion prediction / compensation process using the reference image in the frame memory and / or the reference image expanded by the information expanding means.

Here, the frame memory is composed of a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part of the reference image stored in the long-term frame memory or The total amount of information is reduced.

Further, the information amount of the reference image may be reduced by downsampling in the information reduction step, and the reference image having the reduced information amount may be extended by upsampling in the information extension step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

In such an image information decoding method, at the time of performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction, at least a long term frame memory is obtained by downsampling or the like. The amount of information in the reference image stored in
When the predicted image is generated, the reduced reference image is expanded by upsampling or the like.

As a result, it is possible to reduce the memory size required for the encoding process and the decoding process while minimizing the deterioration of the encoding efficiency and the image quality.

Further, the program according to the present invention is a motion prediction capable of using the image compression information generated in the image information encoding device for inverse orthogonal transformation and prediction of past plural frames and / or future plural frames. A program that causes a computer to execute a process of decoding by compensation, and an information reduction step of reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory, and the information reduction An information extending step of extending the reference image whose amount of information has been reduced by means,
And a motion prediction / compensation step of performing motion prediction / compensation processing using the reference image in the frame memory and / or the reference image expanded by the information expanding means.

Here, the frame memory is
It consists of short-term frame memory and long-term frame memory.
At least part or all of the information amount of the reference image stored in the long-term frame memory is reduced.

The information amount of the reference image may be reduced by downsampling in the information reduction step, and the reference image having the reduced information amount may be extended by upsampling in the information extension step, A reversible coding method and / or an orthogonal transformation method may be used as the information reduction method in the information reduction step, and a lossless decoding method and / or an inverse orthogonal transformation method may be used as the information extension method in the information extension step. .

Such a program is stored in at least a long-term frame memory by downsampling or the like when performing motion prediction / compensation processing capable of using a plurality of past frames and / or a plurality of future frames for prediction. When the predicted image is generated by reducing the amount of information of the reference image, the computer is caused to execute processing for extending the reduced reference image by upsampling or the like.

As a result, it is possible to reduce the memory size required for the encoding process and the decoding process while minimizing the deterioration of the encoding efficiency and the image quality.

[Brief description of drawings]

FIG. 1 is a diagram illustrating a schematic configuration of an image information encoding device according to a first embodiment.

FIG. 2 is a diagram illustrating a schematic configuration of an image information decoding device according to the first embodiment.

FIG. 3 is a diagram showing processing in a vertical direction in an information reduction unit when input image information is an interlaced scan image.

FIG. 4 is an RMPNI embedded in image compression information output from the image information encoding device according to the first embodiment.
It is a figure explaining the information contained in a field.

FIG. 5 is a diagram illustrating information included in an MMCO field embedded in image compression information output from the image information encoding device according to the first embodiment.

FIG. 6 is a diagram illustrating a schematic configuration of an image information encoding device according to a second embodiment.

FIG. 7 is a diagram illustrating a schematic configuration of an image information decoding device according to a second embodiment.

FIG. 8 is an RMPNI embedded in image compression information output from the image information encoding device according to the second embodiment.
It is a figure explaining the information contained in a field.

FIG. 9 is a diagram illustrating information included in an MMCO field embedded in image compression information output from the image information encoding device according to the second embodiment.

FIG. 10 is a diagram illustrating a schematic configuration of a conventional image information encoding device that realizes image compression by orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform and motion prediction compensation.

FIG. 11 is a diagram illustrating a schematic configuration of a conventional image information decoding device corresponding to the image information encoding device.

FIG. 12: H. It is a figure explaining the variable block size of the motion prediction compensation block defined by 26L.

FIG. 13: H. It is a figure explaining the motion prediction compensation process of 1/4 pixel precision defined by 26L.

FIG. 14: H. It is a figure in 26L explaining the method of bidirectional prediction using a B picture.

FIG. 15: It is a figure explaining PTYPE in 26L.

FIG. 16: It is a figure explaining the concept of the multiple frame prediction in 26L.

FIG. 17: R specified in H.26L Annex U
It is a figure explaining the correspondence of CodeNumber and Re-Mapping in the MPNI field.

FIG. 18: M specified in H.26L Annex U
It is a figure explaining the correspondence of Code Number and MMCO operation in the MCO field.

[Explanation of symbols]

10 image information encoding device, 11 A / D conversion unit, 12
Screen rearrangement buffer, 13 adder, 14 orthogonal transformation unit, 15 quantization unit, 16 lossless encoding unit, 17 accumulation buffer, 18 dequantization unit, 19 inverse orthogonal transformation unit,
20 short-term frame memory, 21 information reduction unit, 22 long-term frame memory, 23 information extension unit, 24 motion prediction / compensation unit, 25 rate control unit,
30 image information decoding device, 31 accumulation buffer, 32
Lossless decoding unit, 33 inverse quantization unit, 34 inverse orthogonal transform unit,
35 adder, 36 screen rearrangement buffer, 37 D
/ A conversion unit, 38 motion prediction / compensation unit, 39 short-term frame memory, 40 information reduction unit, 41 long-term frame memory, 42 information extension unit, 50 image information coding device, 51, 71 temporary memory, 52, 72
Information reduction unit, 70 Image information decoding device

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Yoichi Yagasaki             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F term (reference) 5C059 KK08 LB05 LB15 MA00 MA04                       MA05 MA14 MA23 MC11 MC38                       ME01 ME11 NN01 NN14 NN29                       PP05 PP06 PP07 SS06 UA02                       UA05 UA11 UA33                 5J064 AA01 AA04 BA04 BA09 BA16                       BB04 BC01 BC06 BC07 BC08                       BC11 BC16 BC29 BD02 BD03                       BD04

Claims (54)

[Claims] 1. An image information code for compression-encoding input image information by orthogonal transformation and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction to generate image compression information. In the digitizing device, before storing the reference image in the frame memory, an information reducing unit that reduces the information amount of at least a part of the reference image, and the reference image whose information amount is reduced by the information reducing unit are provided. An information expanding means for expanding and a motion predicting / compensating means for performing motion predicting / compensating processing using the reference image in the frame memory and / or the reference image expanded by the information expanding means. Image information encoding device. 2. The information reducing means reduces the information amount of the reference image by downsampling, and the information extending means extends the reference image having the reduced information amount by upsampling. The image information encoding device according to claim 1. 3. The image information coding apparatus according to claim 2, wherein the information reducing means performs downsampling with a reduction ratio of 2: 1. 4. The information reducing means uses {-29,0,88,138,88,0, -29} // as filter coefficients for downsampling with a reduction ratio of 2: 1.
256 is used, The image information coding apparatus of Claim 3 characterized by the above-mentioned. 5. The information reducing means downsamples the image information only in the horizontal direction at a reduction ratio of 2: 1.
The image information coding apparatus according to claim 4, wherein the amount of information is reduced to 1/2. 6. When the input image information is in a progressive scanning format, the information reduction means performs downsampling with a reduction ratio of 2: 1 in both the horizontal and vertical directions to reduce the information amount to ¼. The image information encoding device according to claim 4, wherein the image information encoding device reduces the number of images. 7. If the input image information is in an interlaced scanning format, the information reduction means performs downsampling with a reduction ratio of 2: 1 in both the horizontal and vertical directions, and in the vertical direction, the reduction ratio in field units. 2:
The amount of information is reduced to 1/4 by downsampling 1
5. The image information encoding device according to claim 4, wherein 8. The image information coding apparatus according to claim 7, wherein the information extending means performs vertical upsampling in units of fields. 9. When the input image information is in an interlaced scanning format, the information reducing means has a relation of {−29, 0, 88, 138, 88, 0, about the horizontal direction and the vertical direction of the first field. -29} //
Downsampling with a reduction ratio of 2: 1 is performed using a filter coefficient of 256, and in the horizontal and vertical directions of the second field, a reduction factor of {1, 7, 7, 1} // 16 is used. The image information coding apparatus according to claim 3, wherein the downsampling is performed at 2: 1. 10. The motion prediction / compensation process and the upsampling are performed at the same time when the reference image is downsampled.
The described image information encoding device. 11. When the resolution of the motion vector information of the image compression information is ¼ pixel precision, as filter coefficients for simultaneously performing motion prediction / compensation processing and upsampling, 1: 1 1/8: {- 3,12, -37,485, 71, -21, 6, -1} / 512 2/8: {-3,12, -37,229, 71, -21, 6, -1} / 256 3/8: {- 6,24, -76,387,229, -60,18, -4} / 512 4/8: {-3,12, -39,158,158, -39,12, -3} / 256 5/8: {-4,18, -60,229,387, -76,24, -6} / 512 6/8: {-1, 6, -21, 71,229, -37,12, -3} / 256 7/8: {-1, 6, -21 , 71,485, -37,12, -3} / 512 is used, The image information encoding device according to claim 10. 12. When the resolution of the motion vector information of the image compression information is ⅛ pixel accuracy, the motion prediction / compensation process and the upsampling are performed simultaneously by using a predetermined filter coefficient of 1:16 interpolation. The image information coding apparatus according to claim 10, wherein the image information coding apparatus is performed. 13. The frame memory comprises a short-term frame memory and a long-term frame memory, and the information reduction means at least a part or all of the reference image stored in the long-term frame memory. 3. The image information coding apparatus according to claim 2, wherein the amount of information is reduced. 14. When storing the down-sampled reference image in the long-term frame memory, RMPNI (Re-Mappi) included in the image compression information is stored.
ng of Picture Number Indicator) field, LPIRD (Long-Term Picture Index for Re-Map
Ping-Decimated) field is specified, and when storing in the short term frame memory, ADND (Ab
14. The image information coding apparatus according to claim 13, wherein the presence of a solute Difference of Picture-Decimated field is designated. 15. When storing the down-sampled reference image in the long-term frame memory, an MMCO (Memory Man) included in the image compression information is stored.
The agement Control Operation field has a field that means that one of the reference images down-sampled in the long term frame memory is discarded, and when stored in the short term frame memory, the MMCO 14. The image information coding apparatus according to claim 13, wherein the field has a field which means that one of the down-sampled reference images in the short-term frame memory is discarded. 16. An MMCO (Memory Man) included in the image compression information when the down-sampled reference image is stored in the long-term frame memory.
The agement Control Operation field has a first field that means downsampling one of the reference images in the long term frame memory, and the long term frame memory specified by the first field. The second reference image included in the short-term frame memory is downsampled and stored in the short-term frame memory.
14. The image information encoding apparatus according to claim 13, wherein the reference image included in the short-term frame memory designated by the second field is downsampled. 17. The MMCO (Memory Man) included in the image compression information when the down-sampled reference image is stored in the long-term frame memory.
In the agement Control Operation field, LPIND (Long-Term Picute Index-D) is added to the above reference image.
14. The image information coding apparatus according to claim 13, further comprising a field that means that the image information is assigned. 18. When storing the down-sampled reference image in the long-term frame memory, an MMCO (Memory Man) included in the image compression information is stored.
In the agement Control Operation field, the maximum value MLIP1D (Ma) of the index of the reference image downsampled in the long-term frame memory.
ximum Long-Term Picture Index Plus 1-Decimated)
14. The image information coding apparatus according to claim 13, further comprising a field that means to define the following. 19. When storing the down-sampled reference image in the short-term frame memory, an MMCO (Memory M) included in the image compression information is stored.
followed by the anagement Control Operation field,
DPND (Difference of Picture Numbers-Decimate
14. The image information coding apparatus according to claim 13, wherein d) a field is present, and the MMCO operation is performed on the down-sampled short-term frame memory. 20. The information reducing means uses a lossless encoding method and / or an orthogonal transform method as an information reducing method, and the information extending means uses a lossless decoding method and / or an inverse orthogonal transform method as an information extending method. The image information coding apparatus according to claim 1, wherein: 21. An image information code for compression-encoding input image information by orthogonal transformation and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction to generate image compression information. In the encryption method, an information reducing step of reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory, and the reference image having the information amount reduced in the information reducing step. An information extension step of extending the image, and a motion prediction / compensation step of performing a motion estimation / compensation process using the reference image in the frame memory and / or the reference image extended in the information extension step. An image information encoding method characterized by: 22. In the information reducing step, the information amount of the reference image is reduced by downsampling, and in the information extending step, the reference image of which the information amount is reduced is extended by upsampling. The image information encoding method according to claim 21. 23. In the information reduction step, the reduction ratio is 2: 1.
23. The image information encoding method according to claim 22, wherein the down sampling is performed. 24. The image information encoding method according to claim 22, wherein when the reference image is down-sampled, the motion prediction / compensation process and the up-sampling process are performed at the same time. 25. The frame memory comprises a short-term frame memory and a long-term frame memory, and in the information reducing step, at least a part or all of the reference image stored in the long-term frame memory. 23. The image information encoding method according to claim 22, characterized in that the amount of information is reduced. 26. In the information reducing step, a lossless encoding method and / or an orthogonal transform method is used as an information reducing method, and in the information extending step, a lossless decoding method and / or an inverse orthogonal transform is used as an information extending method. 22. The image information coding method according to claim 21, wherein a method is used. 27. A computer is used to perform compression encoding of input image information by orthogonal transformation and motion prediction / compensation capable of predicting past plural frames and / or future plural frames to generate image compressed information. In the program to be executed by the method, an information reduction step of reducing the information amount of at least a part of the reference image before the reference image is stored in the frame memory, and the information amount reduced in the information reduction step. An information expanding step of expanding a reference image, and a motion predicting / compensating step of performing a motion predicting / compensating process using the image information in the frame memory and / or the reference image expanded in the information expanding step. A program characterized by having. 28. In the information reducing step, the information amount of the reference image is reduced by downsampling, and in the information extending step, the reference image of which the information amount is reduced is extended by upsampling. 28. The program according to claim 27. 29. In the information reduction step, the reduction ratio is 2: 1.
29. The program according to claim 28, wherein the down sampling is performed. 30. When the reference image is down-sampled, motion prediction / compensation processing and up-sampling are performed at the same time.
8. The program described in 8. 31. The frame memory comprises a short-term frame memory and a long-term frame memory, and in the information reduction step, at least a part or all of the reference image stored in the long-term frame memory. 29. The program according to claim 28, characterized in that the amount of information is reduced. 32. In the information reducing step, a lossless encoding method and / or an orthogonal transform method is used as an information reducing method, and in the information extending step, a lossless decoding method and / or an inverse orthogonal transform is used as an information extending method. 28. The program according to claim 27, wherein a method is used. 33. An image in which image compression information generated in an image information encoding device is decoded by inverse orthogonal transformation and motion prediction / compensation capable of using past plural frames and / or future plural frames for prediction. In the information decoding device, before storing the reference image in the frame memory, an information reducing unit that reduces the information amount of at least a part of the reference image, and the reference image whose information amount is reduced by the information reducing unit. And an information predicting unit for performing a motion predicting / compensating process using the reference image in the frame memory and / or the reference image expanded by the information expanding unit. A characteristic image information decoding device. 34. The information reducing means reduces the information amount of the reference image by downsampling, and the information extending means extends the reference image of which the information amount is reduced by upsampling. The image information decoding device according to claim 33. 35. The information reducing means performs downsampling with a reduction ratio of 2: 1.
The described image information decoding device. 36. The information reducing means uses {-29,0,88,138,88,0, -29} // as filter coefficients for downsampling with a reduction ratio of 2: 1.
36. The image information decoding apparatus according to claim 35, wherein 256 is used. 37. When the image compression information is in an interlaced scanning format, the information reducing means is capable of: {-29,0,88,138,88,0, -29} //
Downsampling with a reduction ratio of 2: 1 is performed using a filter coefficient of 256, and in the horizontal and vertical directions of the second field, a reduction factor of {1, 7, 7, 1} // 16 is used. 36. The image information decoding apparatus according to claim 35, wherein downsampling of 2: 1 is performed. 38. When the reference image is down-sampled, the motion prediction / compensation process and the up-sampling are performed at the same time.
4. The image information decoding device described in 4. 39. When the resolution of the motion vector information of the image compression information is ¼ pixel precision, as filter coefficients for simultaneously performing motion prediction / compensation processing and upsampling, 1: 1 1/8: {- 3,12, -37,485, 71, -21, 6, -1} / 512 2/8: {-3,12, -37,229, 71, -21, 6, -1} / 256 3/8: {- 6,24, -76,387,229, -60,18, -4} / 512 4/8: {-3,12, -39,158,158, -39,12, -3} / 256 5/8: {-4,18, -60,229,387, -76,24, -6} / 512 6/8: {-1, 6, -21, 71,229, -37,12, -3} / 256 7/8: {-1, 6, -21 , 71,485, -37,12, -3} / 512 is used. 40. When the resolution of the motion vector information of the image compression information is ⅛ pixel precision, the motion prediction / compensation process and the upsampling are performed simultaneously by using a predetermined filter coefficient of 1:16 interpolation. 39. The image information decoding apparatus according to claim 38, which is performed. 41. The frame memory comprises a short-term frame memory and a long-term frame memory, and the information reducing means at least part or all of the reference image stored in the long-term frame memory. 35. The image information decoding apparatus according to claim 34, wherein the image information decoding apparatus according to claim 34, wherein 42. The information reducing means uses a lossless encoding method and / or an orthogonal transform method as an information reducing method, and the information extending means uses a lossless decoding method and / or an inverse orthogonal transform method as an information extending method. 34. The image information decoding device according to claim 33, wherein: 43. An image in which image compression information generated in an image information encoding device is decoded by inverse orthogonal transform and motion prediction / compensation capable of using a plurality of past frames and / or a plurality of future frames for prediction. In the information decoding method, before storing the reference image in the frame memory, an information reducing step of reducing the information amount of at least a part of the reference image, and the reference image having the information amount reduced by the information reducing means. An information extending step of extending the image, and a motion predicting / compensating step of performing a motion predicting / compensating process using the reference image in the frame memory and / or the reference image expanded by the information expanding means. A characteristic image information decoding method. 44. In the information reducing step, the information amount of the reference image is reduced by downsampling, and in the information extending step, the reference image of which the information amount is reduced is extended by upsampling. The image information decoding method according to claim 43. 45. In the information reduction step, the reduction ratio is 2: 1.
The image information decoding method according to claim 44, wherein the down sampling is performed. 46. The motion prediction / compensation process and the upsampling are simultaneously performed when the reference image is downsampled.
4. The image information decoding method described in 4. 47. The frame memory comprises a short-term frame memory and a long-term frame memory, and in the information reducing step, at least a part or all of the reference image stored in the long-term frame memory. 45. The image information decoding method according to claim 44, characterized in that the amount of information is reduced. 48. In the information reducing step, a lossless encoding method and / or an orthogonal transform method is used as an information reducing method, and in the information extending step, a lossless decoding method and / or an inverse orthogonal transform is used as an information extending method. The image information decoding method according to claim 43, wherein a method is used. 49. A process of decoding image compression information generated by an image information encoding device by inverse orthogonal transform and motion prediction / compensation capable of using past plural frames and / or future plural frames for prediction. In a program that causes a computer to execute, the information reduction step of reducing the information amount of at least a part of the reference image before storing the reference image in the frame memory, and the information amount is reduced by the information reduction means. An information expanding step of expanding the reference image, and a motion predicting / compensating step of performing a motion predicting / compensating process using the reference image in the frame memory and / or the reference image expanded by the information expanding means. A program characterized by having. 50. In the information reducing step, the information amount of the reference image is reduced by downsampling, and in the information extending step, the reference image of which the information amount is reduced is extended by upsampling. The program according to claim 49. 51. In the information reduction step, the reduction ratio is 2: 1.
51. The program according to claim 50, wherein the down sampling is performed. 52. When the reference image is down-sampled, the motion prediction / compensation process and the up-sampling are performed at the same time.
0 described program. 53. The frame memory comprises a short-term frame memory and a long-term frame memory, and in the information reducing step, at least a part or all of the reference image stored in the long-term frame memory. 51. The program according to claim 50, wherein the amount of information of the program is reduced. 54. In the information reducing step, a lossless encoding method and / or an orthogonal transform method is used as an information reducing method, and in the information extending step, a lossless decoding method and / or an inverse orthogonal transform is used as an information extending method. 50. The program according to claim 49, wherein a method is used.