JP2009065714A - Moving image encoder and moving image encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image encoder which performs the sub-band coding of high compression coding efficiency while leaving visually needed image components in the images of low-pass components and turning frame intervals to unequal intervals. <P>SOLUTION: The forward motion compensated images of moving images are generated in a forward motion estimation part 13 and a forward motion compensation part 15, backward motion compensated images are generated in a backward motion estimation part 14 and a backward motion compensation part 16, and the motion compensated image of a higher similarity degree to a target image is decided in a similarity degree decision part 17. The sub-band coding of obtaining a low-pass component image by adding the motion compensated image of the higher similarity degree and the target image and obtaining a high-pass component image by subtracting the low-pass component image from a reference image is obtained in an inter-frame sub-band coding part 21. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention performs motion compensation temporal direction filter processing for performing motion compensation between frames existing in the time direction of a moving image and performing subband division between frames when encoding the moving image. The present invention relates to a moving image encoding apparatus and a moving image encoding method.

Recently, video signals compressed and encoded by BS digital broadcasting, terrestrial digital broadcasting, and the like have been broadcast. In the case of Japanese terrestrial digital broadcasting, broadcasting for mobile reception and fixed reception performed at home and the like is transmitted by segmented OFDM (Orthogonal Frequency Division Multiplexing). High-stability images can be received by HDTV (High Definition TV) image quality in fixed reception and strong error correction capability in mobile reception.
The moving picture coding used in the current broadcasting is compression coded by MPEG (moving picture experts group) -2. Recently, MPEG-4 AVC (Advanced Video Coding), The use of a compression encoding method called H.264 is also being studied. In MPEG-4 AVC, a plurality of reference frames are provided, so that the selection of reference frames has a degree of freedom. As an encoding method for a fixed frame rate signal, the encoding efficiency is improved.

In Patent Documents 1 and 2, as a method of encoding at a variable frame rate, as represented by three-dimensional DCT (discrete cosine transform), three-dimensional wavelet, etc., while performing motion compensation prediction, orthogonality in the time direction An encoding method for performing conversion has been disclosed.
Further, Patent Document 3, Non-Patent Document 1, and Non-Patent Document 2 describe motion compensated temporal direction filtering (hereinafter referred to as MCTF; Motion Compensated Temporal Filtering) that performs inter-frame coding with a configuration different from the conventional configuration. This method is disclosed. The MCTF performs processing in units of a GOP (Group of Picture) composed of a plurality of frames, and performs motion compensation between frames and subband division between frames for the GOP. I am trying to do the correct encoding.
JP-A-6-141301 JP-A-6-217291 JP-T-2003-504987 J.-R. Ohm, "Advanced packet-video coding based on layered VQ and SBC techniques" IEEE Trans. Circuits Syst. Video Technol., VOL.3, JUNE 1993 Seung-Jong Choi, W. Woods, "Motion-Compensated 3-D Subband Coding of Video" IEEE TRANSACTIONS ON IMAGE PROCESSINGVOL.8, NO.2, FEBRUARY 1999

  However, in the motion compensation time direction filter processing according to the conventional method, motion estimation (hereinafter, 2N + 1 frame (hereinafter, 2N + 1 frame) and 2N frame (N is a natural number, hereinafter, 2N frame) and 2N + 1 frame (hereinafter, 2N + 1 frame) are used as units. ME) and motion compensation (hereinafter referred to as MC). An MC image is obtained by performing ME or MC so as to obtain a prediction frame of 2N + 1 frames using pixels in the 2N frame. Subband division between frames is performed between the obtained MC image and 2N + 1 frames. Thereafter, the MCTF process is repeated until a predetermined inter-frame subband division number is obtained for the L (low frequency) component frame obtained by performing the interpolation process on the inverse MC and the non-combined area.

  Considering the scalability in the time direction (hierarchization), by using the L component frame after MCTF as a playback moving image, the frame rate is reduced by 1/2, 1 each time the MCTF subband division hierarchy increases. As a result, it is possible to obtain a result of decimating at an equal interval of 2: 1 recursively between frames, such as / 4 and 1/8. For example, if the input video is 30 fps (frames per second) at equal intervals, the frame rate is reduced by half each time the MCTF inter-frame subband coding advances one layer, that is, 15 fps. An image having an L component frame rate of 5 fps or the like is obtained. Scalability in the time direction can be obtained by using the L component frame after MCTF as a playback moving image.

However, in the case of realizing scalability in the time direction using MCTF in the conventional method, even in the initial stage of lowering the frame rate, it is based on decimation in which the frame interval is regularly equalized. There arises a problem that a frame image including important moving image information is lost.
In addition, in the MCTF in the conventional method, the image frame that is originally important may not be used in the next MCTF due to decimation as described above, and the correlation between frames in the GOP is fully utilized in terms of encoding efficiency. It was not complete and efficient encoding was not performed.

  Therefore, the present invention has been made to solve the above-described problems, and when performing motion compensation time direction filter processing, attention is paid to past frame images and future frame images, so that a low frequency can be obtained. Subband code with high compression coding efficiency that leaves the image components that are visually required to the low-frequency component image among the low-frequency component image and the high-frequency component image converted to the frame rate. It is an object of the present invention to provide a moving image encoding apparatus and a moving image encoding method that can obtain an output.

In order to achieve the above object, the moving image encoding apparatus of the present invention uses a frame having an input moving image as a forward reference image and a frame different from the forward reference image as a forward target image. A forward motion estimation unit that estimates the motion of the forward target image from the forward reference image and obtains forward motion vector information; and the forward reference image is a backward target image. When the forward target image is a backward reference image, backward motion estimation is performed by estimating the backward target image motion from the backward reference image and obtaining backward motion vector information. And a forward direction to obtain a forward motion compensated image for compensating for a motion between the forward reference image and the forward target image based on the forward motion vector information. And a backward motion compensation unit for obtaining a backward motion compensation image for compensating for motion between the backward reference image and the backward target image based on the backward motion vector information. And the degree of similarity between the forward direction motion compensated image and the forward direction target image and the degree of similarity between the backward direction motion compensated image and the reverse direction target image. And a similarity determination unit that generates order information for indicating whether or not is selected.
Here, in the moving image encoding apparatus, the reference image, the target image, the motion compensated image, and the motion vector information in the forward direction or the reverse direction having a high similarity are further calculated based on the order information. A selection unit for selecting, the selected motion compensated image and the target image to add a low frequency component image having a lower rate than the input moving image, and the low frequency component image from the selected reference image An inter-frame subband encoding unit that obtains a high-frequency component image at a higher rate than the input moving image by subtraction may be included. Furthermore, an entropy encoding unit that entropy-encodes the low-frequency component image, the high-frequency component image, the order information, and the selected motion vector information may be included.
In order to achieve the above object, the moving image encoding method of the present invention is configured such that a frame having an input moving image is a forward reference image, and one frame different from the forward reference image is a forward target image. In this case, estimating the motion of the forward target image from the forward reference image to obtain forward motion vector information, and making the forward reference image a backward target image, When the forward target image is a backward reference image, estimating the motion of the backward target image from the backward reference image and obtaining backward motion vector information; and the forward direction Based on the motion vector information of the forward direction, a forward direction motion compensated image for compensating for motion between the forward direction reference image and the forward direction target image is obtained, and the reverse direction Obtaining a backward motion compensated image for compensating for the motion of the backward reference image and the backward target image, based on the motion vector information of the forward direction, the forward motion compensated image, and the forward direction The degree of similarity between the target image and the similarity between the motion compensation image in the reverse direction and the target image in the reverse direction is determined, and order information is generated to indicate which direction is selected. A video encoding method comprising: steps.
Here, in the moving image encoding method, the forward reference image or the reverse reference image having the high similarity, the target image, the motion compensation image, and the motion vector information are further obtained based on the order information. A selection step of selecting, adding the selected motion-compensated image and the target image, a low-frequency component image having a lower rate than the input moving image, and selecting the low-frequency component image from the selected reference image An inter-frame subband encoding step that obtains a high-frequency component image having a higher rate than the input moving image by subtraction. Furthermore, an entropy encoding step for entropy encoding the low-frequency component image, the high-frequency component image, the order information, and the selected motion vector information may be included.

According to the present invention, when a frame having an input moving image is a forward reference image, and one frame different from the forward reference image is a forward target image, the forward reference image is used to generate the forward motion image. When estimating the motion of the target image in the direction and acquiring the motion vector information in the forward direction, the forward reference image is the backward target image, and the forward target image is the backward reference image In addition, the motion of the backward target image is estimated from the backward reference image to obtain backward motion vector information, and the forward reference image and the forward motion vector information are obtained based on the forward motion vector information. A forward motion compensated image for compensating for motion between target images in the forward direction is obtained, and the reference image in the backward direction and the inverse motion image are obtained based on the backward motion vector information. Obtaining a backward motion compensated image for compensating the motion of the forward target image, the similarity between the forward motion compensated image and the forward target image, the backward motion compensated image, and the Since the degree of similarity with the target image in the reverse direction is determined and order information for indicating which direction has been selected is generated. By referring to the order information, the forward direction with high similarity or A motion compensation image in the reverse direction, a target image, and a reference image can be selected.
Further, based on the order information, the reference image having the high similarity or the reference image in the forward direction or the reverse direction, the target image, the motion compensation image, and the motion vector information are selected, and the selected motion compensation image and A low-frequency component image having a lower rate than the input moving image by adding the target image, and a high-frequency component having a higher rate than the input moving image by subtracting the low-frequency component image from the selected reference image If the component image is obtained, the low-frequency component and the high-frequency band converted to the low-frequency frame rate by focusing on the past frame image and the future frame image when performing the motion compensation time direction filter processing. Among the component images, a low-band component image is a dynamic image that provides a sub-band encoded output with high compression encoding efficiency that leaves an image component that is visually required. It can be realized an image encoding apparatus and the moving picture coding method.
Further, if the low-frequency component image, the high-frequency component image, the order information, and the selected motion vector information are multiplexed by entropy coding, the order information, By referring to the selected motion vector information, a compression-coded signal can be decoded from the low-frequency component image and the high-frequency component image. It is possible to realize a moving picture decoding apparatus and a moving picture decoding method capable of decoding a compression-coded signal in which a necessary image component is left with high image quality.

Hereinafter, a moving picture coding apparatus according to an embodiment of the moving picture coding apparatus, the moving picture coding method, the moving picture decoding apparatus, and the moving picture decoding method of the present invention will be outlined with reference to FIGS.
FIG. 1 is a block diagram illustrating a configuration example of a moving image encoding apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram showing an encoding flow of the moving image encoding apparatus according to the embodiment of the present invention.
FIG. 3 is a diagram showing a coding method of the moving picture coding apparatus according to the embodiment of the present invention.
FIG. 4 is an explanatory diagram of the encoding method of the moving image encoding apparatus according to the embodiment of the present invention.
FIG. 5 is an explanatory diagram of an encoding method of the moving image encoding apparatus according to the embodiment of the present invention.

  When the motion image encoding apparatus performs the motion compensation time direction filter processing, the low-frequency component image and the high-frequency component image converted to the low-frequency frame rate by paying attention to the past frame image and the future frame image. An object of the present invention is to realize a moving picture coding apparatus that can obtain a sub-band coded output with high compression coding efficiency, in which an image component that is visually required is left in a low-frequency component image. A parallel input unit for outputting moving images in parallel, one frame of the moving images output in parallel as a first reference image, and one frame immediately after the first reference image as a first target image A forward motion estimation unit that estimates motion of the first target image from the first reference image and acquires forward motion vector information; and the first reference image and the first target. When the first reference image is the second target image and the first target image is the second reference image, the motion of the second target image is estimated from the second reference image. A backward motion estimation unit for obtaining backward motion vector information, and forward motion compensation for compensating for motion between the first reference image and the first target image based on the forward motion vector information. A forward motion compensation unit that obtains an image, and a backward motion compensation that obtains a backward motion compensated image for compensating the motion of the second reference image and the second target image based on the backward motion vector information , The similarity between the forward motion compensation image obtained by the forward motion compensation unit and the first target image, the backward motion compensation image obtained by the backward motion compensation unit, and the previous A similarity determination unit that compares similarities with a second target image and determines which is the higher similarity, and the similarity determination unit that determines that the similarity is high A switching unit that switches to a forward motion compensation unit or the backward motion compensation unit, the forward motion compensation image that is output from the forward motion compensation unit or the backward motion compensation unit that is switched by the switching unit, and the The first target image or the backward motion compensated image and the second target image are added to obtain a compressed image having a lower rate than the moving image, while the first or second reference image and the compressed image are obtained. Subframe subframes for subtracting the first reference image and the first target image or the second reference image and the second target image from the compressed image to obtain a restoration auxiliary image. This is realized by having a code encoder.

The configuration of the moving picture coding apparatus will be described.
The moving image encoding apparatus 1 shown in FIG. 1 includes an input management unit 12, a forward motion estimation unit 13, a backward motion estimation unit 14, a forward motion compensation unit 15, a backward motion compensation unit 16, and a similarity determination unit 17. , Switch 18, switch 19, interframe subband encoding unit 21, quantization unit 25, entropy encoding unit 26, and multiplexing unit 27. A moving image is input to the input management unit 12. An encoded signal is output from the multiplexing unit 27.

The operation of the moving picture encoding apparatus 1 will be described.
First, a GOP (Group of Picture) is formed in the moving image input to the input management unit. Here, in order to simplify the description, a case where 1 GOP is divided into images every 8 frames will be described. Numbers 0 to 7 are assigned in the input order of the divided frame images. The forward motion estimation unit 13 obtains the image data with the numbers 0 and 1. There, the motion vector for predicting the target image from the reference image is detected using the 0th image as the reference image and the 1st image as the target image. The motion vector is detected in the same manner as the motion vector detection method used in MPEG or the like. Next, the forward motion compensation unit 15 acquires the 0th image, the 1st image, and the detected motion vector. There, a motion compensated image is generated based on the detected motion vector.
The reverse direction motion estimation unit 14 and the reverse direction motion compensation unit 16 perform the same operations as those performed by the forward direction motion estimation unit 13 and the forward direction motion compensation unit 15. However, the operations in the backward motion estimation unit 14 and the backward motion compensation unit 16 are compared with the operations in the forward motion estimation unit 13 and the forward motion compensation unit 15, and the 0th image is the target image. The difference is that the first image is the reference image. That is, compared to the case where the forward motion compensation unit 15 generates the forward motion compensation image,
The backward motion compensation unit 16 generates a so-called backward motion compensation image in which a past image is generated based on a future image.

In the similarity determination unit 17, the similarity between the motion compensation image generated by the forward motion compensation unit 15 and the first target image, the motion compensation image generated by the backward motion compensation unit 16, and the 0th target image Are compared, and it is determined which motion compensation image generated by which motion compensation unit is more similar. As a determination method, for example, a mean square error for each pixel between the motion compensated image and the target image is obtained and compared, and the smaller error is determined as having high similarity. Based on the determination result made by the similarity determination unit 17, the switch 19 is switched to a terminal from which a motion image having a high similarity is obtained.
In the inter-frame subband encoding unit 21, a reference image, a target image, and a motion compensated image are obtained via the switch 19. The input target image and the motion compensated image are added to generate a low-frequency component image. A low-frequency component image is subtracted from the reference image to generate a high-frequency component image.

In the quantization unit 25, each of the low-frequency component image and the high-frequency component image is quantized to obtain quantized data. In the entropy encoding unit 26, the redundancy included in the quantized data is reduced. Coded data that is visually redundant is also reduced.
Of the motion vectors obtained by the forward motion estimator 13 or the motion vectors obtained by the backward motion estimator 14, the entropy encoder 26 determines the one determined as similar by the similarity determiner 17. The motion vector is switched by the switch 18 and inputted and encoded. In the multiplexing unit 27, the encoded low-frequency component image, high-frequency component image, and motion vector are time-division multiplexed and generated as an encoded signal and output.
Next, a motion compensation image is obtained using the second image as a reference image, a third image as a target image, and a motion compensation image is obtained using the second image as a target image and the third image as a reference image. The coded signal generated by determining the similarity is generated. Similarly, an encoded signal is generated for each pair of images of Nos. 4-5 and 6-7.

Hierarchization of encoded data will be described with reference to FIG.
In (1) of the figure, the low-frequency component image generated from the 0th image and the 1st image (hereinafter sometimes referred to as L image) is L10, and the 0th image and the 1st image. A high-frequency component image generated from the image (hereinafter sometimes referred to as H image) is represented as H14. L11 and H15 are generated from the second and third images. Similarly, L12, H16, L13, and H17 are generated. L10 to L13 and H14 to H17 generated here are primary image data.
L10 and L11 can be input to the moving picture encoding apparatus 1 to generate LL20 and LH22. LL21 and LH23 can be generated from L12 and L13. The data generated here is secondary hierarchical image data.
The generated secondary hierarchical image data can be input to the moving image encoding apparatus 1 to generate tertiary hierarchical image data that are LLL30 and LLH31.

FIG. 2 (2) shows a transmission method of primary to tertiary hierarchical image data.
Encoded signals each having a packet structure are transmitted in the order of LLL30, LLH31, LH22, LH23, H14, H15, H16, and H17. Here, LLL 30 is an image signal having a frame rate of 1/8. The LLL 30 is transmitted with a particularly strong error correction code. A mobile receiving device such as a vehicle receives the LLL 30. Although it is a signal transmitted through an inferior transmission path whose propagation state changes at high speed, data with a low transmission rate and a strong error correction code can be received normally. Images are received at 3.75 (30/2 ³ ) images per second.

On the other hand, the fixed receiving device installed in the home receives all the signals of LLL 30 to H 17 transmitted. The image data H14 to H17 are assigned standard error correction codes and transmitted at a normal transmission rate. Stable reception is performed by a fixed antenna having a ground clearance.
In the receiver that has received the signals of LLL30 to H17, LL20 and LL21 are decoded using LLL30 and LLH31. L10 to L13 are decoded using the decoded LL20 and LL21 and the LH22 and LH23 obtained by reception. The images 0 to 7 are decoded using the received H14 to H17. In fixed reception, the frame rate is received as 30 images per second.

The motion estimation unit and the motion compensation unit in the forward direction and the reverse direction will be described with reference to FIG.
(1) shown in the figure is an operation time relationship in the forward direction motion estimation unit 13 and the forward direction motion compensation unit 15 that uses the image A in the past as a reference image and the image B in the future direction as a target image. Is shown. (2) shows the time relationship of the operations in the backward motion estimation unit 14 and the backward motion compensation unit 16. Respective L images and H images obtained by performing forward and reverse motion prediction and motion compensation may or may not change depending on how the small images constituting the screen move.

The reference image and the target image will be described with reference to FIG.
In the same figure, image 0 is an image on the left side of the circle shown in (1a), and image 1 is an image on the right side of the circle shown in (1b). The L image is generated by adding the image of the screen (1a) and the image of the screen (1b). If the addition is performed as it is, an L image in which two circles are arranged is obtained. In the case of a normal image, the image is blurred, which is not preferable. Therefore, the circle of (1a) is moved to the right until it overlaps with (1b). The amount of movement at that time is a motion vector, and an image generated by movement is a motion compensated image. When the motion compensated image and the image of (1b) are added, an image without deterioration is obtained as an L image. The H image is an image obtained by subtracting the L image from the reference image. This is an example in which subband coding for obtaining an L image and an H image using forward motion prediction and motion compensated images is performed.
In this operation, even when (1a) is the target image and (1b) is the reference image, L images without overlapping are obtained in the same manner.

A case where the image 0 is (2a) and the image 1 is (2b) will be described.
Here, both squares are in the same position. The black triangle behind the square is moving. In this case, the square motion vector is 0, and only the triangular motion vector changes. However, the motion vector is detected as a new black triangle appearing rather than being detected as a moving triangle. In (2a), there are only small area triangles, and it is not possible to perform motion prediction for large area triangles. Therefore, an accurate motion vector cannot be obtained and a suitable motion compensated image cannot be generated. When (2a) is used as a reference image, it is difficult to create a motion compensated image for the target image (2b).
When (2b) is a reference image and (2a) is a target image, a motion compensated image for the target image (2a) can be created using a part of the black triangle of (2b). That is, when an image having a texture different from the surrounding appears on the screen, it is easier to create a motion compensated image by selecting an image having a lot of different texture information as a reference image.

  (3a) shows the position of the reference image used to predict the target image (3b), which is used when generating a motion compensated image using the past image as a reference image for the zoomed out image. It is a thing. (4b) shows the position of the reference image when a motion compensated image is generated using the future image as a reference image and the image shown in the past (4a) as a target image. Here, the positions of the images not referenced in (3a) and (4b) are indicated by halftone dots. The information amount of the H image generated in this case tends to be smaller when the halftone portion is the reference image. A lot of information is taken into the L image.

The preferred relationship between the position of the reference image and the target image will be further described with reference to FIG.
Here, it is assumed that the image 0 is (4a) and the image 1 is (4b).
(5a) uses the image (4a) as a reference image, and the reference image portion used when generating the motion compensated image for the target image (4b) is an area indicated by a cross line. (6a) is a motion compensation image obtained as a result. The area indicated by the slanting line at the lower left is an error area that is generated as a white area where the halftone dot area should be made.
(5b) shows a referenced part of an image used when the image (4b) is a reference image and the image (4a) is a target image. An image (6b) is obtained as a motion compensated image. This image is the same as the image (4a). That is, the same motion compensated image as the target image is obtained.

  As described above, when a motion compensated image with little error is obtained with respect to the target image, the motion compensated image and the target image are added to obtain a low-frequency component image (L image) with little distortion. The data amount of the high-frequency component image (H image) obtained by subtracting the low-frequency component image from the reference image can reduce the information amount corresponding to the small amount of distortion components included in the L image. Encoding efficiency in subband encoding can be increased.

  The moving image encoding apparatus 1 includes a parallel input unit 12 that outputs moving images in parallel, one frame of the moving images output in parallel as a first reference image, and a frame immediately after the first reference image. When the frame is the first target image, the forward motion estimation unit 13 that estimates the motion of the first target image from the first reference image and acquires forward motion vector information, and the first reference image And the first target image are replaced, the first reference image is the second target image, and the first target image is the second reference image, the movement of the second target image from the second reference image Based on the forward motion vector information, and a motion between the first reference image and the first target image is obtained based on the forward motion vector information. A forward motion compensation unit 15 for obtaining a forward motion compensation image for compensation, and a backward direction for compensating for the motion of the second reference image and the second target image based on the backward motion vector information. A backward motion compensation unit 16 for obtaining a motion compensation image, a similarity between the forward motion compensation image obtained by the forward motion compensation unit and the first target image, and a backward motion compensation unit. The similarity determination unit 17 that compares the similarity between the backward motion compensated image and the second target image and determines which is the higher similarity, and the similarity determination unit The switching unit 19 that switches to the forward motion compensation unit or the backward motion compensation unit that is determined to be high, and the forward motion compensation unit or the backward motion compensation unit that is switched by the switching unit. The output of the forward direction motion compensated image and the first target image or the backward direction motion compensated image and the second target image are added to obtain a compressed image having a lower rate than the moving image, while the first Alternatively, the second reference image and the compressed image are subtracted to obtain the restoration auxiliary image for restoring the first reference image and the first target image or the second reference image and the second target image from the compressed image. Since the inter-frame subband encoding unit 21 is included, when the motion compensation time direction filter processing is performed, the frame rate is converted to a low-frequency frame rate by paying attention to the past frame image and the future frame image. Of the low-frequency component image and the high-frequency component image, the low-frequency component image has a high compression coding efficiency so that the image component that is visually required is left. It is possible to realize a moving image encoding apparatus that can obtain a broadband encoded output.

≪Detailed explanation of video coding≫
Hereinafter, a moving image encoding device, a moving image encoding method, a moving image decoding device, and a moving image decoding method will be described in detail with reference to FIGS.
FIG. 6 to FIG. 8 are conceptual diagrams showing the process of motion compensation time direction filter processing (hereinafter referred to as MCTF).
FIG. 9 is a flowchart showing basic MCTF processing.
FIG. 10 is a flowchart showing a series of MCTF encoding processes for 1 GOP worth of moving image frames.

First, a series of processing steps of MCTF encoding will be described with reference to FIG.
First, a moving image frame to be encoded is acquired (step S401). Thereafter, moving image frames for 1 GOP are accumulated (step S402). Here, 1 GOP is assumed to be 8 frames in order to simplify the story. A frame group to be subject to MCTF is set from the moving image frames for 1 GOP (step S403). Here, since the octave division is not performed on the L component frame, all frames of 1 GOP become a frame group to be subjected to MCTF.
Basic MCTF processing is performed on such a frame group (step S404). Here, a detailed description of the basic MCTF process is omitted here for the sake of later description. After that, the L component frames obtained by the basic MCTF process are aligned and set as the next octave division target frame group (step S405).

In MCTF, it is determined whether a predetermined number of octave divisions has been reached (step S406).
If the predetermined number of octave divisions has not been reached, the process proceeds to step S403, where the obtained L component frame group is repeated as the next MCTF target frame group.
If the predetermined number of octave divisions has been reached, quantization is performed on the subband coefficient information that is the L component frame and the H component frame obtained by the basic MCTF processing (step S407).
Bitstream is obtained by performing predetermined entropy coding on the quantized subband coefficient information and motion vector information obtained by MCTF ME (step S408), and performing MUX on the obtained coded output. (Step S409), a series of MCTF encoding processes for 1 GOP moving image input is completed.
Here, at the time of MUX, it is preferable to add a GOP header storing information necessary for configuring one GOP for each GOP. The GOP header may include at least information on the number of frames in one GOP and the number of octave divisions of subband coding. The GOP header may further include an order information header described later.

Next, basic MCTF processing will be described with reference to FIGS. 9 and 6 to 8.
FIG. 6 shows the process of the first-order decomposition of the octave decomposition performed recursively for the L component frame in the MCTF subband division.
In order to set a frame group that is a target of the basic MCTF (step S301), 2N frames as indicated by 602 to 605 in FIG. It is set so that 2N + 1 frames are the processing unit. Here, N is a positive integer starting from 0. Normally, the L component frame storing the L component subband coefficient information obtained by subband division is the target frame group, but here, since subband division is the first time, the frame accumulated as 1 GOP is targeted. Set as a frame group.
First, as indicated by 606 in FIG. 6 for two frames 602 in FIG. 6, ME is performed in the forward direction and the reverse direction, respectively, to obtain a motion vector (step S302). Here, the forward direction means that a motion vector is obtained by searching the 0 frame side with the 1 frame side as a reference. Similarly, the reverse direction refers to obtaining a motion vector by searching the 1 frame side with the 0 frame side as a reference.
MC is performed based on the motion vector obtained in each direction for the forward direction and the reverse direction (step S303). Here, as indicated by reference numeral 606 in FIG. 6, in the forward direction, the 0 frame side is the reference for performing MC images and the 1 frame side is for performing ME / MC. In the reverse direction, the 1 frame side is the reference for MC images, and the 0 frame side is the reference for ME / MC.

Thereafter, the similarity between the MC image created in the forward direction and the reverse direction and the target frame that is a reference when performing MC is obtained (step S304). Here, in order to determine the degree of similarity, generally, a method is employed in which MSE (Mean Square Error) is measured and the degree of similarity is determined based on a smaller value. In determining similarity, the sum of absolute values of differences between frames may be simply measured as an error between frames, and the similarity may be determined based on a smaller value. The degree of similarity may be determined by measuring the number of pixels in the non-bonded region when MC is performed, and determining the degree of similarity by making this value smaller. It should be noted that there is no particular limitation here as long as the superiority or inferiority of the error can be determined.
It is determined which direction has a higher similarity between the similarity between the MC image in the forward direction and the reference target frame and the similarity between the MC image in the reverse direction and the reference target frame. Here, it is determined whether the similarity is high in the forward direction (step S305).
If it is determined that the similarity in the forward direction is higher, the MC information and the motion vector obtained by the ME / MC in the forward direction are adopted, and the order information indicating that the adopted direction is the forward direction Is output (step S306). Here, in 606 of FIG. 6, the order information is represented by a + symbol on the 1 frame side for the forward direction and a − symbol on the 0 frame side for the reverse direction.

When it is determined that the similarity in the reverse direction is higher, the MC image and the motion vector obtained by the ME / MC in the reverse direction are adopted, and the order information indicating that the adopted direction is the reverse direction Is output (step S307). Here, as shown by reference numeral 607 in FIG. 6, the discussion proceeds assuming that the reverse direction is selected.
Thereafter, inter-frame subband encoding is performed using the selected MC image and the reference target frame (step S308). The storage destination of the subband coefficient information that is the output after encoding is controlled based on the order information. When the order information indicates the forward direction, the storage destination is controlled such that the L component subband coefficient information is stored in the 0 frame side and the H component subband coefficient information is stored in the 1 frame side. Similarly, when the order information represents the reverse direction, the storage destination is controlled so that the L component subband coefficient information is stored on the 1 frame side and the H component subband coefficient information is stored on the 0 frame side. . Here, since the order information represents the reverse direction, the 0 frame side is the target frame, the 1 frame side is the MC image, and subband coding between frames that is orthogonal transform is performed on these 2 frames, As indicated by reference numeral 608 in FIG. 6, an H component frame storing H component subband coefficient information on the 0 frame side based on the order information, an L component frame storing L component subband coefficient information on the 1 frame side, and Thus, the storage location of the encoded subband coefficient information is controlled.

Next, as indicated by reference numeral 609 in FIG. 6, inverse MC is performed based on the motion vector information (step S309). The motion vector information to be used corresponds to the order information. Here, the inverse MC is performed by using the motion vector information obtained by the ME in the reverse direction in the reverse direction. Further, the frame to be subjected to inverse MC is an L component frame, and here, it is performed on the L component frame whose storage destination is controlled to one frame side by the order information.
Thereafter, interpolation of pixels that could not be filled with inverse MC is performed (step S310). In general, the pixel region that cannot be filled with the inverse MC is a non-bonded region that is not related between the 0 frame and the 1 frame during ME. Therefore, usually, the pixels in the region at the same position as the position of the non-bonded region are acquired from the original frame, that is, one frame here, and the pixels are interpolated.
The L component frame after inverse MC obtained in this way is a candidate for the next basic MCTF process. Then, it is determined whether there is a frame to be processed in the current octave division (step S311). If there is no frame to be processed in the current octave division, the basic MCTF process is completed. In this case, since 2 to 7 frames remain to be processed in the current octave division, the process returns to step S301 to set 2 frames and 3 frames, which are frames to be subjected to the next basic MCTF process. To do.

After that, the basic MCTF processing is performed in the same way, and between 2 and 3 frames, the reverse direction is selected in the forward direction and the reverse direction, and the L component frame after reverse MC is generated on the 3 frame side. To do. Between the 4th frame and the 5th frame, the forward direction is selected in the selection of the forward direction and the reverse direction, and an L component frame after reverse MC is generated on the 4th frame side. Between the 6th frame and the 7th frame, the forward direction is selected in the selection of the forward direction and the reverse direction, and an L component frame after reverse MC is generated on the 6th frame side.
In this way, by repeating the basic MCTF process until there is no frame to be processed in the current octave division, and obtaining the L component frames after inverse MC such as 610, 615, 620, and 625 in FIG. Complete the basic MCTF process. Here, the obtained H component frame, motion vector information, order information, and the like are subjected to subsequent quantization and entropy coding. Further, the L component frame group after inverse MC indicated by 610, 615, 620, and 625 of FIG. 6 obtained by this basic MCTF process is set as a frame group to be subjected to octave division of the next MCTF.
Here, assuming that 1 GOP is 8 frames as in the present embodiment, the frame interval is 0, 2, 4, 6th frame, or 1, 3, 5, 7th frame in the conventional MCTF. Frames are output so that are equally spaced. However, in MCTF, the L component frame group after inverse MC obtained for each subband division is such that the frame intervals are unequal, such as the first, third, fourth, and sixth frames in this embodiment. Output a frame. This unequal-interval frame output is obtained by performing forward ME / MC and backward ME / MC, and obtaining the similarity between the MC image obtained in each direction and the frame subject to MC. This is an effect caused by performing the subsequent MCTF processing using the result of ME / MC in the direction with the high similarity.

FIG. 7 shows the process of the second-order decomposition of the octave decomposition performed recursively for the L component frame in the MCTF subband division.
The basic process flow is the same as the basic MCTF process shown in FIG.
First, as indicated by reference numeral 701 in FIG. 7, the L component frame group after inverse MC indicated by reference numerals 610, 615, 620, and 625 in FIG. 6 is set as a frame group to be subject to octave division in the second-order decomposition. . Thereafter, 1 frame, 3 frames, 4 frames, and 6 frames are set as a frame group to be subjected to basic MCTF processing.
After that, the basic MCTF process is performed in the same way, and between 1 and 3 frames, the reverse direction is selected in the forward direction and the reverse direction, and the L component frame after reverse MC is generated on the 3 frame side. To do. Between the 4th frame and the 6th frame, the forward direction is selected in the selection of the forward direction and the reverse direction, and an L component frame after reverse MC is generated on the 4th frame side.
In this way, the basic MCTF processing is repeated until there is no frame to be processed in the current octave division, and the basic MCTF processing is performed by obtaining the L component frames after inverse MC such as 708 and 713 in FIG. Complete.

FIG. 8 shows the process of the third-order decomposition of the octave decomposition performed recursively for the L component frame in the MCTF subband division.
The basic process flow is the same as the basic MCTF process shown in FIGS.
First, as indicated by reference numeral 801 in FIG. 8, the L component frame group after inverse MC indicated by reference numerals 708 and 713 in FIG. 7 is set as a frame group to be subject to octave division in the third-order decomposition. Thereafter, 3 frames and 4 frames are set as a frame group to be subjected to basic MCTF processing.
Thereafter, the basic MCTF process is performed in the same manner, and the forward direction is selected in the forward direction and the backward direction between the 3rd frame and the 4th frame, and the L component frame after the reverse MC is generated on the 3rd frame side.
In this way, the basic MCTF process is repeated until there is no frame to be processed in the current octave division, and the basic MCTF process is completed by obtaining an L component frame after reverse MC as shown by 807 in FIG. .

FIG. 15 is a flowchart showing the basic inverse MCTF process.
FIG. 16 is a flowchart showing a series of MCTF decoding processing until a moving image frame for 1 GOP is obtained from a bit stream.
First, a series of processing steps of MCTF decoding will be described with reference to FIG.
In FIG. 16, a bit stream for decoding is acquired (step S601).
Thereafter, DEMUX is performed on the obtained bit stream (step S602), thereby obtaining a subband coefficient information bit string, a motion vector information bit string, and an order information bit string.
By performing entropy decoding on the subband coefficient information bit sequence, motion vector information bit sequence, and order information bit sequence obtained by DEMUX (step S603), subband coefficient information on the L and H components, motion vector information, and order information Get.
Inverse quantization is performed on the subband coefficient information of the L component and the H component (step S604).

Information for constituting 1 GOP is accumulated (step S605). Here, the L component frame, the H component frame, the motion vector information, and the order information necessary for constituting one GOP are acquired from inverse quantization and entropy decoding.
An information group to be subjected to basic inverse MCTF processing is set (step S606).
Thereafter, basic MCTF processing is performed (step S607).
After that, it is determined whether the number of divisions is a predetermined octave (step S608).
If the predetermined number of octave divisions has not been reached, the basic inverse MCTF processing is performed after the L component frame group synthesized by the basic inverse MCTF processing is set as the target of the next basic inverse MCTF processing. Repeat until number.
When the predetermined number of octave divisions is reached, the obtained L component frame group is output as a decoded moving image. Here, the L component frame in which the octave division number is 0 is a complete decoded moving image after decoding all the L component frames and the H component frames.

Next, basic inverse MCTF processing will be described with reference to FIGS. 11 and 15. Here, the description proceeds on the assumption that the order information can be acquired for each H component frame.
First, order information regarding a frame to be processed is acquired (step S501). Here, the order information is assumed to be forward. In FIG. 11, the forward direction is represented by “+” and the backward direction is represented by “−” in the order information.
In addition, in order to perform basic inverse MCTF processing, an information group to be subjected to inverse MCTF is set (step S502). Here, it is assumed that 1 GOP is 8 frames, and the number of octave divisions of subband encoding performed on the encoding side is decomposed to the third order. Each component frame is in the order of L3-3, H3-4, H2-1, H2-6, H1-0, H1-2, H1-5, H1-7 as shown in the upper right frame group in FIG. Are input and set so as to be subject to basic inverse MCTF processing in descending order of the number of octave divisions. Here, L3-3 indicates that “L3” is an L component frame whose octave division number is the third-order decomposition, and “−3” indicates that the frame position within 1 GOP stored at the time of encoding is 3 frames. Represents an eye. H3-4 indicates that “H3” is an H component frame whose octave division number is the third-order decomposition, and “−4” indicates that the frame position in 1 GOP stored at the time of encoding is the fourth frame. It represents that. The same applies to H2-1, H2-6, H1-0, H1-2, H1-5, and H1-7. However, on the decoding side here, it should be noted that since the order information is acquired for each H component frame, the original accurate frame position cannot be specified until all the decoding is completed.

Each area indicated by 1101 to 1121 in FIG. 11 visually indicates a time interval in a GOP occupied by one frame when the decoded frame is displayed as a playback video after being scalable.
When the L component frame is used for moving image reproduction as scalability in the time direction, moving image reproduction is performed at the timing of L3-3 stored in 101 of FIG. Here, since the storage position cannot be specified yet, reproduction is performed assuming that it is arranged at the center of the time interval indicated by 1101 in FIG.
In the set of frames for performing the basic inverse MCTF processing, first, L3-3 frames and H3-4 frames are acquired in order to perform decoding on a frame whose octave division number is the third-order decomposition. Next, paying attention to 1102 and 1103 in FIG. 11, when the order information is in the forward direction, the L component frame is stored closer to the frame position 0 frame and the H component frame is stored closer to the frame position 7 frame. . When the order information is in the reverse direction, the L component frame and the H component frame are stored in positions opposite to each other. Here, since the order information is the forward direction, the L component frame is stored in 1102 of FIG. 11, and the H component frame is stored in 1103 of FIG.

Thereafter, MC is performed on the L component frame based on the order information and the motion vector information (step S503). Here, the L3-3 frame 1101 in FIG. 11 is rearranged in 1102 in FIG. 11, and then MC is performed according to the motion vector information to obtain an MC image.
Using the MC image created based on the order information and the motion vector information, inter-frame subband decoding is performed with the H component frame stored in 1103 of FIG. 11 (step S504).
Thereafter, the inverse MC is performed on the L component frame obtained by decoding on the MC image side, that is, 1104 in FIG. 11 based on the order information and the motion vector information (step S505), and cannot be filled with the inverse MC. The interpolated pixels are interpolated (step S506).
Through the above processing steps, the second-order decomposition L component frame stored in 1104 and 1105 of FIG. 11 can be obtained from the third-order decomposition L component frame and the H component frame.
Thereafter, it is determined whether or not there is a frame to be processed in the current octave division (step S507). This time, decoding is from octave division third-order decomposition to second-order decomposition. Since there is no frame to be processed, the basic inverse MCTF processing is completed.
At this stage, when the L component frame is reproduced as a moving image as scalability in the time direction, the moving image is reproduced at the timing of the L component frame stored in 1104 and 1105 of FIG. Here, since the storage position cannot be specified yet, reproduction is performed assuming that it is arranged at the center of the time interval indicated by 1104 and 1105 in FIG.

Next, in order to perform decoding on a frame whose octave division is second-order decomposition, attention is paid to the L component frames 1104 and 1105 in FIG. 11, and the H2-1 frame is acquired and accompanying order information is acquired. To do. Here, the order information corresponding to the H2-1 frame is in the reverse direction. Further, the time interval indicated by 1104 in FIG. 11 is equally divided, and 1106 and 1107 in FIG. 11 are the time intervals to be stored.
After that, the L component frame and H2-1 frame of 1104 in FIG. 11 are stored according to the order information. The L component frame 1104 in FIG. 11 is stored in the 1107 side in FIG. 11 out of 1106 and 1107 in FIG. 11, which is a section formed by equally dividing 1104 in FIG. . The H2-1 frame corresponding to the H component frame is stored on the 1106 side in FIG.
Thereafter, MC, interframe subband decoding based on order information and motion vector information, which are normal basic inverse MCTF processes, and interpolation of pixels that cannot be filled with inverse MC and inverse MC are performed.
Thereafter, it is determined whether or not there is a frame to be processed in the current octave division. This time, decoding is performed from the second-order decomposition of the octave division to the first-order decomposition. Since there are the L component frame 1105 and the H2-6 frame that is the H component frame in FIG. Then, when the decoded L component frames such as 1112 and 1113 in FIG. 11 are obtained, the inverse MCTF processing for the frame whose octave division is the second-order decomposition is completed.

At this stage, when the L component frame is to be played back as a moving picture as scalability in the time direction, the moving picture is played back at the timing of the L component frame stored in 1110 to 1113 in FIG. Here, since the storage position cannot be specified yet, reproduction is performed assuming that it is arranged at the center of the time interval indicated by 1110 to 1113 in FIG.
Similarly, a final decoded frame can be obtained by repeating the basic inverse MCTF process until there is no frame to be processed for a frame whose octave division is first-order decomposition.
The frame positions of the obtained L component frames are the time intervals 1114 to 1121 in FIG. 11 which are the time intervals between the original frames. Therefore, the playback moving image at this stage is played back at the correct position and the correct frame rate. Is done.

As described above, in the case of the configuration in which the order information is acquired for each H component frame in the inverse MCTF, there is no problem in the decoding process itself, but when realizing scalability in the time direction, the frame at the time of encoding is used. Even though the intervals are encoded at unequal intervals, the decoding side displays the images at equal intervals that are not the positions of the original decoded frames.
Therefore, on the encoding side, when order information corresponding to each H component frame is output, it is preferable to generate an order information header by converting the information into the information shown in FIG. 12 and transmit it every GOP.
Here, as shown in FIG. 12, the order information corresponding to each H component frame is first converted into order information bits such as “0” for the forward direction and “1” for the reverse direction. Then, in order from the smallest octave division number, that is, the order information bits corresponding to the H2 component frame and the order corresponding to the H3 component frame in order from the order information bits corresponding to the H1 component frame obtained when the octave division is first-order decomposed. Align with information bits. By incorporating the sequence information bit strings arranged in this way into the header of 1 GOP as the sequence information header, it becomes possible to obtain the sequence information regarding the entire frame of 1 GOP when decoding 1 GOP.

Next, a method for specifying the positions of the frames encoded at unequal intervals on the encoding side from the order information header generated on the encoding side will be described with reference to FIG.
Reference numeral 1301 in FIG. 13 denotes an order information header obtained from a GOP header in which information necessary for configuring one GOP is stored. Also, from the GOP header, the number of frames of 1 GOP and the number of octave divisions of subband coding can be acquired. Here, the description will proceed assuming that the number of frames of 1 GOP is 8 frames and the number of octave divisions is a third-order decomposition.
First, in the order information bit string in the obtained order information header, the number of bits of the order information corresponding to the H component frame at the time of each octave division number is expressed as “1GOP frame number / (2 ^ current octave division number”. ) ". Here, since the number of frames of 1 GOP is 8 frames and the number of octave divisions is the third-order decomposition, the order corresponding to the H component frames when the octave division number is the first-order decomposition is shown as 1301 in FIG. The number of bits of the order information of H1, which is information, is 8 / (2 ^ 1) = 4 bits. Similarly, the number of bits of the order information of H2 which is the order information corresponding to the H component frame when the octave division number is the second order decomposition is 8 / (2 ^ 2) = 2 bits, and the octave division number is the third order decomposition. The number of bits of the order information of H3 which is the order information corresponding to the H component frame at the time of is 8 / (2 ^ 3) = 1 bit. In this way, the order information bit string 1306 of H1, the order information bit string 1312 of H2, and the order information bit string 1316 of H3 are specified from the order information bit string stored in the order information header 1301.

Next, as indicated by reference numerals 1302 to 1305 in FIG. 13, a frame position specifying memory for specifying the frame positions from L0 to L3 is secured. It is assumed that an integer indicating a frame position of 1 GOP can be stored in each block represented by 1302 to 1305 in FIG. Here, a positive integer from 0 to 7 can be stored in each block. Further, each frame position specifying memory reserves “1GOP frame number / (2 ^ current octave division number)” blocks.
First, information for specifying the position of the L0 frame is acquired. This is obtained by simply storing 0 to 7 as indicated by reference numeral 1307 in FIG.
After that, as indicated by reference numeral 1307 in FIG. 13, each order information bit of the H1 order information 1306 is associated with every two blocks of the L0 frame position specifying memory for each bit.

After that, according to the associated order information bit, if the order information bit is “0”, the value stored on the right side of the two associated blocks is adopted, and if the order information bit is “1”, it corresponds. The value stored on the left side of the attached two blocks is adopted. In FIG. 13, the adopted values correspond to 1308 to 1311 in FIG. This value is stored in the L1 frame position specifying memory as indicated by 1313 in FIG.
Similarly, by applying the H2 order information 1312 to the L1 frame position specifying memory, 1314 and 1315 in FIG. 13 are adopted and stored in the L2 frame position specifying memory as indicated by 1317 in FIG.
Further, by applying the H3 order information 1316 to the L2 frame position specifying memory, 1318 of FIG. 13 is adopted and stored in the L3 frame position specifying memory as indicated by 1319 of FIG.
Through such a process, the storage position of the L component frame in each octave division number of subband decoding can be specified in advance from the order information bit string of the order information header.

By acquiring the frame storage position information at the time of decoding from the order information header, the frame storage position at the time of decoding shown in FIG. 11 can be determined as shown in FIG.
First, in 1401 in FIG. 14, the contents of the L3 frame position specifying memory 1319 in FIG. 13 are referred to in order to store the L3-3 frame at the frame position in the GOP. Here, since it is 3, the L3-3 frame can be stored at the position of the third frame in the GOP. Also, when storing H3-4 frames, the number of octave divisions after the basic inverse MCTF processing is second-order decomposition, so the two blocks of the L2 frame position specifying memory in FIG. The H3-4 frame is stored in the fourth frame that is not.
As described above, the contents of the frame position specifying memory in FIG. 13 indicate the frame positions to be stored within the respective frame intervals indicated by reference numerals 1401 to 1421 in FIG. That is, the contents of the L3 frame position specifying memory 1319 in FIG. 13 indicate the frame storage position at 1401 in FIG. Further, the L2 frame position specifying memory 1317 in FIG. 13 shows the frame storage positions in 1402 and 1403 and 1404 and 1405 in FIG. Further, the contents of the L1 frame position specifying memory 1313 in FIG. 13 indicate the frame storage positions in 1406 to 1409 and 1410 to 1413 in FIG.
As described above, when the order information header can be obtained from the GOP header, the frame position when encoded at unequal intervals at the time of encoding is displayed, and the frame at the correct frame position is also displayed at the correct frame interval on the decoding side. Will be able to.

FIG. 17 is a detailed configuration of the encoding apparatus.
17 includes at least an input management unit 1702, a forward ME unit 1703, a backward ME unit 1704, a forward MC unit 1705, a backward MC unit 1706, a similarity determination unit 1707, a switch 1 (1708), and a switch 2 ( 1709), a control unit 1710, an interframe subband encoding unit 1711, an inverse MC unit 1712, a pixel interpolation unit 1713, an interframe subband encoding management unit 1714, a quantization unit 1715, an entropy encoding unit 1716, and a MUX unit 1717. Consists of.
The input management unit 1702 has a function of managing input video frames. In general, the input management unit 1702 acquires the number of frames necessary for configuring the GOP from the input moving image frame, and a forward ME unit 1703, a backward ME unit 1704, a forward MC unit 1705, which will be described later, The reverse direction MC unit 1706, the similarity determination unit 1707, the inter-frame subband encoding unit 1711, the reverse MC unit 1712, and the pixel interpolation unit 1713 have a function of notifying a frame to be processed.

  The forward ME unit 1703 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. Here, N is a positive integer starting from 0. The forward ME unit 1703 has a function of performing forward ME on at least two frames, 2N frames, and 2N + 1 frames acquired from the input management unit 1702 and acquiring motion vector information. The forward ME is processed on the assumption that the 2N frame side is a frame to be searched for specifying a motion vector, and the 2N + 1 frame side is a frame serving as a reference when performing ME. The forward ME unit 1703 has a function of notifying the forward MC unit 1705 of at least the acquired motion vector information. Furthermore, it has a function of notifying the acquired motion vector information to the inverse MC unit 1712 and the entropy encoding unit 1716 via the switch 1 (1708).

  The backward ME unit 1704 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. Here, N is a positive integer starting from 0. The backward ME unit 1704 has a function of performing backward ME on at least two frames, 2N frames, and 2N + 1 frames acquired from the input management unit 1702 and acquiring motion vector information. The ME in the reverse direction is processed on the assumption that the 2N + 1 frame side is a frame searched for specifying a motion vector, and the 2N frame side is a frame used as a reference when performing ME. . The backward ME unit 1704 has a function of notifying at least the acquired motion vector information to the backward MC unit 1706. Furthermore, it has a function of notifying the acquired motion vector information to the inverse MC unit 1712 and the entropy encoding unit 1716 via the switch 1 (1708).

  The forward MC unit 1705 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. In addition, it has a function of acquiring motion vector information obtained by the forward ME from the forward ME unit 1703. The forward MC unit 1705 has a function of performing forward MC on 2N frames and 2N + 1 frames based on the acquired motion vector information and acquiring forward MC images. The forward MC is a frame on the side where an MC image is acquired by performing MC based on motion vector information on the 2N frame side, and the 2N + 1 frame side is a frame used as a reference when performing ME. Process as a thing. The acquired forward MC image has a function of notifying the similarity determination unit 1707. In addition, it has a function of notifying the inter-frame subband encoding unit 1711 of the forward MC image via the switch 2 (1709).

  The backward MC unit 1706 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. Further, it has a function of acquiring motion vector information obtained from the backward ME unit 1704 by the backward ME. The reverse MC unit 1706 has a function of performing reverse MC on 2N frames and 2N + 1 frames based on the acquired motion vector information and acquiring a reverse MC image. The MC in the reverse direction is a frame on the side of acquiring an MC image by performing MC on the 2N + 1 frame side based on motion vector information, and the 2N frame side is a frame used as a reference when performing ME The processing is performed as follows. The acquired backward MC image has a function of notifying the similarity determination unit 1707. In addition, it has a function of notifying the inter-frame subband encoding unit 1711 of the MC image in the reverse direction via the switch 2 (1709).

  The similarity determination unit 1707 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. Further, it has a function of acquiring a forward MC image from the forward MC unit 1705 and a reverse MC image from the backward MC unit 1706. The similarity determination unit 1707 has a function of obtaining a forward similarity from between a forward MC image and 2N + 1 frames. In addition, it has a function of obtaining the similarity in the reverse direction from between the 2N frame and the MC image in the reverse direction. Here, the similarity is generally measured by measuring MSE (Mean Square Error) and determining the similarity based on a smaller value. In determining similarity, the sum of absolute values of differences between frames may be simply measured as an error between frames, and the similarity may be determined based on a smaller value. The degree of similarity may be determined by measuring the number of pixels in the non-bonded region when MC is performed, and determining the degree of similarity by making this value smaller. It should be noted that there is no particular limitation here as long as the superiority or inferiority of the error can be determined. Furthermore, the similarity determination unit 1707 has a function of comparing the measured forward similarity with the reverse similarity to identify an MC image with a higher similarity. If the direction of the specified MC image is the forward direction, the switch 1 (1708) and the switch 2 (1709) are switched to the respective terminal a side, and if the direction of the specified MC image is the reverse direction, the switch 1 (1708) and switch 2 (1709) have a function of switching to the respective terminal b side. The similarity determination unit 1707 has a function of notifying the control unit 1710 and the entropy encoding unit 1716 of order information for indicating whether the selected MC image is in the forward direction or the reverse direction.

The switch 1 (1708) is for selecting motion vector information notified from the forward ME unit 1703 and the backward ME unit 1704 according to the similarity determination unit 1707.
The switch 2 (1709) is for selecting motion vector information notified from the forward MC unit 1705 and the backward MC unit 1706 according to the similarity determination unit 1707.
The control unit 1710 has a function of acquiring order information for indicating whether the MC image selected from the similarity determination unit 1707 is in the forward direction or in the reverse direction. Further, it has a function of controlling operations of at least the inter-frame subband encoding unit 1711, the inverse MC unit 1712, the pixel interpolation unit 1713, and the inter-frame subband encoding management unit 1714. The reverse MC unit 1712 may have a function of notifying the overlapping order information of each reference region when performing reverse MC. However, in order to simplify the following description, it is assumed that the overlapping order information of each reference region is a predetermined order determined in advance by the encoding device and the decoding device, and the overlapping order information of each reference region is There is no need to mention the transfer. In this predetermined order, processing is performed in raster order from the reference area arranged at the upper left to the lower right. The control unit 1710 may have a function of notifying the pixel interpolation unit 1713 of pixel interpolation mode information for switching a pixel interpolation method. However, here, in order to simplify the following description, it is assumed that the pixel interpolation mode information is a predetermined pixel interpolation method determined in advance by the encoding device and the decoding device, and reference is made to the transfer of the pixel interpolation mode information. There is no need to do. The pixel interpolation method here specifies an interpolation source frame based on the order information acquired from the similarity determination unit 1707, and uses a pixel at a position to be interpolated in the interpolation source frame. The control unit 1710 has a function of managing the storage position of the processed frame from the position in the GOP of the currently processed frame and the state of band division between the frames. Further, the control unit 1710 has a function of notifying the inter-frame subband encoding management unit 1714 of band division state information indicating the state of band division and frame storage position information. The control unit 1710 acquires from the similarity determination unit 1707 when controlling at least the operations of the inter-frame subband encoding unit 1711, the inverse MC unit 1712, the pixel interpolation unit 1713, and the inter-frame subband encoding management unit 1714. It is preferable to provide a function for notifying order information.

  The inter-frame subband encoding unit 1711 has a function of acquiring at least two frames, 2N frames, and 2N + 1 frames to be processed from the frames in the GOP managed by the input management unit 1702. Here, based on the order information obtained from the control unit 1710, a function of selectively acquiring the frame that is the target of the MC out of the 2N frame and the 2N + 1 frame may be provided. In addition, it has a function of acquiring an MC image via the switch 2 (1709). The inter-frame subband encoding unit 1711 has a function of identifying which one of 2N frames and 2N + 1 frames is a target frame when creating an MC image from the obtained order information, and creates an MC image and an MC image. It has a function of performing subband coding between predetermined frames with the target frame that is the target when performing. In addition, the L component of the obtained subband coefficient information is stored on the MC image side to form an L component frame, and the H component of the obtained subband coefficient information is stored on the target frame side specified by the order information. It has a function as a frame. It has a function of notifying the generated L component frame to the inverse MC unit 1712 and also notifying the generated H component frame to the inter-frame subband encoding management unit 1714.

  The inverse MC unit 1712 has a function of acquiring order information from the control unit 1710. The inter-frame subband encoding unit 1711 has a function of acquiring an L component frame. It has a function of acquiring motion vector information via the switch 1 (1708). It has a function of performing motion compensation by applying the acquired motion vector information in the reverse direction, that is, a function of performing inverse MC and generating an L component frame after inverse MC. The inverse MC unit 1712 has a function of notifying the pixel interpolation unit 1713 of the generated L component frame after inverse MC. For a region where pixels are not filled by inverse MC, the pixel interpolation unit 1713 may be notified of pixel interpolation region information. Here, in order to simplify the processing, the pixel interpolation area information stores a value indicating that interpolation is necessary for the L component frame after inverse MC, for example, a value of “−1”, and the pixel interpolation unit 1713. Then, pixel interpolation area information is acquired based on this information. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information.

The pixel interpolation unit 1713 has a function of acquiring order information from the control unit 1710. The inverse MC unit 1712 has a function of acquiring an L component frame after inverse MC. It has a function that can acquire at least two frames, 2N frames, and 2N + 1 frames to be processed as frames before processing from the frames in the GOP managed by the input management unit 1702. Here, based on the order information obtained from the control unit 1710, a function may be provided in which a frame in which MC is performed out of 2N frames and 2N + 1 frames is selectively acquired as a pre-processing frame. The pixel interpolation unit 1713 has a function of performing pixel interpolation based on pixel interpolation area information, which is information relating to an area in which pixels are not filled with the inverse MC, in the acquired L component frame after inverse MC. The determination of the region where the pixel is not filled is usually acquired from the control unit 1710. Alternatively, a function of acquiring pixel interpolation area information from the inverse MC unit 1712 may be provided. In order to simplify the processing, the pixel interpolation area information is assumed to store a value indicating that interpolation is necessary for the L component frame after inverse MC, for example, a value of “−1”. The pixel interpolation area information is acquired based on the above. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information. The L component frame after pixel interpolation has a function of notifying the inter-frame subband encoding management unit 1714.
The inter-frame subband coding management unit 1714 has a function of acquiring an H component frame from the inter-frame subband coding unit 1711 and an L component frame obtained by performing pixel interpolation from the pixel interpolation unit 1713. Further, the control unit 1710 has a function of acquiring band division state information and frame storage position information. Based on the band division state information and the frame storage position information, it has a function of storing and notifying the L component frame and the H component frame to the correct positions. The notification destination of each frame is determined based on the band division state information. When it is necessary to perform band division between frames in the GOP, the input management unit 1702 is notified again as an input frame, and the band division is performed. When there is no need to perform the above, the function of notifying the quantization unit 1715 as an output frame is provided.

The quantization unit 1715 has a function of performing predetermined quantization on the L component frame and the H component frame that are subband coefficient information acquired from the interframe subband encoding management unit 1714. It has a function of notifying subband coefficient information after quantization to an entropy encoding unit 1716 described later.
The entropy encoding unit 1716 includes at least subband coefficient information such as an L component frame and an H component frame acquired from the quantization unit 1715, motion vector information acquired via the switch 1 (1704), and a similarity determination unit 1707. It has a function of performing predetermined entropy coding on the acquired order information and generating a subband coefficient information bit string, a motion vector information bit string, and an order information bit string. It has a function of notifying the MUX unit 1717 described later of the generated subband coefficient information bit string, motion vector information bit string, and order information bit string.
The MUX unit 1717 has a function of multiplexing the acquired subband coefficient information bit sequence, motion vector information bit sequence, order information bit sequence, and various control information necessary for decoding, generating a bit stream, and encoding the output.

Next, the operation of the encoding apparatus shown in FIG. 17 will be described below with the steps shown in the flowcharts of FIGS.
First, the input management unit 1702 acquires a moving image frame 1701 to be encoded (step S401), and accumulates 1 GOP worth of moving image frames (step S402). Here, for the sake of simplicity, it is assumed that 1 GOP is composed of 8 frames.
Next, in octave division of subband encoding when performing MCTF, a frame group to be subjected to subband encoding between frames is set (step S403). Here, since subband division is the first time, the acquired moving image frame for 1 GOP is applied to the frame group to be processed instead of the processed L component frame. From the next time onward, the processed L component frame group acquired from the inter-frame subband encoding management unit 1714 is applied as a frame group to be processed. Thereafter, in order to perform basic MCTF processing (step S404), a frame group to be subjected to basic MCTF processing is set (step S301). Here, from the input management unit 1702, 2N frames and 2N + 1 frames, which are two frames in one GOP as a frame group to be subjected to basic MCTF processing, are transmitted in the forward ME unit 1703, the reverse ME unit 1704, and the forward MC unit 1705. The input management unit 1702 is set so that the backward MC unit 1706, the similarity determination unit 1707, the inter-frame subband encoding unit 1711, and the pixel interpolation unit 1713 can be acquired.

The forward ME unit 1703 obtains 2N frames and 2N + 1 frames, which are frames to be processed, from the input management unit 1702, and obtains motion vector information in the forward direction by performing ME on the forward direction. Thereafter, the forward MC unit 1705 is notified of the obtained motion vector information, and is also notified to the inverse MC unit 1712 and the entropy encoding unit 1716 via the switch 1 (1708). The backward ME unit 1704 obtains 2N frames and 2N + 1 frames, which are frame groups to be processed, from the input management unit 1702, and obtains backward motion vector information by performing ME in the backward direction. After that, the motion vector information is notified to the reverse MC unit 1706, and is also notified to the reverse MC unit 1712 and the entropy encoding unit 1716 via the switch 1 (1708). Through such a process, ME can be performed in the forward direction / reverse direction to obtain a motion vector (step S302).
Thereafter, the forward MC unit 1705 acquires the motion vector information in the forward direction from the forward ME unit 1703. Also, a 2N frame that is a frame on the side performing forward MC is acquired from the input management unit 1702. A forward MC is performed from the forward motion vector information and 2N frames to generate a forward MC image. The generated forward MC image is notified to the similarity determination unit 1707 and to the inter-frame subband encoding unit 1711 via the switch 2 (1709). The backward MC unit 1706 acquires backward motion vector information from the backward ME unit 1704. Also, 2N + 1 frames, which are frames on the side performing backward MC, are obtained from the input management unit 1702. MC in the reverse direction is performed from the motion vector information in the reverse direction and the 2N + 1 frame to generate an MC image in the reverse direction. The generated MC image in the reverse direction is notified to the similarity determination unit 1707 and is also notified to the interframe subband encoding unit 1711 via the switch 2 (1709). Through this process, MC can be performed based on the motion vector information in the forward / reverse direction (step S303).

When the MC image in each direction is obtained, the similarity determination unit 1707 acquires the forward MC image from the forward MC unit 1705, and the reference used when creating the forward MC image from the input management unit 1702 A 2N + 1 frame that is a frame is acquired. A forward similarity is obtained between this forward MC image and 2N + 1 frames. Also, a reverse MC image is acquired from the reverse MC unit 1706, and a 2N frame, which is a target frame used as a reference when creating the reverse MC image from the input management unit 1702, is acquired. The similarity in the reverse direction is obtained between the MC image in the reverse direction and the 2N frame. Through such a process, the similarity between the MC image created in the forward direction / reverse direction and the target frame when the MC is performed can be obtained (step S304).
Thereafter, the similarity in the forward direction and the similarity in the reverse direction are compared (step S305), and the MC image having the higher similarity, that is, the error having the smaller error is employed. Further, order information for indicating the adopted direction is generated and notified to the control unit 1710 and the entropy encoding unit 1716 (step S306 or step S307). In order to notify each unit of the MC image and the motion vector information in the adopted direction, the similarity determination unit 1707 switches the switch 1 (1708) and the switch 2 (1709). When the forward MC image is selected, both the switch 1 (1708) and the switch 2 (1709) are switched to the a terminal side. When an MC image in the reverse direction is selected, both switches 1 (1708) and 2 (1709) are switched to the b terminal side.

After the similarity determination unit 1707 switches the switch 1 (1708) and the switch 2 (1709), the control unit 1710 acquires order information from the similarity determination unit 1707.
The control unit 1710 makes an encoding request to the inter-frame subband encoding unit 1711 based on the band division state information indicating the state of the band division and the frame storage position information, and notifies the order information.
The inter-frame subband encoding unit 1711 acquires order information from the control unit 1710. Further, an MC image is acquired via the switch 2 (1709). It determines whether the MC image acquired from the order information is in the forward direction or in the reverse direction, and acquires the target frame serving as the MC reference from the input management unit 1702. Inter-subband encoding is performed on the acquired MC image and target frame (step S308), and an L component frame is generated using the L component subband coefficient information as a frame position on the MC image side. An H component frame is generated using the subband coefficient information as a frame position on the target frame side. The generated L component frame is notified to the inverse MC unit 1712. Also, the H component frame is notified to the inter-frame subband encoding management unit 1714.
Thereafter, the control unit 1710 makes a reverse MC request to the reverse MC unit 1712 and notifies the order information.

The inverse MC unit 1712 acquires the encoded L component frame from the inter-frame subband encoding unit 1711. Also, motion vector information is acquired via the switch 1 (1708). Thereafter, a function for performing motion compensation by applying the acquired motion vector information in the reverse direction, that is, reverse MC is performed (step S309), and the L component frame after reverse MC is notified to the pixel interpolation unit 1713.
Thereafter, the control unit 1710 makes a pixel interpolation request to the pixel interpolation unit 1713 and notifies the order information.

The pixel interpolation unit 1713 acquires the L component frame after the inverse MC from the inverse MC unit 1712. Further, order information is acquired from the control unit 1710. From the acquired order information, a frame at the same position as the L component frame after reverse MC, that is, a frame to be an interpolation source before encoding processing on the MC side is acquired from the input management unit 1702. Thereafter, the pixel of the corresponding pixel interpolation area is specified from the frame that is the interpolation source based on the pixel interpolation area information, and the interpolation of the pixels that could not be filled with the inverse MC is performed on the L component frame after the inverse MC ( In step S310, the L component frame after pixel interpolation is notified to the inter-frame subband coding management unit 1714.
The inter-frame subband encoding management unit 1714 acquires the L component frame from the pixel interpolation unit 1713 and the H component frame from the inter-frame subband encoding unit 1711. Thereafter, it is determined whether or not there is a frame to be processed in the current octave division (step S311). If the octave division has not reached the predetermined division number, the input management unit 1702 is notified again of the L component frame. When the predetermined number of octave divisions is reached, the L component frame and the H component frame group generated by each octave division acquired so far are notified to the quantization unit 1715 as subband coefficient information after subband coding.
The quantization unit 1715 performs quantization on the L component frame and the H component frame group generated by each octave division (step S407), and notifies the subband coefficient information after quantization to the entropy encoding unit 1716 described later. To do.

The entropy encoding unit 1716 includes at least subband coefficient information such as an L component frame and an H component frame acquired from the quantization unit 1715, motion vector information acquired via the switch 1 (1704), and a similarity determination unit 1707. Predetermined entropy coding is performed on the acquired order information (step S408), and a subband coefficient information bit string, a motion vector information bit string, and an order information bit string are generated. The generated subband coefficient information bit string, motion vector information bit string, and order information bit string are notified to the MUX unit 1717 described later.
The MUX unit 1717 multiplexes the acquired subband coefficient information bit string, motion vector information bit string, order information bit string, and various control information necessary for decoding to form a bit stream (step S409), and outputs the encoded bit stream. .
By performing the processing as described above, a series of operations of the encoding device can be completed.

<Application example of encoding device>
FIG. 18 shows a configuration of an application example of the encoding device. The difference from the encoding apparatus shown in FIG. 17 is that an overlap determination unit 1801 and an error correction unit 1802 are newly provided.
Hereinafter, a part having a functional change and a newly added part will be described.
The input management unit 1702 newly has a function of notifying the identifying unit 1802 of overlapping frames to be processed.
The control unit 1710 has a new function of controlling the overlap specifying unit 1801 and the error correction unit 1802. In addition, a function for notifying the overlap specifying unit 1801 and the error correction unit 1802 of order information is newly provided.
The inter-frame subband encoding unit 1711 changes the notification destination of the function for notifying the H component frame to the error correction unit 1802.
The interframe subband encoding management unit 1714 changes the function of acquiring the H component frame from the interframe subband encoding unit 1711 so as to acquire the H component frame from the error correction unit 1802.

The overlap identifying unit 1801 acquires order information from the control unit 1710. The overlap identifying unit 1801 has a function of acquiring motion vector information from the input management unit 1702 via 2N frames, 2N + 1 frames, and the switch 1 (1708). Based on the acquired order information, from the acquired motion vector information and the shape information of the reference area, an area that is multiple-referenced when motion compensation is performed (hereinafter referred to as a multiple reference area) is specified, and multiple reference area information It has the function to generate. When performing motion compensation, that is, inverse motion compensation (hereinafter referred to as inverse MC), by applying the obtained motion vector information in the reverse direction to the L component frame after the band division, which reference region corresponds to which multiple reference It has a function of identifying whether the area is inversely MC and generating multiple reference area-reference area relationship information, which is information for representing the relationship between the corresponding multiple reference areas and a plurality of reference areas. Further, in each reference region reversed MC to the multiple reference region, it is specified which part in each reference region is involved in the multiple reference region, and an error correction unit 1802 to be described later corrects an error. As information for indicating error correction target area information. The overlap identifying unit 1801 has a function of notifying the error correction unit 1802 of multiple reference region information, multiple reference region-reference region relationship information, and error correction target region information.
The error correction unit 1802 has a function of acquiring multiple reference region information, multiple reference region-standard region relationship information, and error correction target region information from the overlap determination unit 1801. The error correction unit 208 has a function of calculating error information for correcting an error for the error correction target region in the reference region related to the multiple reference region of the H component frame from the acquired information. It has a function of adding the calculated error information to a region where error correction of the H component frame acquired from the inter-frame subband encoding unit 1711 is necessary. It has a function of notifying the inter-frame subband coding management unit 1714 of the H component frame subjected to error correction.
By providing the functions of the respective units as described above, it is possible to eliminate the dependency of the decoding process, which has been a problem in the conventional decoding device, and to facilitate the configuration of the decoding device.

Next, the operation of the application example of the encoding device shown in FIG. 18 will be described below with reference to the steps shown in the flowcharts of FIGS. However, the operation shown in FIG. 10 is the same as that shown in FIG. Step S404 in FIG. 10 corresponds to FIG.
In FIG. 18, the operations from step S701 to step S707 in FIG. 25 are the same as the operations in FIG.
When the processing by the similarity determination unit 1707 is completed, the control unit 1710 issues an encoding request to the inter-frame subband encoding unit 1711 as in the configuration of FIG. Subband encoding is performed between the acquired frames (step S708). The generated L component frame is notified to the inverse MC unit 1712, and the H component frame is notified to the error correction unit 1802.

Thereafter, the control unit 1710 notifies the overlap specifying unit 1801 of the order information.
The overlap identifying unit 1801 acquires order information from the control unit 1710. Also, motion vector information is acquired via the switch 1 (1708). Further, 2N frames and 2N + 1 frames are acquired from the input management unit 1702 based on the order information. Multiple reference area information is generated from the acquired motion vector information and reference area shape information (step S709). When performing motion compensation, that is, inverse motion compensation (hereinafter referred to as inverse MC), by applying the obtained motion vector information in the reverse direction to the L component frame after the band division, which reference region corresponds to which multiple reference Whether the area is inversely MC is identified, and multiple reference area-reference area relationship information, which is information for representing the relationship between the corresponding multiple reference area and a plurality of reference areas, is generated. Further, in each reference region reversed MC to the multiple reference region, it is specified which part in each reference region is involved in the multiple reference region, and an error correction unit 1802 to be described later corrects an error. Error correction target area information is generated as information for indicating After such a process, the overlap identifying unit 1801 notifies the error correction unit 1802 of the multiple reference region information, the multiple reference region-reference region relationship information, and the error correction target region information.

The error correction unit 1802 acquires multiple reference region information, multiple reference region-reference region relationship information, and error correction target region information from the overlap determination unit 1801. Also, an H component frame is acquired from the inter-frame subband encoding unit 1711. An error is corrected for the error correction target area in the reference area related to the multiple reference area of the H component frame from the acquired multiple reference area information, multiple reference area-reference area relation information, and error correction target area information. Error information is calculated. The calculated error information is added to the area where the error correction of the acquired H component frame is necessary. The H component frame subjected to error correction is notified to the inter-frame subband coding management unit 1714. Through such processing, error components are corrected for pixels in the multiple reference region (step S710).
After that, similarly to the configuration of FIG. 17, the inverse MC unit 1712 performs inverse MC based on the motion vector information (step S711) and the pixel interpolation unit 1713 does not fill with the inverse MC (step S712).
Thereafter, the inter-frame subband coding management unit 1714 acquires an L component frame from the pixel interpolation unit 1713 and an H component frame from the error correction unit 1802. Then, it is determined whether there is a frame to be processed in the current octave division (step S713). If the octave division has not reached the predetermined division number, the input management unit 1702 is notified again of the L component frame. When the predetermined number of octave divisions is reached, the L component frame and the H component frame group generated by each octave division acquired so far are notified to the quantization unit 1715 as subband coefficient information after subband coding.
The subsequent processing is the same as in FIG.
By performing the processing as described above, the operation of the series of encoding devices of the present invention can be completed.
The present invention includes a program for causing the central processing control device 1903, which is a computer shown in FIG. This program may be read from a recording medium and taken into the central processing control apparatus 1903, or may be transmitted via a communication network and taken into the central processing control apparatus 1903.

≪Decryption device≫
Next, in order to describe the configuration of the MCTF decoding apparatus, first, the configuration of the conventional MCTF decoding apparatus will be described below.
FIG. 20 is a functional block diagram showing a configuration example of a decoding device using a conventional general inverse MCTF.
20 includes at least a DEMUX unit 2002, an entropy decoding unit 2003, an inverse quantization unit 2004, an input management unit 2005, a correspondence relationship specifying unit 2006, a first inter-frame decoding unit 2007, and a second inter-frame decoding unit 2008. , A pixel interpolation unit 2009 and an inter-frame decoding management unit 2010.
The DEMUX unit 2002 demultiplexes the acquired bit stream (hereinafter referred to as DEMUX), and obtains a coefficient information bit string after subband division (hereinafter referred to as subband coefficient information bit string), a motion vector information bit string, and decoding. It has a function of generating various control information and notifying a subband coefficient information bit string and a motion vector information bit string to an entropy decoding unit 2003 (to be described later).
The entropy decoding unit 2003 has a function of acquiring the subband coefficient information bit sequence and the motion vector information bit sequence notified from the DEMUX unit 2002. It has a function of performing entropy decoding on the acquired subband coefficient information bit sequence and motion vector information bit sequence, and generating coefficient information after subband division (hereinafter, subband coefficient information) and motion vector information. It has a function of notifying the generated subband coefficient information to an inverse quantization unit 2004 described later. Further, it has a function of notifying the generated motion vector information to the correspondence specifying unit 2006.

The inverse quantization unit 2004 has a function of acquiring subband coefficient information from the entropy decoding unit 2003. It has a function of performing predetermined inverse quantization on the acquired subband coefficient information. It has a function of notifying subband coefficient information after inverse quantization to the input management unit 2005 described later.
The input management unit 2005 has a function of acquiring subband coefficient information after inverse quantization from the inverse quantization unit 2004. The obtained subband coefficient information after inverse quantization is an L component frame including L component information after subband division and an H component frame including H component information, and each component frame is a frame in a GOP after decoding. It has a function of managing in association with the position. In general, the input management unit 2005 acquires subband coefficient information that can configure the number of frames necessary to reconstruct a GOP from the inverse quantization unit 2004, and a correspondence relationship specifying unit 2006 and a pixel interpolation unit 2009 described later. Has a function of notifying a frame to be processed.

  The correspondence relationship specifying unit 2006 has a function of acquiring motion vector information from the entropy decoding unit 2003. Also, an L component frame storing L component subband coefficient information after subband division to be decoded and an H component frame storing H component subband coefficient information are acquired from the input management unit 2005. It has the function to do. Here, the L component frame corresponds to the position on the 2N frame side after decoding, and the H component frame corresponds to the position on the 2N + 1 frame side after decoding. Using the acquired motion vector information, specify which L component subband coefficient information in the L component frame the subband coefficient information of each H component in the H component frame corresponds to, and subband coefficient It has a function of generating correspondence information. Here, in the subband coefficient correspondence information, each L component subband coefficient information holds the spatial position in the L component frame, so that the storage position of the decoded subband coefficient information can be specified. . Similarly, the subband coefficient information of each H component in correspondence holds the spatial position in the H component frame, so that the storage position of the decoded subband coefficient information can be specified. In addition, when there is a multiple reference relationship between subband coefficient information of a specific L component and a plurality of H component subband coefficient information, the L component subband to be decoded first based on a predetermined method. The band coefficient information and the H component subband coefficient information are specified, and the subband coefficient correspondence information has a function of reflecting the priority when decoding is performed. The correspondence specifying unit 2006 has a function of notifying the first interframe decoding unit 2007 of the acquired L component frame, H component frame, and generated subband coefficient correspondence relationship information.

The first inter-frame decoding unit 2007 has a function of acquiring from the correspondence specifying unit 2006 the L component frame on the 2N frame side, the H component frame on the 2N + 1 frame side, and the generated subband coefficient correspondence information. First, the L component subband coefficient information to be decoded and the H component subband coefficient information on the 2N + 1 frame side are identified from the acquired subband coefficient correspondence information, and band synthesis is performed by predetermined subband synthesis. It has a function of storing subband coefficient information of two L components that differ in the time direction obtained by band synthesis in corresponding spatial positions on the 2N frame side and 2N + 1 frame side based on the subband coefficient correspondence information. Notifying the second interframe decoding section 2008 of the L component subband coefficient information stored in the 2N frame side and 2N + 1 frame side after band synthesis, subband coefficient information not subjected to band synthesis, and subband coefficient correspondence information It has the function to do.
The second inter-frame decoding unit 2008 stores the L component sub-band coefficient information stored in the 2N frame side and the 2N + 1 frame side after band synthesis from the first inter-frame decoding unit 2007, and the sub-band coefficient not subjected to band synthesis. Information and subband coefficient correspondence information. Correspondence between 2N frame side L component subband coefficient information already decoded by the first interframe decoding unit 2007 and H component subband coefficient information that has not been decoded by the first interframe decoding unit 2007 The relationship is specified from the subband coefficient correspondence information, band synthesis is performed by predetermined subband synthesis, and subband coefficient information of two L components different in the time direction obtained by band synthesis is subband coefficient correspondence information. 2N frame side, 2N + 1 frame side corresponding to the spatial position. It has a function of notifying the pixel interpolation unit 2009 of L component subband coefficient information stored on the 2N frame side after band synthesis. It has a function of notifying the inter-frame decoding management unit 2010 of the L component subband coefficient information stored on the 2N + 1 frame side.

The pixel interpolation unit 2009 has a function of acquiring L component subband coefficient information stored on the 2N frame side from the second inter-frame decoding unit 2008. The pixel interpolation unit 2009 has a function of specifying a pixel interpolation region that has not been decoded by subband synthesis within the 2N frame, and interpolating pixels based on a predetermined pixel interpolation method. It has a function of notifying the generated 2N frame to the inter-frame decoding management unit 2010.
The inter-frame decoding management unit 2010 has a function of acquiring 2N frames subjected to pixel interpolation from the pixel interpolation unit 2009. It has a function of acquiring 2N + 1 frames from the second inter-frame decoding unit 2008. The inter-frame decoding management unit 2010 determines whether or not to continue band synthesis, and if it is necessary to perform band synthesis between frames in the GOP, the 2N frame and 2N + 1 frame are again input frames in the GOP. To the input management unit 2005, and when there is no need to perform band synthesis, it has a function of outputting a moving image frame as a decoding result.

FIG. 26 is a flowchart showing the operation of the decoding apparatus using the conventional general inverse MCTF shown in FIG. The operation of the decoding apparatus using the conventional general inverse MCTF shown in FIG. 20 will be described below with reference to FIG.
First, the DEMUX unit 2002 acquires an input bit stream (step S801). DEMUX is performed on the acquired bit stream (step S802), and the generated subband coefficient information bit sequence and motion vector information bit sequence are notified to the entropy decoding unit 2003.
Next, the entropy decoding unit 2003 acquires a subband coefficient information bit sequence and a motion vector information bit sequence from the DEMUX unit 2002, and performs predetermined entropy decoding on each piece of information (step S803). Thereafter, the sub-band coefficient information obtained after entropy coding is notified to the inverse quantization unit 2004. Also, the motion vector information is notified to the correspondence relationship specifying unit 2006.
After that, the inverse quantization unit 2004 performs inverse quantization on the subband coefficient information acquired from the entropy decoding unit 2003 (step S804). The subband coefficient information after inverse quantization is notified to the input management unit 2005.

The input management unit 2005 acquires the subband coefficient information after dequantization from the dequantization unit 2004, and sets the L component frame including the L component information and the H component frame including the H component information after the subband division. By storing these frames as information for one GOP (step S805), each component frame is managed in association with the frame position in the decoded GOP. Thereafter, the correspondence relationship specifying unit 2006 and the pixel interpolation unit 2009 are notified of a frame to be processed (step S806).
Thereafter, the correspondence relationship specifying unit 2006 acquires motion vector information from the entropy decoding unit 2003. Also, an L component frame storing L component subband coefficient information after subband division to be decoded and an H component frame storing H component subband coefficient information are acquired from the input management unit 2005. To do. Using the acquired motion vector information, it is identified which subband coefficient information of each L component in the L component frame is associated with the subband coefficient information of each H component in the H component frame (step S807). Then, subband coefficient correspondence information is generated. Here, in the subband coefficient correspondence relationship information, the subband coefficient information of each L component holds the spatial position in the L component frame so that the storage position of the decoded subband coefficient information can be specified. . Similarly, the subband coefficient information of each H component in the correspondence relationship holds the spatial position in the H component frame, so that the storage position of the decoded subband coefficient information can be specified. In addition, when there is a multiple reference relationship between subband coefficient information of a specific L component and a plurality of H component subband coefficient information, the L component subband to be decoded first based on a predetermined method. The band coefficient information and the H component subband coefficient information are specified, and the priority in decoding is reflected in the subband coefficient correspondence information. The correspondence specifying unit 2006 notifies the first interframe decoding unit 2007 of the acquired L component frame, H component frame, and the generated subband coefficient correspondence information.

  Next, the first interframe decoding unit 2007 obtains the L component frame on the 2N frame side, the H component frame on the 2N + 1 frame side, and the generated subband coefficient correspondence relationship information from the correspondence relationship specifying unit 2006. By identifying the subband coefficient information of the L component to be decoded first and the subband coefficient information of the H component on the 2N + 1 frame side from the acquired subband coefficient correspondence information, band synthesis is performed by predetermined subband synthesis. The first inter-frame decoding is performed (step S808), and two L component subband coefficient information different in the time direction obtained by band synthesis is converted into the 2N frame side, 2N + 1 based on the subband coefficient correspondence information. Store in the corresponding spatial position on the frame side. Notifying the second interframe decoding section 2008 of the L component subband coefficient information stored in the 2N frame side and 2N + 1 frame side after band synthesis, subband coefficient information not subjected to band synthesis, and subband coefficient correspondence information To do.

  Then, the second inter-frame decoding unit 2008 stores the L component sub-band coefficient information stored in the 2N frame side and the 2N + 1 frame side after the band synthesis from the first inter-frame decoding unit 2007, and the sub-band that is not subjected to the band synthesis. Band coefficient information and subband coefficient correspondence information are acquired. Correspondence between 2N frame side L component subband coefficient information already decoded by the first interframe decoding unit 2007 and H component subband coefficient information that has not been decoded by the first interframe decoding unit 2007 By identifying the relationship from the subband coefficient correspondence information and performing band synthesis by predetermined subband synthesis, the second interframe decoding is performed using the result of the first interframe decoding (step S809). ) And the subband coefficient information of two L components different in the time direction obtained by band synthesis are stored in the corresponding spatial positions on the 2N frame side and 2N + 1 frame side based on the subband coefficient correspondence information. After that, the pixel interpolation unit 2009 is notified of the subband coefficient information of the L component stored on the 2N frame side after band synthesis. In addition, the L-component subband coefficient information stored on the 2N + 1 frame side is notified to the interframe decoding management section 2010.

Next, the pixel interpolation unit 2009 obtains the L component subband coefficient information stored on the 2N frame side from the second inter-frame decoding unit 2008. The pixel interpolation unit 2009 identifies a pixel interpolation region that has not been decoded by subband synthesis in the 2N frame, and interpolates pixels based on a predetermined pixel interpolation method, thereby performing the first interframe decoding. Interpolation of pixels that could not be filled is performed (step S810). The generated 2N frame is notified to the inter-frame decoding management unit 2010.
Thereafter, the inter-frame decoding management unit 2010 acquires 2N frames subjected to pixel interpolation from the pixel interpolation unit 2009 and also acquires 2N + 1 frames from the second inter-frame decoding unit 2008. The inter-frame decoding management unit 2010 determines whether or not to continue band synthesis, and if it is necessary to perform band synthesis between frames in the GOP, the 2N frame and 2N + 1 frame are again input frames in the GOP. To the input management unit 2005, and when it is not necessary to perform band synthesis, a moving image frame is output as a decoding result.
By performing the processing as described above, a series of decoding processing using inverse MCTF is completed.

Next, the configuration of the MCTF decoding device will be described.
The MCTF decoding device shown in FIG. 21 is realized by providing at least an order information analysis unit 2101 with respect to the configuration of the conventional MCTF decoding device shown in FIG.
The order information analysis unit 2101 has a function of acquiring an order information header from the entropy decoding unit 2003. Alternatively, when the order information header cannot be acquired, the order information added for each H component frame may be acquired. By performing the analysis as shown in FIG. 13 from this order information, the frame storage position specifying information for specifying the order relation and the frame storage position of the decoded L component frames in one GOP for each octave division is obtained. It has a function to generate. The generated frame storage position information has a function of notifying the correspondence relationship specifying unit 2006 and the interframe decoding management unit 2010.

  The correspondence relationship specifying unit 2006 further has a function of acquiring frame storage position specifying information from the order information analyzing unit 2101. Based on the acquired frame storage position specifying information, the L component frame storing the L component subband coefficient information after subband division to be decoded from the input management unit 2005, and the H component subband coefficient information are stored. It has a function of acquiring the stored H component frame. Here, from the acquired frame storage position information, it has a function of specifying whether the frame that is currently processed is subjected to forward ME / MC or reverse ME / MC. . In the case of the forward direction, the L component frame corresponds to the position on the 2N frame side after decoding, and the H component frame corresponds to the position on the 2N + 1 frame side after decoding. In the reverse direction, the L component frame corresponds to the position on the 2N + 1 frame side after decoding, and the H component frame corresponds to the position on the 2N frame side after decoding. Using the acquired motion vector information, specify which L component subband coefficient information in the L component frame the subband coefficient information of each H component in the H component frame corresponds to, and subband coefficient It has a function of generating correspondence information. Here, in the subband coefficient correspondence information, each L component subband coefficient information holds the spatial position in the L component frame, so that the storage position of the decoded subband coefficient information can be specified. . Similarly, the subband coefficient information of each H component in correspondence holds the spatial position in the H component frame, so that the storage position of the decoded subband coefficient information can be specified. In addition, when there is a multiple reference relationship between subband coefficient information of a specific L component and a plurality of H component subband coefficient information, the L component subband to be decoded first based on a predetermined method. The band coefficient information and the H component subband coefficient information are specified, and the subband coefficient correspondence information has a function of reflecting the priority when decoding is performed. The correspondence specifying unit 2006 has a function of notifying the first interframe decoding unit 2007 of the acquired L component frame, H component frame, and generated subband coefficient correspondence relationship information.

FIG. 27 is a flowchart showing the operation of the decoding apparatus using the inverse MCTF shown in FIG. The operation of the decoding apparatus shown in FIG. 21 will be described below using FIG.
First, the DEMUX unit 2002 acquires an input bit stream (step S901). DEMUX is performed on the acquired bit stream (step S902), and the generated subband coefficient information bit string, motion vector information bit string, and order information bit string are notified to the entropy decoding unit 2003.
Next, the entropy decoding unit 2003 acquires a subband coefficient information bit sequence, a motion vector information bit sequence, and an order information bit sequence from the DEMUX unit 2002, and performs predetermined entropy decoding on each piece of information (step S903). Thereafter, the sub-band coefficient information obtained after entropy coding is notified to the inverse quantization unit 2004. Also, the motion vector information is notified to the correspondence relationship specifying unit 2006. The order information bit string is notified to the order information analysis unit 2101.
After that, the inverse quantization unit 2004 performs inverse quantization on the subband coefficient information acquired from the entropy decoding unit 2003 (step S904). The subband coefficient information after inverse quantization is notified to the input management unit 2005.

The input management unit 2005 acquires the subband coefficient information after dequantization from the dequantization unit 2004, and sets the L component frame including the L component information and the H component frame including the H component information after the subband division. By storing these frames as information for one GOP (step S905), each component frame is managed in association with the frame position in the decoded GOP.
The order information analysis unit 2101 acquires the order information bit string that is the content of the order information header from the entropy decoding unit 2003 (step S906). By performing the analysis as shown in FIG. 13 for the acquired order information bit string, the frame storage position for specifying the order relation and the frame storage position of the decoded L component frames in one GOP for each octave division Generate specific information. The generated frame storage location information is notified to the correspondence specifying unit 2006 and the interframe decoding management unit 2010.
Thereafter, the input management unit 2005 notifies the correspondence specifying unit 2006 and the pixel interpolation unit 2009 of the frame to be processed (step S907).

  Thereafter, the correspondence relationship specifying unit 2006 acquires motion vector information from the entropy decoding unit 2003. Further, frame storage position specifying information is acquired from the order information analysis unit 2101. Based on the acquired frame storage position specifying information, the L component frame storing the L component subband coefficient information after subband division to be decoded from the input management unit 2005, and the H component subband coefficient information are stored. The stored H component frame is acquired. Here, it is specified from the acquired frame storage position information whether the frame to be processed currently has been subjected to forward ME / MC or reverse ME / MC. In the case of the forward direction, the L component frame corresponds to the position on the 2N frame side after decoding, and the H component frame corresponds to the position on the 2N + 1 frame side after decoding. In the reverse direction, the L component frame corresponds to the position on the 2N + 1 frame side after decoding, and the H component frame corresponds to the position on the 2N frame side after decoding. Using the acquired motion vector information, it is specified which subband coefficient information of each L component in the H component frame is associated with which subband coefficient information of each L component in the H component frame (step S908). Then, subband coefficient correspondence information is generated. Here, in the subband coefficient correspondence information, each L component subband coefficient information holds the spatial position in the L component frame, so that the storage position of the decoded subband coefficient information can be specified. . Similarly, the subband coefficient information of each H component in correspondence holds the spatial position in the H component frame, so that the storage position of the decoded subband coefficient information can be specified. In addition, when there is a multiple reference relationship between subband coefficient information of a specific L component and a plurality of H component subband coefficient information, the L component subband to be decoded first based on a predetermined method. The band coefficient information and the H component subband coefficient information are specified, and the priority in decoding is reflected in the subband coefficient correspondence information. The correspondence specifying unit 2006 notifies the first inter-frame decoding unit 2007 of the acquired L component frame, H component frame, and the generated subband coefficient correspondence information.

  Next, if the first inter-frame decoding unit 2007 is forward from the correspondence specifying unit 2006, the L component frame on the 2N frame side, the H component frame on the 2N + 1 frame side, and the generated subband coefficient correspondence information To get. In the reverse direction, the L component frame on the 2N + 1 frame side, the H component frame on the 2N frame side, and the generated subband coefficient correspondence information are acquired. The L component subband coefficient information to be decoded first from the acquired subband coefficient correspondence information and the H component subband coefficient information on the 2N + 1 frame side in the forward direction and the 2N frame side in the reverse direction are specified. Then, by performing band synthesis by predetermined subband synthesis, first inter-frame decoding is performed (step S909), and subband coefficient information of two L components that are different in the time direction obtained by band synthesis is Based on the subband coefficient correspondence information, the data is stored in the corresponding spatial positions on the 2N frame side and 2N + 1 frame side. Notifying the second interframe decoding section 2008 of the L component subband coefficient information stored in the 2N frame side and 2N + 1 frame side after band synthesis, subband coefficient information not subjected to band synthesis, and subband coefficient correspondence information To do.

  Then, the second inter-frame decoding unit 2008 stores the L component sub-band coefficient information stored in the 2N frame side and the 2N + 1 frame side after the band synthesis from the first inter-frame decoding unit 2007, and the sub-band that is not subjected to the band synthesis. Band coefficient information and subband coefficient correspondence information are acquired. L component sub-band coefficient information already decoded by the first inter-frame decoding unit 2007 for the 2N frame side in the forward direction and 2N + 1 frame side for the reverse direction, and decoded by the first inter-frame decoding unit 2007 By identifying the correspondence with the subband coefficient information of the H component that is not the target from the subband coefficient correspondence information, and performing band synthesis by predetermined subband synthesis, the result of the first inter-frame decoding is obtained. The second inter-frame decoding is performed by using the second inter-frame decoding (step S910), and two L component sub-band coefficient information obtained in the band direction is obtained on the 2N frame side based on the sub-band coefficient correspondence information. Store in the corresponding spatial position on the 2N + 1 frame side. After that, the pixel interpolation unit 2009 is notified of the L component subband coefficient information stored in the 2N frame side in the forward direction after band synthesis and in the 2N + 1 frame side in the reverse direction. Further, the inter-frame decoding management unit 2010 is notified of the L component subband coefficient information stored in the 2N + 1 frame side in the forward direction and in the 2N frame side in the reverse direction.

Next, the pixel interpolation unit 2009 obtains subband coefficient information of the L component stored on the 2N frame side in the forward direction from the second inter-frame decoding unit 2008 and on the 2N + 1 frame side in the reverse direction. The pixel interpolation unit 2009 identifies a pixel interpolation region that has not been decoded by subband combining within 2N frames in the forward direction and 2N + 1 frames in the reverse direction, and selects pixels based on a predetermined pixel interpolation method. Interpolation is performed for pixels that could not be filled by the first inter-frame decoding (step S911). If the generated forward direction, the 2N frame is notified to the inter-frame decoding management unit 2010, and if it is the reverse direction, the 2N + 1 frame is notified.
After that, the inter-frame decoding management unit 2010 obtains 2N frames in the forward direction when the pixel interpolation is performed from the pixel interpolation unit 2009, and 2N + 1 frames in the reverse direction, and from the second inter-frame decoding unit 2008. If it is forward, 2N + 1 frames are acquired, and if it is backward, 2N frames are acquired. Further, it has a function of acquiring frame storage position information from the order information analysis unit 2101. Using the acquired frame storage position information, the acquired 2N frame, 2N + 1 frame is stored in the frame storage position in 1 GOP. The inter-frame decoding management unit 2010 determines whether or not to continue band synthesis, and if it is necessary to perform band synthesis between frames in the GOP, the 2N frame and 2N + 1 frame are again input frames in the GOP. To the input management unit 2005, and when it is not necessary to perform band synthesis, a moving image frame is output as a decoding result.
By performing the processing as described above, a series of decoding processing using inverse MCTF is completed.
The present invention includes a program for causing the central processing control device 2203, which is a computer shown in FIG. This program may be read from a recording medium and taken into the central processing control device 2203, or may be transmitted via a communication network and taken into the central processing control device 2203.

≪Decoding device application examples≫
FIG. 23 is a functional block diagram illustrating a process from input of a bit stream to output of a decoded moving image frame in an application example of the MCTF decoding apparatus. The function of each block shown in FIG. 23 is shown below.
As shown in FIG. 23, an application example of the decoding apparatus includes a DEMUX unit 2002, an entropy decoding unit 2003, an inverse quantization unit 2004, an input management unit 2301, an MC unit 2302, an inter-frame subband decoding unit 2303, an overlap The sequence control unit 2304, the inverse MC unit 2305, the pixel interpolation unit 2306, the inter-frame subband decoding management unit 2307, and the sequence information analysis unit 3208 are configured at least.
The DEMUX unit 2002 demultiplexes the acquired bitstream (hereinafter referred to as DEMUX) to generate a subband coefficient information bit string, a motion vector information bit string, an order information bit string, and various control information necessary for decoding, The subband coefficient information bit string, motion vector information bit string, and order information bit string have a function of notifying the entropy decoding unit 2003 described later.

The entropy decoding unit 2003 has a function of acquiring a subband coefficient information bit sequence, a motion vector information bit sequence, and an order information bit sequence notified from the DEMUX unit 2002. The obtained subband coefficient information bit string, motion vector information bit string, and order information bit string are subjected to entropy decoding to generate subband coefficient information, motion vector information, and order information. It has a function of notifying the generated subband coefficient information to an inverse quantization unit 2004 described later. In addition, it has a function of notifying the generated motion vector information to the MC unit 2302 and the inverse MC unit 2305 described later. Furthermore, it has a function of notifying the generated order information to the order information analysis unit 2308.
The inverse quantization unit 2004 has a function of acquiring subband coefficient information from the entropy decoding unit 2003. It has a function of performing predetermined inverse quantization on the acquired subband coefficient information. It has a function of notifying sub-band coefficient information after inverse quantization to an input management unit 2301 described later.
The input management unit 2301 has a function of acquiring subband coefficient information after inverse quantization from the inverse quantization unit 2004. The acquired subband coefficient information after inverse quantization has a function of managing as an L component frame including L component information after subband division and an H component frame including H component information. In general, the input management unit 2301 obtains subband coefficient information that can configure the number of frames necessary to reconstruct the GOP from the dequantization unit 2004, and an MC unit 2302, which will be described later, interframe subband decoding. A function of notifying a unit 2303 and a pixel interpolation unit 2306 of a frame to be processed.

  The MC unit 2302 has a function of acquiring frame storage position specifying information from the order information analysis unit 2308. Based on the acquired frame storage position specifying information, the L component frame storing the subband coefficient information of the L component after subband division to be decoded from the input management unit 2301 and the subband coefficient information of the H component are stored. It has a function of acquiring the stored H component frame. Here, from the acquired frame storage position information, it has a function of specifying whether the frame that is currently processed is subjected to forward ME / MC or reverse ME / MC. . In the case of the forward direction, the L component frame corresponds to the position on the 2N frame side after decoding, and the H component frame corresponds to the position on the 2N + 1 frame side after decoding. In the reverse direction, the L component frame corresponds to the position on the 2N + 1 frame side after decoding, and the H component frame corresponds to the position on the 2N frame side after decoding. In the forward direction, it is assumed that the 2N frame is a frame on the encoding side where a motion vector search is performed when performing motion estimation. Further, it is assumed that the 2N + 1 frame is a frame serving as a reference when performing motion estimation on the encoding side. If it is in the reverse direction, the 2N + 1 frame is assumed to be a frame on the side where a motion vector is searched when performing motion estimation on the encoding side. Further, it is assumed that the 2N frame is a frame serving as a reference when performing motion estimation on the encoding side. The MC unit 2302 usually has a predetermined range for extending an area outside the frame in order to allow an out-of-area reference to an L component frame corresponding to 2N frames in the forward direction and 2N + 1 frames in the reverse direction. It has a function to perform padding processing. Here, when out-of-region reference is prohibited on the encoding side, out-of-region reference may be prohibited in the MC unit 2302. When out-of-region reference is prohibited, it is not always necessary to perform a predetermined padding process. The MC unit 2302 has a function of acquiring motion vector information from the entropy decoding unit 2003. The MC unit 2302 has a function of generating a motion compensation frame (hereinafter referred to as an MC frame) based on the L component frame after padding, the shape information of the corresponding region, and motion vector information. Here, the shape information of the reference region used when performing MC is 2N + 1 frames in the forward direction and 2N frames in the reverse direction in the encoding side motion estimation unit (hereinafter referred to as ME unit). Since the area is divided into a plurality of reference areas so as not to overlap each other based on a predetermined area dividing method, it is necessary to acquire shape information of the reference area. However, in order to simplify the following description, it is assumed that the shape of the reference region is a predetermined 4 × 4 block shape determined in advance by the encoding device and the decoding device, and the shape information of the reference region is exchanged. Need not be mentioned. The MC unit 2302 has a function of notifying the inter-frame subband decoding unit 2303 of the MC frame generated based on the L component frame.

The inter-frame subband decoding unit 2303 applies to the H component frame corresponding to the 2N + 1 frame in the forward direction when decoding from the input management unit 2301, the 2N frame corresponding to the 2N frame in the reverse direction, and the L component frame from the MC unit 2302 A function of acquiring an MC frame in which MC is performed. The obtained MC frame and H component frame are subjected to band synthesis by predetermined subband synthesis, and two L component frames obtained by band synthesis that differ in the time direction are stored in the corresponding 2N frame and 2N + 1 frame. It has a function. In the forward direction, 2N frames are notified, and in the reverse direction, 2N + 1 frames are notified to the inverse MC unit 2305, which will be described later. In addition, 2N + 1 frames are transmitted in the forward direction, and 2N frames are transmitted in the reverse direction. Has a function of notifying the system management unit 2307.
The overlapping order control unit 2304 has a function of notifying the reverse MC unit 2305 of the overlapping order information of each reference region when performing reverse MC. However, in order to simplify the following description, it is assumed that the overlapping order information of each reference region is a predetermined order determined in advance by the encoding device and the decoding device. There is no need to mention the transfer. The predetermined order here is such that the overlapping order control unit 2304 controls the reverse MC unit 2305 by performing processing in a raster order from the reference region arranged at the upper left toward the lower right.

  The inverse MC unit 2305 has a function of acquiring motion vector information from the entropy decoding unit 2003 and 2N frames in the forward direction and 2N + 1 frames in the reverse direction from the interframe subband decoding unit 2303. Based on the obtained motion vector information and shape information of the reference area, the overlap is controlled based on the overlapping order of each reference area for 2N frames in the forward direction after band synthesis and 2N + 1 frames in the reverse direction. A function for performing motion compensation by applying the acquired motion vector information in the reverse direction in accordance with the control of the order control unit 2304, that is, performing reverse MC. For example, it has a function of generating 2N + 1 frames. The reverse MC unit 2305 has a function of notifying the pixel interpolation unit 2306 of 2N frames in the forward direction after the generated reverse MC and 2N + 1 frames in the reverse direction. For a region where pixels are not filled by inverse MC, the pixel interpolation unit 2306 may be notified of pixel interpolation region information. Here, in order to simplify the processing, the pixel interpolation area information is a value indicating that interpolation is necessary for 2N frames in the forward direction after reverse MC, and 2N + 1 frames in the reverse direction, for example, “−”. 1 ”is stored, and the pixel interpolation unit 2306 acquires pixel interpolation area information based on this information. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information.

  If the forward direction before decoding from the input management unit 2301 is 2N frames, the pixel interpolation unit 2306 is an L component frame corresponding to 2N + 1 frames if it is backward, and if it is the forward direction after reverse MC from the inverse MC unit 2305. It has a function to acquire 2N frames and 2N + 1 frames in the reverse direction. The pixel interpolation unit 2306 is based on the pixel interpolation area information that is information about the area in which the pixels are not filled with the inverse MC in the 2N frame in the forward direction after the obtained reverse MC and in the 2N + 1 frame in the reverse direction. It has a function of interpolating pixels. The determination of an area where pixels are not filled may have a function of acquiring pixel interpolation area information from the inverse MC unit 2305. In order to simplify the processing, the pixel interpolation area information is a value indicating that interpolation is required for 2N frames in the forward direction after reverse MC, and 2N + 1 frames in the reverse direction, for example, “−1”. The pixel interpolation area information is acquired based on this information. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information. The forward direction after pixel interpolation has a function of notifying the inter-frame subband decoding management unit 2307 of 2N frames and the reverse direction of 2N + 1 frames.

The inter-frame subband decoding management unit 2307 has a function of acquiring 2N frames in the forward direction when the pixel interpolation is performed from the pixel interpolation unit 2306 and 2N + 1 frames in the reverse direction. Further, the inter-frame subband decoding unit 2303 has a function of acquiring 2N + 1 frames in the forward direction and 2N frames in the reverse direction. Further, it has a function of acquiring frame storage position information from the order information analysis unit 2308. Using the acquired frame storage position information, the acquired 2N frame, 2N + 1 frame is stored in the frame storage position in 1 GOP. The inter-frame subband decoding management unit 2307 determines whether or not to continue band synthesis, and when it is necessary to perform band synthesis between frames in the GOP, the 2N frame and 2N + 1 frame are again included in the GOP. In order to handle it as an input frame, it has a function of notifying the input management unit 2301 and outputting a moving image frame as a decoding result when it is not necessary to perform band synthesis.
The order information analysis unit 3208 has a function of acquiring an order information header from the entropy decoding unit 2003. Alternatively, when the order information header cannot be acquired, the order information added for each H component frame may be acquired. By performing the analysis as shown in FIG. 13 from this order information, the frame storage position specifying information for specifying the order relation and the frame storage position of the decoded L component frames in one GOP for each octave division is obtained. It has a function to generate. The generated frame storage position information has a function of notifying the MC unit 2302 and the interframe subband decoding management unit 2307.

15 and 16 are flowcharts showing the operation of an application example of the decoding apparatus shown in FIG. The operation of the decoding apparatus shown in FIG. 23 will be described below with reference to FIGS.
First, the DEMUX unit 2002 acquires an input bit stream (step S601). Thereafter, DEMUX is performed on the acquired bit stream (step S602), thereby generating a subband coefficient information bit string, a motion vector information bit string, an order information bit string, and various kinds of control information necessary for decoding. The coefficient information bit sequence, motion vector information bit sequence, and order information bit sequence are notified to the entropy decoding unit 2003 described later.
The entropy decoding unit 2003 acquires the subband coefficient information bit sequence, the motion vector information bit sequence, and the order information bit sequence notified from the DEMUX unit 2002, and performs entropy decoding on each information (step S603). The subband coefficient information generated by entropy decoding is notified to the inverse quantization unit 2004, and the generated motion vector information is notified to the MC unit 2302 and the inverse MC unit 2305. Further, the generated order information is notified to the order information analysis unit 2308.
Thereafter, the inverse quantization unit 2004 acquires subband coefficient information from the entropy decoding unit 2003, and performs inverse quantization (step S604). The subband coefficient information after the inverse quantization is notified to the input management unit 2301.

The input management unit 2301 acquires subband coefficient information after dequantization from the dequantization unit 2004 and manages it as an L component frame including L component information and an H component frame including H component information after subband division. . In addition, the input management unit 2301 accumulates information for configuring one GOP by acquiring, from the inverse quantization unit 2004, subband coefficient information that can configure the number of frames necessary to reconstruct the GOP ( Step S605). Thereafter, the MC unit 2302, the interframe subband decoding unit 2303, the inverse MC unit 2305, and the pixel interpolation unit 2306 are notified of the frame to be processed by the inverse MCTF (step S606).
The order information analysis unit 3208 acquires the order information header from the entropy decoding unit 2003 (step S501). Alternatively, when the order information header cannot be acquired, the order information added for each H component frame may be acquired. By performing the analysis as shown in FIG. 13 from this order information, the frame storage position specifying information for specifying the order relation and the frame storage position of the L component frames after decoding in one GOP for each octave division is obtained. Generate. The generated frame storage location information is notified to the MC unit 2302 and the interframe subband decoding management unit 2307.
The input management unit 2301 sets an information group to be subjected to inverse MCTF in order to perform basic inverse MCTF processing (step S502). Here, the L component frame and the H component frame to be subjected to the basic inverse MCTF process are specified and set.

  The MC unit 2302 acquires frame storage position specifying information from the order information analysis unit 2308. Based on the acquired frame storage position specifying information, the L component frame storing the subband coefficient information of the L component after subband division to be decoded from the input management unit 2301 and the subband coefficient information of the H component are stored. The stored H component frame is acquired. Here, it is specified from the acquired frame storage position information whether the frame to be processed currently has been subjected to forward ME / MC or reverse ME / MC. In the case of the forward direction, the L component frame corresponds to the position on the 2N frame side after decoding, and the H component frame corresponds to the position on the 2N + 1 frame side after decoding. In the reverse direction, the L component frame corresponds to the position on the 2N + 1 frame side after decoding, and the H component frame corresponds to the position on the 2N frame side after decoding. In the forward direction, it is assumed that the 2N frame is a frame on the encoding side where a motion vector search is performed when performing motion estimation. Further, it is assumed that the 2N + 1 frame is a frame serving as a reference when performing motion estimation on the encoding side. If it is in the reverse direction, the 2N + 1 frame is assumed to be a frame on the side where a motion vector is searched when performing motion estimation on the encoding side. Further, it is assumed that the 2N frame is a frame serving as a reference when performing motion estimation on the encoding side. The MC unit 2302 usually has a predetermined range for extending an area outside the frame in order to allow an out-of-area reference to an L component frame corresponding to 2N frames in the forward direction and 2N + 1 frames in the reverse direction. Perform padding. Here, when out-of-region reference is prohibited on the encoding side, out-of-region reference may be prohibited in the MC unit 2302. When out-of-region reference is prohibited, it is not always necessary to perform a predetermined padding process. The MC unit 2302 acquires motion vector information from the entropy decoding unit 2003. The MC unit 2302 generates a motion compensation frame (hereinafter referred to as an MC frame) based on the L component frame after padding, the shape information of the corresponding region, and the motion vector information (step S503). Here, the shape information of the reference region used when performing MC is 2N + 1 frames in the forward direction and 2N frames in the reverse direction in the encoding side motion estimation unit (hereinafter referred to as ME unit). Since the area is divided into a plurality of reference areas so as not to overlap each other based on a predetermined area dividing method, it is necessary to acquire shape information of the reference area. However, in order to simplify the following description, it is assumed that the shape of the reference region is a predetermined 4 × 4 block shape determined in advance by the encoding device and the decoding device, and the shape information of the reference region is exchanged. Need not be mentioned. The MC unit 2302 notifies the inter-frame subband decoding unit 2303 of the MC frame generated based on the L component frame.

The inter-frame subband decoding unit 2303 performs MC on the H component frame corresponding to the 2N + 1 frame in the forward direction and the 2N frame in the reverse direction from the input management unit 2301, and the MC unit 2302 performs the L component frame in the reverse direction. Get the MC frame. The obtained MC frame and H component frame are subjected to inter-frame subband decoding by predetermined subband synthesis (step S504), and the generated two L component frames different in the time direction are associated with the corresponding 2N frame, 2N + 1. Store in frame. 2N frames in the forward direction and 2N + 1 frames in the reverse direction are notified to the reverse MC unit 2305, and 2N + 1 frames in the forward direction and 2N frames in the reverse direction are notified to the interframe subband decoding management unit 2307.
The inverse MC unit 2305 acquires motion vector information from the entropy decoding unit 2003, and 2N frames in the forward direction and 2N + 1 frames in the reverse direction from the interframe subband decoding unit 2303. Based on the acquired motion vector information and shape information of the reference region, an overlapping order control unit 2304 that controls the 2N frames in the forward direction after band synthesis and 2N + 1 frames in the reverse direction based on the overlapping order of each reference region. In accordance with the control, the obtained motion vector information is applied in the reverse direction to perform reverse MC (step S505), and 2N frames are generated in the forward direction after the reverse MC and 2N + 1 frames are generated in the reverse direction. The pixel interpolation unit 2306 is notified of 2N frames in the forward direction after the generated reverse MC and 2N + 1 frames in the reverse direction. For a region where pixels are not filled by inverse MC, the pixel interpolation unit 2306 may be notified of pixel interpolation region information. Here, in order to simplify the processing, the pixel interpolation area information stores a value indicating the interpolation area in the 2N frame in the forward direction after reverse MC and 2N + 1 frame in the reverse direction, for example, a value of “−1”. The pixel interpolation unit 2306 acquires pixel interpolation area information based on this information. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information.

The pixel interpolation unit 2306 is an L component frame corresponding to 2N frames in the forward direction before decoding from the input management unit 2301 and 2N + 1 frames in the reverse direction, 2N frames in the forward direction after reverse MC from the reverse MC unit 2305, and in the reverse direction. Obtain 2N + 1 frames. In the 2N frame in the forward direction after the acquired reverse MC and in the 2N + 1 frame in the reverse direction, pixel interpolation is performed based on the pixel interpolation region information that is information related to the region in which the pixel is not filled with the reverse MC (step S506). The determination of the area where the pixel is not filled may have a function of acquiring pixel interpolation area information from the inverse MC unit 2305. In order to simplify the processing, the pixel interpolation area information stores a value indicating the interpolation area, for example, a value of “−1” in 2N frames in the forward direction after reverse MC and 2N + 1 frames in the reverse direction. It is assumed that pixel interpolation area information is acquired based on this information. In the following description, it is not necessary to refer to the exchange of pixel interpolation area information. The inter-frame subband decoding management unit 2307 is notified of 2N frames in the forward direction after pixel interpolation and 2N + 1 frames in the reverse direction.
The inter-frame subband decoding management unit 2307 obtains 2N frames in the forward direction and 2N + 1 frames in the forward direction that have undergone pixel interpolation from the pixel interpolation unit 2306, and 2N + 1 in the forward direction from the inter-frame subband decoding unit 2303. In the reverse direction, 2N frame is acquired.

Thereafter, it is determined whether there is a frame to be processed in the current octave division number (step S507). If there is a frame to be processed, N is updated, the next group of frames to be processed in the current octave division number is set, and the process is repeated. If there is no frame to be processed, it is determined from the position of the acquired frame whether the number of octave divisions is a predetermined number, and it is determined whether to continue band synthesis (step S608). If it is necessary to perform band synthesis on the input component, the input management unit 2301 is notified to treat the processed L component frame group as an input frame in the GOP again, and the processing from step S606 is repeated. When it is not necessary to perform band synthesis, a series of decoding processing is completed by outputting a moving image frame as a decoding result.
By providing a decoding apparatus having the configuration as shown in FIG. 23 that operates based on the above-described method, it is possible to eliminate the dependency relationship between the coefficients during decoding and to perform efficient decoding. It becomes.

The present invention includes a program for causing the central processing control device 2403, which is a computer shown in FIG. 24, to realize the functions of the application example of the decoding device. This program may be read from a recording medium and taken into the central processing control device 2403, or may be transmitted via a communication network and taken into the central processing control device 2403.
In addition, the subband coding operation has been described by a method realized by the conventionally used addition method and subtraction method. The calculation method subtracts the motion compensated reference image from the target image, and then adds the updated target image to the image obtained by the subtraction so that the low frequency component image (L image) and the high frequency component image are added. It can also be realized by a calculation method that obtains (H image). Even in such a case, the arrangement of the reference image and the target image is set to the forward direction or the reverse direction, and image information including many elements (textures) that are difficult to predict motion is arranged toward the reference image. The amount can be reduced and a large amount of image information can be included in the L image.

  As described above, the moving picture encoding apparatus shown in the present embodiment includes the input management unit 1702 that outputs moving pictures in parallel, and one frame of the moving pictures output in parallel as a first reference picture, Forward motion estimation that estimates motion of the first target image from the first reference image and obtains forward motion vector information when one frame immediately after the first reference image is the first target image When the first reference image is used as the second target image, and the first target image is used as the second reference image, the second reference is replaced with the unit 1703, the first reference image, and the first target image. A backward motion estimation unit 1704 that estimates motion of the second target image from an image to obtain backward motion vector information, and the first reference image based on the forward motion vector information. And a forward motion compensation unit 1705 for obtaining a forward motion compensation image for compensating for motion between the first reference image and the first target image, and the second reference image and the second target based on the backward motion vector information. A backward motion compensation unit 1706 for obtaining a backward motion compensation image for compensating for the motion of the image, and a similarity between the forward motion compensation image obtained by the forward motion compensation unit and the first target image, and The similarity determination unit 1707 compares the similarity between the backward direction motion compensated image obtained by the backward direction motion compensation unit and the second target image, and determines which is the higher degree of similarity. A switching unit 1709 that switches to the forward motion compensation unit or the backward motion compensation unit that is determined to be high by the similarity determination unit, and the switching unit The forward motion compensation image output from the forward motion compensation unit or the backward motion compensation unit and the first target image or the backward motion compensation image and the second target image are added to the moving image. While obtaining a compressed image at a lower rate than the image, the first reference image and the first target image or the second reference image are subtracted from the compressed image by subtracting the first or second reference image from the compressed image. And an inter-frame subband encoding unit 1711 for obtaining a restoration auxiliary image to be restored to the second target image, so that when performing motion compensation time direction filter processing, past frame images and future frame images Of the low-frequency and high-frequency component images converted to a low-frequency frame rate, the low-frequency component image is visually necessary. Therefore, it is possible to realize a moving picture encoding apparatus that can obtain a subband encoded output with high compression encoding efficiency in which an image component to be left is left and frame intervals are unequal.

Also, since important frames can be selected from the results of forward / reverse ME / MC when performing MCTF, when trying to achieve scalability in the time direction using MCTF, which was a problem in the conventional method, As a result of the decimation at equal intervals generated in the process of inter-frame subband encoding of MCTF, encoding can be performed so that an important frame as a motion is sufficiently reflected in the generated L component frame. Encoding efficiency can also be performed with high efficiency by fully utilizing the correlation between frames in the GOP.
Furthermore, by making the frame interval when the frame rate is changed by scalability in the time direction unequal, it is possible to obtain a moving image reproduction that is more natural than a moving image having the same frame rate by the conventional MCTF.
In order to multiplex the arrangement order of frame images and motion vector information used for encoding by the moving image encoding apparatus to generate an encoded signal, a low frame rate moving image including image information particularly important for viewing is reproduced. A moving image reproducing method can be provided and a moving image reproducing device can be realized. It is possible to provide a computer program for a moving image reproduction apparatus that is driven and operated by a computer.

  The present invention can be applied to a moving picture encoding apparatus that performs subband division between frames in the time direction of a moving picture and a decoding apparatus thereof.

It is a block diagram which shows the structural example of the moving image encoder which concerns on implementation of this invention. It is a figure which shows the flow of an encoding of the moving image encoder which concerns on implementation of this invention. It is a figure which shows the encoding method of the moving image encoder which concerns on implementation of this invention. It is explanatory drawing of the encoding method of the moving image encoder which concerns on implementation of this invention. It is explanatory drawing of the encoding method of the moving image encoder which concerns on implementation of this invention. It is a figure for showing the process of the subband decomposition | disassembly of MCTF encoding of this invention. It is a figure for showing the process of the subband decomposition | disassembly of MCTF encoding of this invention. It is a figure for showing the process of the subband decomposition | disassembly of MCTF encoding of this invention. It is a flowchart for showing the process of basic MCTF processing of the present invention. It is a flowchart for showing the process of MCTF processing of the present invention. In MCTF decoding of this invention, it is a figure for showing the process in case the frame position at the time of decoding is not decided. It is a figure for showing the procedure at the time of comprising an order information header in MCTF encoding and decoding of the present invention. In MCTF decoding of this invention, it is a figure for showing the process of producing | generating the information for determining the frame position at the time of decoding from an order information header. In MCTF decoding of this invention, it is a figure for showing the process process at the time of determining the frame position at the time of decoding from an order information header. It is a flowchart for showing the process of basic inverse MCTF processing of the present invention. It is a flowchart for showing the process of inverse MCTF processing of the present invention. It is a block diagram for showing the MCTF encoding apparatus structure of this invention. It is a block diagram for showing the structure of the application example in the MCTF encoding apparatus of this invention. It is a functional block diagram showing the information processing apparatus which performs the MCTF encoding program of this invention. It is a block diagram for showing the structure of the conventional MCTF decoding apparatus. It is a block diagram for showing the structure of the MCTF decoding apparatus of this invention. It is a functional block diagram showing the information processing apparatus which performs the MCTF decoding program of this invention. It is a block diagram for showing the structure of the application example of the MCTF decoding apparatus of this invention. It is a functional block diagram showing the information processing apparatus which performs the MCTF decoding program of this invention. It is a flowchart for showing the process of basic MCTF processing of the present invention. It is a flowchart showing operation | movement of the MCTF decoding apparatus using the conventional general reverse MCTF. It is a flowchart showing operation | movement of the MCTF decoding apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Moving image encoder 12 Input management part 13 Forward direction motion estimation part 14 Reverse direction motion estimation part 15 Forward direction motion compensation part 16 Reverse direction motion compensation part 17 Similarity determination part 18, 19 Switch 21 Interband subband encoding Unit 25 quantization unit 26 entropy encoding unit 27 multiplexing unit 101 moving image frame 102 input management unit 103 MCTF unit 104 quantization unit 105 entropy encoding unit 106 MUX unit 107 bit stream 1700 MCTF unit 1701 moving image frame 1702 input management Unit 1703 Forward direction ME unit 1704 Reverse direction ME unit 1705 Forward direction MC unit 1706 Reverse direction MC unit 1707 Similarity determination unit 1708 Switch 1
1709 Switch 2
1710 Control unit 1711 Interframe subband coding unit 1712 Inverse MC unit 1713 Pixel interpolation unit 1714 Interframe subband coding management unit 1715 Quantization unit 1716 Entropy coding unit 1717 MUX unit 1718 Bit stream 1801 Overlap identification unit 1802 Error correction Unit 1900 Information processing device 1901 Input device 1902 Output device 1903 Central processing control device 1904 External storage device 1905 Temporary storage device 1906 Communication device 2000 Inverse MCTF unit 2001 Bit stream 2002 DEMUX unit 2003 Entropy decoding unit 2004 Inverse quantization unit 2005 Input management Unit 2006 correspondence relation specifying unit 2007 first inter-frame decoding unit 2008 second inter-frame decoding unit 2009 pixel interpolation unit 2010 inter-frame decoding management unit 2011 Image frame 2101 Order information analysis unit 2200 Information processing device 2201 Input device 2202 Output device 2203 Central processing control device 2204 External storage device 2205 Temporary storage device 2206 Communication device 2300 Inverse MCTF unit 2301 Input management unit 2302 MC unit 2303 Inter-frame subband decoding Conversion unit 2304 overlap order control unit 2305 inverse MC unit 2306 pixel interpolation unit 2307 inter-frame subband decoding management unit 2308 order information analysis unit 2400 information processing device 2401 input device 2402 output device 2403 central processing control device 2404 external storage device 2405 temporarily Storage device 2406 communication device

Claims (6)

When a frame having an input moving image is a forward reference image and one frame different from the forward reference image is a forward target image, movement of the forward target image from the forward reference image A forward motion estimator that estimates forward motion vector information and
When the forward reference image is a backward target image and the forward target image is a backward reference image, the movement of the backward target image is estimated from the backward reference image. A backward motion estimation unit for acquiring backward motion vector information;
A forward motion compensation unit that obtains a forward motion compensated image for compensating for motion between the forward reference image and the forward target image based on the forward motion vector information;
A backward motion compensation unit for obtaining a backward motion compensated image for compensating for motion between the backward reference image and the backward target image, based on the backward motion vector information;
The degree of similarity between the forward direction motion compensated image and the forward direction target image and the degree of similarity between the backward direction motion compensated image and the backward target image are determined, and which direction is selected A degree-of-similarity determination unit that generates order information for indicating whether or not
A moving picture encoding apparatus having: The moving image encoding apparatus according to claim 1,
further,
A selection unit that selects the reference image in the forward direction or the backward direction with the high similarity based on the order information, the target image, the motion compensation image, and the motion vector information;
The selected motion compensated image and the target image are added to lower the low frequency component image at a lower rate than the input moving image, and the input moving image is subtracted from the selected reference image An inter-frame subband encoding unit that obtains a high-frequency component image at a higher rate than the image;
A moving picture encoding apparatus having: The moving picture encoding apparatus according to claim 2, wherein
further,
An entropy encoding unit that entropy-encodes the low-frequency component image, the high-frequency component image, the order information, and the selected motion vector information;
A moving picture encoding apparatus having: When a frame having an input moving image is a forward reference image and one frame different from the forward reference image is a forward target image, movement of the forward target image from the forward reference image Estimating the motion vector information in the forward direction by estimating
When the forward reference image is a backward target image and the forward target image is a backward reference image, the motion of the backward target image is estimated from the backward reference image. Obtaining motion vector information in the reverse direction;
Based on the forward motion vector information, a forward motion compensated image for compensating for motion between the forward reference image and the forward target image is obtained, and the backward motion vector information is used as the backward motion vector information. Obtaining a backward motion compensated image for compensating for the motion of the backward reference image and the backward target image,
The degree of similarity between the forward direction motion compensated image and the forward direction target image and the degree of similarity between the backward direction motion compensated image and the backward target image are determined, and which direction is selected Generating order information to indicate whether or not
A video encoding method comprising: The moving picture encoding method according to claim 4, wherein
further,
A selection step of selecting the reference image with high similarity or the forward direction based on the order information, the target image, the motion compensation image, and the motion vector information;
The selected motion compensated image and the target image are added to lower the low frequency component image at a lower rate than the input moving image, and the input moving image is subtracted from the selected reference image An inter-frame subband encoding step to obtain a high-frequency component image at a higher rate than the image;
A video encoding method comprising: The moving picture encoding method according to claim 5, wherein
further,
An entropy encoding step for entropy encoding the low-frequency component image, the high-frequency component image, the order information, and the selected motion vector information;
A video encoding method comprising:

Images