JPH11155147A - Image reproduction method, image coder, and image coding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a image reproduction method in which the arithmetic operation of each block in a frame is devised to properly omit it, so as to reduce the arithmetic amount and coding and decoding of a bit stream executed at high speed by the communication media use coding algorithm, and to provided the image coding method and the image coder. SOLUTION: First, all blocks that are not referenced from any block in a Frame C are detected from among blocks of a Frame B (figure shows only a block b1 ), based on motion vectors mv(C0 )-mv(C3 ) resulting from decoding all motion vectors that reference the Frame B to be skipped, an IDCT arithmetic operation to the detected blocks is omitted so as to reduce the arithmetic amount. In an example shown in the figure, the IDCT arithmetic operation required for four blocks in the Frame B and four blocks in the Frame C substantially is reduced to 7 blocks by omitting the IDCT arithmetic operation with respect to the block b1 .

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image reproducing method, an image encoding apparatus, and an image encoding method. More specifically, the present invention relates to an image reproducing method capable of performing high-speed skip reproduction of image data and a high-speed image data. TECHNICAL FIELD The present invention relates to an image encoding device and an image encoding method capable of encoding an image.

[0002]

2. Description of the Related Art Recently, international standardization of multimedia-related image coding has been rapidly advanced. For example, JPEG (Joint Photographic Exper
ts Group), H.264, which is an encoding standard for video communication media. 263, M which is an encoding standard for moving image storage media
PEG (Moving Picture Experts Group) and the like. In each of these encoding methods, an encoding algorithm that realizes a bit rate (transmission rate) suited to each purpose is adopted.

For example, JPEG uses Huffman coding and D
Color data thinning by CT (Discrete Cosine Transform) is used. On the other hand, H. 263
In recent years, moving picture compression coding schemes, such as MPEG and MPEG, use DCT as in JPEG to reduce the amount of spatial information and use MC (Motion) to reduce the amount of temporal information.
Most adopt a MC + DCT hybrid method using Compensation (motion compensation).

[0004] DCT is a type of Fourier transform. By transforming a two-dimensional image into a two-dimensional frequency, DCT can be separated into a low-frequency component that is easily seen by human eyes and a high-frequency component that is difficult to distinguish. Can be ubiquitous. 8x8
To the equation (1), and its inverse transform ID
Equation (2) shows the CT conversion equation.

[0005]

(Equation 1)

[0006]

(Equation 2)

Here, C (u) = 1 / √2 (u = 0) C (u) = 1 (u ≠ 0) C (v) = 1 / √2 (v = 0) C (v) = 1 (v ≠ 0) IDCT (i, j): pixel value DCT (u, v): DCT coefficient

The MC is an inter-frame prediction in which an input image is divided into blocks of a certain size, a motion vector is prepared for each block, and a motion vector is predicted from a previously reproduced image (predicted image) by a motion vector. Inter-frame prediction encoding is a method of encoding the difference between an input image and a previous reproduced image.
A new reproduced image is added to the previous reproduced image. Most regions of the moving image are similar to the previous image, and the information amount of the inter-frame difference is generally smaller than the information amount of the original input image. Therefore, the inter-frame prediction improves the coding efficiency.

Hereinafter, the MC + DCT hybrid system will be briefly described. FIG. 15 is a diagram for explaining the encoding concept of the MC + DCT method, and FIG.
It is a figure which shows the encoding and decoding processing of DCT hybrid system.

In FIG. 15, Curr is a frame being encoded, and Pred is a frame encoded immediately before Curr. Usually, each frame is, for example, 16
It is divided into × 16 blocks. For a block X having a Curr, detecting a block Y that is most similar to the block X from Pred and calculating a motion vector is called ME (Motion Estimation). Then, the MC predicts the block X with reference to the block Y detected by the ME. That is,
In a normal case, an encoding process is performed by calculating an error (X-Y) between a frame image being encoded and a predicted image motion-compensated by a motion vector, and subjecting the error to DCT.

Next, the encoding process and the decoding process in FIG. 16 will be described.

First, prediction is used (inter) or not (in) for an input block to be encoded.
tra). Hereinafter, each case will be described (in the case of intra, the switch 30b is turned off on the intra side and the switch 30j is turned off, and
In the case of r, the switch 30b is turned on the inter side and the switch 30j is turned on). And in the case of intra,
The DCT of the input block is calculated (DCT 30c), and the calculated DCT coefficient is quantized with an arbitrary quantization width (Q30
d) Then, after the image data quantized together with the quantization width is encoded (encoding 30e), the image data is transferred as an encoded block to a communication path or the like. Further, as local decoding for restoring an image to be used as an original image for the next prediction, the quantized block is inversely quantized (IQ30f), and further, DC
The IDCT of the T coefficient is calculated (IDCT 30g). Then, it is stored in the predictor 30i via the adder 30h.

In the case of inter, a difference between the input block and a predicted image generated by using motion compensation from a previously coded image output from the predictor is calculated (subtracter 30a). The DCT of this difference is calculated (DCT
30c) Further, after quantizing the DCT coefficients to encode the quantization width and the quantized difference image together with the motion vector (decoding 30e), the image is transferred to a communication path or the like as an encoded block. Further, the quantized difference image is inversely quantized (IQ
30f), calculate the IDCT of the DCT coefficient (IDCT3
0g), by adding the predicted image used in the difference calculation at the input stage (adder 30h), local decoding for restoring the image used for the next prediction is performed, and the predictor 3
0i.

The decoding is the reverse of the above coding. That is, by decoding the encoded block (decoding 30k), the block is divided into a motion vector (inter only), a quantization width, and a quantization image. Then, the quantized data is dequantized (IQ30l) and inverse DCT (IQ
DCT30m) to obtain intra images and i
An nter difference image is obtained. If the image is an intra image, there is no difference, and if the image is output as it is, the image is reproduced. int
Since the er image is difference data, the predictor 30o reproduces a predicted image from the already reproduced image and the motion vector, and adds the difference data to the predicted image (adder 30n) to reproduce the image.

Next, even when the same moving picture is coded, H.264 is used. 26
3 is intended to be used for videoconferencing, videophone, etc., so it is an essential condition that it can be encoded (encoded) in real time. MPEG is intended to reproduce high-efficiency, high-quality images. Therefore, FIG. 1
As shown in FIG. 7 and FIG. 18, the structures of the bit streams are different from each other, and their decoding (decoding) methods are also different.

In the following, H. The difference between the H.263 and MPEG decoding methods will be described. FIG. 263
FIG. 18 is a diagram showing the bit stream structure of MPEG in a frame unit, and FIG. 18 is a diagram showing the bit stream structure of MPEG in a frame unit.

In FIGS. 17 and 18, "I"
Indicates a frame composed of an I-picture (Intra-coded picture).
“P” indicates a frame composed of P pictures (Predictive-coded pictures), and “B” indicates a B picture (Bidirectionally coded picture).
ictive-coded picture: a bidirectional predictive coded image), and the characteristics of each picture (one image constituting a moving image) are as follows.

I picture: When encoded, only closed information is used in one picture. In other words, when decoding, an image can be reconstructed using only the information of the I picture itself. This coding scheme is generally inefficient,
If this is inserted everywhere, random access and high-speed reproduction can be performed. P picture: Predicted image (image used as a reference for taking a difference)
, Use an I-picture or P-picture that is located earlier in time and has already been decoded. B-picture: made from an I-picture or P-picture that is located earlier in time and has already been decoded as a predicted picture, an I-picture or P-picture that is located later in time and has already been decoded, and both. Three types of interpolated images are used.

In general, an I-frame requires several times the amount of information as a P-frame, so it is difficult to adapt to a communication method having a constant communication capacity per time, and to improve coding efficiency. Therefore, the encoding algorithm for communication media is characterized in that, as shown in FIG. 17, there is no I-frame in the middle frame after the first frame.

Referring to FIG. In the decoding method of H.263, the I frame is decoded at time t1, and
At 2, the P frame is decoded based on the I frame decoded at time t1. Thereafter, generally, at time tn, the P frame is decoded based on the frame decoded at time t (n-1).

On the other hand, the encoding algorithm for storage media such as MPEG is characterized in that an I-frame also exists in an intermediate frame as shown in FIG. High-speed skip reproduction can be performed by a method such as decoding only a frame.

Referring to FIG. 18, in the MPEG decoding method, an I frame is decoded at time t1, and a B frame subsequently input at times t2 and t3 is not decoded at time t4 at time t1. I
The P frame is decoded based on the frame. Next, the B frame is decoded based on the previously decoded I frame and P frame. Further, at time t
The B frame input at 5, t6 is not decoded,
At time t7, the P frame is decoded based on the P frame decoded at time t4. Since then
Table 1 below shows the correspondence between the decoded frame and the reference frame when decoding is performed in the same manner.

[0023]

[Table 1]

In Table 1, in the left column, frames to be decoded are numbered in the order of input, and in the right column, reference frames for decoding the frames in the left column are shown. .

According to Table 1, at the time of high-speed skip reproduction, if only I frames that do not require a reference frame (indicated by "none" in the table) are to be reproduced, B input at time t2 (2) The frame from the frame to the B (9) frame input at time t9 can be skipped.

Also, considering that the image quality is somewhat improved compared to the reproduced image of only the I frame, and only the I frame and the P frame are reproduced, in the example of Table 1, two out of three frames are skipped. It becomes possible.

[0027]

SUMMARY OF THE INVENTION However, the conventional H.264 standard has been described. In the coding algorithm for communication media such as H.263, there is only a method of decoding one frame at a time as described above. For example, a bit stream coded by the coding algorithm for communication media is stored in a storage device connected to a personal computer or the like. In the case of storing and playing back, there is a problem that high-speed skip playback cannot be performed as in the above-described encoding for storage media such as MPEG.

Further, such a problem also leads to a problem that when the power of a CPU for decoding is not sufficient, synchronization with audio data having a small amount of information and a high bit rate as compared with an image cannot be performed. .

It is known that DCT, ME, and IDCT require an extremely large amount of calculation in coding by the MC + DCT hybrid method. However, in the conventional method as described above, basically, When processing one frame, it is necessary to perform both DCT and IDCT calculations for each block, which requires an enormous amount of calculation.

Therefore, it is difficult to speed up the encoding process, and a high-speed processor is required to perform the encoding in real time, which causes a problem that the cost is increased.

Accordingly, an object of the present invention is to reduce the amount of calculation by devising the calculation for each block in a frame and arbitrarily omitting the calculation, and to speed up the coding and decoding of a bit stream by a communication medium coding algorithm. It is an object of the present invention to provide an image reproduction method, an image encoding device, and an image encoding method that can be performed in a computer.

[0032]

According to the first aspect of the present invention,
An image reproducing method for decoding and reproducing image data encoded by an image encoding method using a motion vector for motion-compensating each block of a current frame with reference to a block constituting a frame located temporally ahead In skipping and reproducing an arbitrary number of frames, the motion vector is previously decoded and stored in a storage unit for all blocks to be skipped and all the blocks in the skipped frame, and stored in the storage unit. Based on the obtained motion vector, the reference relation of each block is sequentially and retrospectively calculated from the skipped frame toward the temporally previous frame to detect a skipped block having the motion vector. And performing a predetermined decoding process on the detected block to reproduce an image.

According to the image reproducing method of the first aspect of the present invention, a predetermined decoding process can be performed only on blocks necessary for decoding a frame after skipping. , And the high-speed skip reproduction can be realized. This also solves the problem that synchronization with audio data cannot be achieved due to insufficient processing capacity of the CPU.

According to the second aspect of the present invention, the image is encoded by an image encoding method using a motion vector for motion-compensating each block of the current frame with reference to a block constituting a frame located earlier in time. In an image reproducing method for decoding and reproducing image data, when an arbitrary number of frames are skipped and reproduced, the motion vectors are previously decoded for all blocks to be skipped and all the blocks of the skipped frames. Based on the motion vector stored in the storage means and based on the motion vector stored in the storage means, reference is not made to a block at a different position in a frame located earlier in time, and different blocks in a frame located later in time are referred to. A block that is not referred to from the block at the position is detected from all the frames to be skipped, and the detected block is At the same position in the frame, a block sequence formed by being present over a plurality of frames is further detected, and a predetermined decoding process for all blocks over a plurality of frames constituting the block sequence is collectively executed at once. It is characterized by reproducing images.

According to the image reproducing method of the second aspect of the present invention, the blocks at different positions in the preceding and succeeding frames are referred to,
Since the decoding process can be executed at once for a plurality of blocks over a plurality of frames that are not related to each other and refer only to the block at the same position in the frame, the processing amount at the time of skip reproduction is reduced, High-speed skip reproduction can be realized. This also allows the CP
It is also possible to solve the problem that synchronization with audio data cannot be achieved due to insufficient processing capacity of U.

According to a third aspect of the present invention, a motion vector for calculating a motion vector for motion-compensating each block of a current frame of an input image with reference to a block constituting a frame positioned temporally before the input image. Calculating means, coding means for converting image data into orthogonal transform coefficients in block units by a predetermined orthogonal transform operation and coding the image data, and converting the image data in block units coded by the coding means into the predetermined A local decoding means for locally decoding by an inverse transformation operation of the orthogonal transformation operation, wherein the motion vector calculated by the motion vector calculation means for each block of the current frame is temporally A determining unit that determines whether or not the block refers to a block at the same position in a frame located before; When it is determined that the motion vector does not refer to a block at the same position in a temporally preceding frame, the reference block in the temporally preceding frame is determined as After local decoding by the local decoding unit to reconstruct the block of the reference image, and calculating the difference between the block of the input image and the block of the reference destination, the encoding unit performs the predetermined orthogonal transform operation. If the motion vector is determined by the determination means to refer to a block at the same position in a frame located earlier in time, The block of the input image is changed by the encoding unit without performing the local decoding process by the local decoding unit on the reference block in the frame located before. The orthogonal transform coefficient, and encoding the image data by calculating the difference between the orthogonal transform coefficient of the reference block already calculated when encoding the frame located earlier in time. Features.

According to the image coding apparatus of the present invention, the motion vector calculating means refers to the block constituting the frame located temporally before the input image and refers to the block of the current frame of the input image. A motion vector for compensating the motion of each block is calculated, and the motion vector calculated by the motion vector calculation means for each block of the current frame is determined by the discriminating means. If it is determined that the motion vector does not refer to a block at the same position in a frame located earlier in time based on the result of determining whether or not the The reference block in the frame located immediately ahead is locally decoded by the local decoding means to reconstruct the block of the reference image, and the input image After calculating the difference between the block and the reference block, the encoding unit performs the predetermined orthogonal transformation operation to encode the image data, and the frame in which the motion vector is located earlier in time. If it is determined that it refers to the block at the same position in the reference block in the frame located earlier in time, without performing the local decoding process by the local decoding means, Calculate the difference between the orthogonal transform coefficient obtained by transforming the block of the input image by the encoding means and the orthogonal transform coefficient of the reference block already calculated when encoding the temporally preceding frame. To encode the image data.

According to a fourth aspect of the present invention, the coding of image data is performed by an image coding method using a motion vector for motion-compensating each block of the current frame with reference to a block constituting a frame located earlier in time. In the image coding method of performing the encoding, it is determined whether the motion vector calculated for each block of the current frame refers to the block at the same position in the frame located earlier in time, When it is determined that the motion vector does not refer to a block at the same position in a temporally located frame, the reference block in the temporally located frame is locally decoded. After reconstructing the block of the reference image and calculating the difference between the block of the input image and the block of the reference image, a predetermined orthogonal transformation operation is performed to perform Encoding, and when it is determined that the motion vector refers to a block at the same position in the temporally preceding frame, a reference destination in the temporally preceding frame is determined. Without performing the local decoding process on the block, the orthogonal transform coefficient calculated by performing the predetermined orthogonal transform operation on the block of the input image, and already calculated when encoding the frame located earlier in time. And encoding the image data by calculating a difference between the orthogonal transform coefficient of the reference block and the orthogonal transform coefficient.

According to the image encoding apparatus of the invention described in claim 3 or the image encoding method of the invention described in claim 4,
Based on the motion vector calculation result, for a block satisfying the condition that the motion vector refers to the block at the same position in the previous frame, for example, the linearity of a predetermined orthogonal transform operation such as DCT is determined. The difference between the orthogonal transform coefficient (for example, DCT coefficient) of the input image and the orthogonal transform coefficient of the predicted image can be calculated instead of the difference between the input image and the predicted image. Appear, the calculation amount is large, for example, I
The calculation of the inverse transform of the predetermined orthogonal transform operation such as DCT can be omitted, and the image encoding process can be speeded up. In addition, this makes it possible to perform image encoding processing at a practical speed with an inexpensive processor without using a high-performance processor or the like,
The manufacturing cost of the image encoding device can be reduced.

[0040]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an image reproducing method and an image encoding method according to the present invention will be described in detail with reference to FIGS.

(First Embodiment) Hereinafter, an image reproducing apparatus 1 to which an image reproducing method according to the present invention is applied will be described in detail with reference to FIGS. In the present embodiment,
As described in H. Image data encoded by a communication medium encoding algorithm such as H.263 in a storage device connected to, for example, a personal computer or the like,
A description will be given of a case where the amount of calculation is reduced by omitting the IDCT calculation, the MC calculation, and the like for blocks not referenced from the frame to be decoded during skip reproduction when reproducing the stored image data.

First, the configuration will be described. In FIG. 1, the image reproducing apparatus 1 includes a control unit 2, an input device 3, a RAM 4, a display device 5, a storage device 6, and a storage medium 7, and each unit is connected by a bus 8.

The control unit 2 stores, in the RAM 4, an application program specified from various application programs corresponding to the image reproducing device 1 stored in the storage device 6 and various instructions or data input from the input device 3. In accordance with the input instruction and the input data, various processes are executed in accordance with the application program stored in the RAM 4, the processing results are stored in the work memory 4 a in the RAM 4, and the display device 5 indicate. Then, the processing result stored in the work memory 4a is stored in a storage destination in the storage device 6 which is instructed to be input from the input device 3.

The control unit 2 executes a process of decoding encoded image data input from the storage device 6 or from an external device (not shown) via a communication line.

For example, H. In H.263 and other standards, inter-frame prediction by MC, DCT coefficient (frequency data) conversion of pixel values (image data) by DCT operation, quantization, and the like are performed at the time of encoding. A bit stream is allocated with the number of bits.

Therefore, the control unit 2 reads the bit stream for every predetermined number of bits in accordance with the standard and decodes the data into various data such as DCT coefficients and motion vectors.
Further, each data is inversely quantized with a predetermined inverse quantization width according to the standard and information in the bit stream.

Then, by performing the IDCT operation shown in the above equation (2) on the dequantized data,
The inversely quantized DCT coefficients (frequency data) are converted into pixel values (image data), and if necessary, are added to prediction data and output to the display device 5.

In the n-frame skipping process (see FIG. 4) described later, the control unit 2 first sets Fr = 0 (fr: frame number) of the frame before skipping.
After decoding ame0, the skipped frame f
Decode all motion vector data up to Frame (n + 1) of r = n + 1, and then decode Frame (n +
Since 1) needs to decode all blocks,
BKn + for all blocks in Frame (n + 1)
By setting 1, j = 1 (j: block number), the IDC
Performing the T operation is set, and then processing for detecting a block that needs an IDCT is performed for all frames.

That is, the control unit 2 sets Frame (n)
Are initialized with BKn, j = 0 for all blocks, and Frame (n + 1) is referenced from each block of Frame (n + 1) since all blocks are BKn + 1, j = 1. The difference data and the like are decoded with BKn, j = 1 for each block of Frame (n), and BKn, j = 0 for the other blocks of Frame (n).

When the calculation of the block of Frame (n) is completed, the control unit 2 initializes all the blocks of Frame (n-1) by setting BKn-1, j = 0, and further performs initialization. For each block of Frame (n-1) referred to from each block of BKn, j = 1 in Frame (n), the difference data and the like are decoded as BKn-1, j = 1, and Frame (n-1) is decoded. ), BKn-1, j = 0 for the other blocks.

After that, the control unit 2 executes the same calculation
When the block for which the pixel value needs to be calculated is set as BKfr, j = 1, the BKfr, j = 1
The pixel value is obtained by performing the IDCT operation on all the blocks of. At this time, since the IDCT calculation for the block where BKfr, j = 0 can be omitted, the amount of calculation by the control unit 2 is reduced.

The input device 3 includes a keyboard having a cursor key, numeric input keys, various function keys, and the like, and a mouse as a pointing device, and outputs to the control unit 2 a signal of a key pressed by the keyboard. At the same time, an operation signal from the mouse is output to the control unit 2. Other input devices may be used as needed.

The RAM (Random Access Memory) 4 has a work memory 4a for storing designated application programs, input instructions, input data, processing results, and the like.

The display device 5 is a CRT (Cathode Ray Tub).
e) It is constituted by a liquid crystal display device or the like, and displays display data according to a display control signal input from the control unit 2.

The storage device 6 has a storage medium 7 in which programs, data and the like are stored in advance, and this storage medium 7 is constituted by a magnetic or optical recording medium or a semiconductor memory. The storage medium 7 is fixedly provided in the storage device 6 or is detachably mounted. The storage medium 7 includes various application programs, menu display programs, and print processing corresponding to the image reproducing device 1. A program and data processed by each processing program are stored.

The program, data, and the like stored in the storage medium 7 may be configured to be received and stored from another device connected via a communication line or the like. The storage medium 7 is stored in another connected device.
May be provided so that programs, data, and the like stored in the storage medium 7 are used via a communication line.

Next, the operation will be described. First, in order to briefly explain the principle of an image reproducing method applied to the image reproducing apparatus 1 according to the first embodiment, a one-dimensional image is assumed to be viewed from a direction as shown in FIG. In the case of (1), the one-frame skip process will be described with reference to FIG.

In FIG. 3, a0 to a3 are Frame
Blocks constituting A, b0 to b3 are blocks constituting FrameB, c0 to c3 are blocks constituting FrameC, and mv (b0) to mv (b3).
Is a motion vector in which each block of Frame B refers to Frame A, and mv (c0) to mv (c3).
Is a motion vector in which each block of Frame C refers to Frame B.

In FIG. 3, Frame A is a frame that has already been decoded and exists as image information, and a case where Frame B is skipped and Frame C is decoded will be considered. Where Frame
A, B and C correspond to the above-mentioned H.264. 263, and is encoded by a communication medium encoding algorithm such as H.263.
frameB is a P picture that refers to FrameA, and FrameC is a P picture that refers to FrameB.

First, the control unit 2 sets the motion vector mv (b
0) to mv (b3) and mv (c0) to mv (c3)
Are decoded, and based on the motion vectors mv (c0) to mv (c3) referring to FrameB to be skipped, FrameC is selected from the blocks of FrameB.
Block that is not referenced from any of the blocks.

In the example of FIG. 3, the block c0 refers to the block b0, the block c1 refers to the block b2, and the block c2 is a part of the block b2 and the block b2.
3, block c3 refers to block b3. Therefore, block b1 of Frame B
Is detected as not being an image necessary as a reference image when decoding the blocks c0 to c3 of FrameC.

As described above, from among the blocks of the frame to be skipped, all the blocks which are not referred to by any block of the next frame are detected (only the block b1 in FIG. 3), and the IDCT operation for the detected block is performed. Is omitted, the amount of calculation can be reduced. In the example of FIG.
IDCT required for FrameC, 4 in FrameC
The operation is reduced to seven operations by omitting the IDCT operation on block b1.

Next, the case of n-frame skip processing will be described with reference to FIGS. 4 and 5 based on the principle of one-frame skip processing described above.

FIG. 4 is a flowchart showing a procedure of n-frame skip processing by the image reproducing apparatus 1 to which the image reproducing method of the present invention is applied, and FIG. 5 is a view for explaining a decoding order when n = 3. It is. In FIG. 4, BKfr, j means the j-th block in the fr frame in FIG. 5. If the block requires an IDCT operation, BKfr, j = 1; otherwise, BKfr, j. j = 0.

In the following description, Frame 0 of fr = 0
Are decoded frames, and fr = 1, 2,...
It is assumed that n frames of n are skipped. FIG.
In the above, Frame0 is a decoded frame,
This shows a case where three frames of fr = 1, 2, 3 are skipped. Here, each frame of fr = 1 to 4 corresponds to the aforementioned H.264 frame. 263, and is encoded by a communication medium encoding algorithm such as H.263.
This is a P picture that refers to frame0,
Is a P picture that refers to Frame1, each frame is composed of a P picture that refers to the immediately preceding frame.

First, after decoding Frame 0, the control unit 2 decodes all motion vector data up to Frame (n + 1) of fr = n + 1 (Step S).
1). In the example of FIG. 5, since n = 3, all the motion vector data up to Frame 4 are decoded.

Next, the control unit 2 sets Frame (n +
Since 1) needs to decode all blocks,
BKn + for all blocks in Frame (n + 1)
1, j = 1 (step S2), and then a loop of steps S3 and S4 for detecting blocks that need IDCT for all frames (i =
n + 1 to 2). In FIG. 5, BK4, j = 1 is set for all the blocks in Frame 4, and the processing shifts to processing for each block of each frame of fr = 3 to 1.

First, the control unit 2 sets BKi-1, j (i = n +
Initialization is performed for all blocks of Frame (n) corresponding to 1) with BKn, j = 0 (step S).
3) In addition, since Frame (n + 1) has BKn + 1, j = 1 for all blocks, BKn, j = for each block of Frame (n) referenced from each block of Frame (n + 1). The difference data and the like are decoded as 1 and BKn, j = 0 for the other blocks of Frame (n) (step S4). In FIG.
After initializing BK3, j = 0 for all the blocks in Frame3, BK3, j = 1 is set only for the block referred to by each block of Frame4, and BK3, j = It remains at 0.

When the calculation of the block of Frame (n) is completed, the control unit 2 decrements the value of i of BKi-1, j and sets the Fra corresponding to BKi-1, j (i = n).
BKn-1, j = for all blocks of me (n-1)
Initialization is performed by setting it to 0 (step S3).
For each block of Frame (n-1) referenced from each block of BKn, j = 1 of me (n), BKn-
Decode the difference data etc. with 1, j = 1, and set Frame
For the other blocks of (n-1), BKn-1,
j = 0 is set (step S4). In FIG. 5, Frame
2, BK2, j = 0 is initialized for all blocks, and BK2, j = 1 is set only for the blocks referred to by each block in which BK3, j = 1 in Frame3. Block remains at BK2, j = 0.

Thereafter, the same operation is repeated until Frame 1, and the block for which the pixel value needs to be calculated is BKfr, j
When = 1 is set, the pixel value is obtained by performing the IDCT operation on all the blocks where BKfr, j = 1 (step S5). At this time, since the IDCT calculation for the block with BKfr, j = 0 can be omitted, the amount of calculation by the control unit 2 is reduced. Also, IDCT
As for the blocks that do not need to perform the calculations, it is not necessary to perform other calculations such as MC, so that the calculation amount of the control unit 2 is also reduced.

Although the first embodiment has been described for a one-dimensional case, the same applies to a two-dimensional case.
For example, for a 16 × 16 block, j = 1to
By setting it to 256, processing can be performed on all blocks.

As described above, according to the image reproducing apparatus 1 of the first embodiment to which the image reproducing method of the present invention is applied, in the n-frame skip processing, the control unit 2 first sets the frame before skipping. After decoding Frame0,
Decode all the motion vector data up to the skipped frame Frame (n + 1), and then
BKn + 1, j = 1 for all blocks of e (n + 1)
Is set to perform the IDCT operation, and then processing for detecting a block requiring the IDCT is performed for all frames.

That is, the control unit 2 sequentially determines all the blocks of the frame to be skipped from the temporally succeeding frame to each of the blocks referred to by each block of BKfr, j = 1 in the next frame. For BKfr, j = 1
To decode the difference data, etc., and perform the same operation on the other blocks by setting BKfr, j = 0.
When the block for which the pixel value needs to be calculated is set as BKfr, j = 1, the IDCT operation is performed on all the blocks with BKfr, j = 1 to obtain the pixel value. At this time, since the IDCT calculation for the block where BKfr, j = 0 can be omitted, the amount of calculation by the control unit 2 is reduced.

Therefore, in general, when skipping image data encoded by a communication medium encoding algorithm in which image data of an I picture is not inserted into a bit stream, it is not necessary to perform an operation. Since the configuration can omit the operation for the block, high-speed skip reproduction can be realized. This also solves the problem that synchronization with audio data cannot be achieved due to insufficient processing capacity of the CPU.

(Second Embodiment) In the first embodiment described above, a case has been described where the amount of calculation is reduced by omitting the IDCT calculation for blocks not referenced from the frame to be decoded during skip reproduction. However, the amount of calculation can be reduced by using the linearity of the IDCT.

Therefore, in the second embodiment, the above-described H.264 is used. 263 is stored in a storage device connected to, for example, a personal computer or the like, and the skipped reproduction is performed when reproducing the stored image data. IDCT for DCT coefficients as shown in equation (2)
The case where the amount of calculation is reduced by processing the IDCT calculation on a block over a plurality of frames not related to MC by one calculation using the linearity of the above will be described.

Hereinafter, an image reproducing apparatus according to a second embodiment to which the image reproducing method according to the present invention is applied will be described in detail.

The image reproducing apparatus according to the second embodiment basically has substantially the same configuration as the image reproducing apparatus 1 according to the first embodiment. , And only the portions different from those of the first embodiment will be described below with reference to FIGS. 1, 6, and 7.

In FIG. 1, in the n-frame skipping process (see FIG. 6) described later, the control unit 2 first decodes Frame 0 of fr = 0, which is the frame before skipping, and then sets fr = n + 1 F
Decodes all motion vector data up to frame (n + 1), and then, based on the decoded motion vectors, a block sequence that satisfies a condition described later for all blocks of Frame0 to Frame (n + 1) and is not involved in MC Is detected.

Further, the control unit 2 adds a DCT coefficient to each of the detected block strings for each of the block strings, performs an IDCT operation on the added DCT coefficient, and finally, The pixel values of the first frame of the column are added. This allows the conventional method to perform multiple I
Calculations requiring DCT operation can be processed by one IDCT operation.

Next, the operation will be described. First, in order to briefly explain the principle of the image reproducing method applied to the image reproducing apparatus according to the second embodiment, a one-dimensional image is assumed to be viewed from the direction shown in FIG. In this case, the one-frame skip processing will be described with reference to FIG.

In FIG. 3, a0 to a3 are Frame
Blocks constituting A, b0 to b3 are blocks constituting FrameB, c0 to c3 are blocks constituting FrameC, and mv (b0) to mv (b3).
Is a motion vector in which each block of Frame B refers to Frame A, and mv (c0) to mv (c3).
Is a motion vector in which each block of Frame C refers to Frame B.

In FIG. 3, Frame A is a frame which has already been decoded and exists as image information, and a case where Frame B is skipped and Frame C is decoded will be considered. Where Frame
A, B and C correspond to the above-mentioned H.264. 263, and is encoded by a communication medium encoding algorithm such as H.263.
frameB is a P picture that refers to FrameA, and FrameC is a P picture that refers to FrameB.

First, the control unit 2 sets the motion vector mv (b
0) to mv (b3) and mv (c0) to mv (c3)
Are decoded, and based on the decoded motion vector, two conditions of not performing MC and not being referred to as a prediction source of MC from a block of the next frame are satisfied from each of the FrameB blocks. Detect block sequence. That is, a block sequence that is not involved in MC at all is detected from the skipped frames.

In the example shown in FIG. 3, since the block c0 refers to the block b0 at the same position in the immediately preceding frame, no MC is performed, and the frame c0 does not operate.
The block b0 of the eB is composed of the blocks c0 to
Since it is not referenced from any block of c3,
The above two conditions are satisfied. Furthermore, Fra
Blocks b0 and a0 of meB and FrameA also satisfy the above two conditions. Therefore, the block sequence of a0 → b0 → c0 satisfies the above condition.

Here, the IDCT shown in the above equation (2) is
Since DCT coefficients are linearly transformed for blocks at the same position in each frame, for example, when a plurality of DCT coefficients are x0, x1,..., Xn-1, the following equation (3) is used.
Holds.

IDCT (x0) + IDCT (x1) +... + IDCT (xn-1) = IDCT (x0 + x1 +... + Xn-1) (3)

By the way, the DCT coefficient of the block b0 is c
oefb0, the DCT coefficient of block c0 is
In the conventional decoding process, when obtaining the pixel value of the block c0 in FIG. 3 satisfying the above two conditions, first, the pixel value b0 is converted into the pixel value a0 by the following equation (4). After calculating using the calculated pixel value b0, the pixel value c0 of the block c0 is obtained by the calculation of the following equation (5).

B0 = a0 + IDCT (coefb0) (4) c0 = b0 + IDCT (coeff0) (5)

When the above equations (4) and (5) are combined into one equation, the following equation (6) is obtained. According to the above equation (3), this is expressed by the following equation (7). be able to.

C0 = a0 + IDCT (coefb0) + IDCT (coefc0) (6) c0 = a0 + IDCT (coefb0 + coefc0) (7)

That is, the two IDCT operations of the equations (4) and (5) are replaced by one IDCT operation of the equation (7).
It can be processed by CT operation. This means
For example, if all the blocks at the same position for n frames to be skipped satisfy the above two conditions, the processing that conventionally performed the IDCT operation n times is performed as follows.
This shows that the amount of calculation can be significantly reduced as a process by one IDCT calculation.

Note that M satisfying the above two conditions
Blocks that are not involved in C are assigned a code that is small in terms of variable length code.
The two conditions are often satisfied, and the image reproducing method applied to the image reproducing apparatus according to the second embodiment is effective.

Next, the case of n-frame skip processing based on the principle of the one-frame skip processing will be described with reference to FIGS.

FIG. 6 is a flowchart showing the procedure of n-frame skip processing by the image reproducing apparatus to which the image reproducing method of the present invention is applied. FIG. 7 is a diagram for explaining the decoding order when n = 4. Yes, the motion vector is represented by a broken line as in FIG. In the following description, BKfr, j means the j-th block in the fr frame in FIG.

In the following description, Frame 0 of fr = 0
Is a decoded frame, and fr = 1, 2,.
It is assumed that n frames of n are skipped. FIG.
In the above, Frame0 is a decoded frame,
This shows a case where four frames of fr = 1, 2, 3, and 4 are skipped. Here, each frame of fr = 1 to 5 corresponds to the above-described H.264 frame. H.263, and is encoded by a communication media encoding algorithm such as Frame1.
Is a P picture that refers to Frame 0, and Fra
Each frame is composed of a P picture that refers to the immediately preceding frame, such as me2 is a P picture that refers to Frame1.

First, after decoding Frame 0, the control unit 2 decodes all motion vector data up to Frame (n + 1) of fr = n + 1 (step S1).
1). In the example of FIG. 7, since n = 4, all the motion vector data up to Frame 5 are decoded.

Next, the control unit 2 sets the Frame based on the motion vector decoded in step S11.
A block sequence that satisfies the above two conditions is detected for all the blocks from 0 to Frame (n + 1) (step S12). In FIG. 7, part of BK4,2 and BK
BK performing MC by referring to part of 4,3
Blocks other than BK5,3, which is the next frame of BK4,3 referred to from BK5,2 and BK5,2, satisfy the above condition, and the following four block sequences exist.

[1] BK0,1 → BK1,1 → BK2,1 → BK
3,1 → BK4,1 → BK5,1 [2] BK0,2 → BK1,2 → BK2,2 → BK3,2 → BK4,2 [3] BK0,3 → BK1,3 → BK2,3 → BK3, 3 → BK4,3 [4] BK0,4 → BK1,4 → BK2,4 → BK3,4 → BK4,4
→ BK5,4

Further, the control unit 2 adds a DCT coefficient to each of the detected block strings for each of the block strings (step S13), and performs an IDCT operation on the added DCT coefficients (step S13). (Step S14) Finally, the pixel value of the first frame of each block sequence is added (Step S15).

In the block sequence [1], if the DCT coefficient of BKfr, j is represented as coefBKfr, j, the following equation (8) is calculated in step S13, and the equation (8) is calculated in step S14. I) for the calculation result of
IDCT (Σcoef) is executed as the DCT operation, and the calculation of the following equation (9) is executed in step S15.

Σcoef = coefBK4,1 + coefBK3,1 + coefBK2,1 + coefBK1,1 (8) (pixel value of BK5,1) = IDCT (Σcoef) + (pixel value of BK0,1) 9) In this case, in the conventional method, a calculation requiring four IDCT operations can be processed by one IDCT operation.

If the same calculation is performed for each of the block sequences [2] to [4], three IDCT operations are performed in the case of [2] and [3]. In this case, four IDCT operations can be processed by one IDCT operation.

In the second embodiment, a one-dimensional case has been described, but the same applies to a two-dimensional case.
For example, for a 16 × 16 block, j = 1to
By setting it to 256, processing can be performed on all blocks.

As described above, according to the image reproducing apparatus of the second embodiment to which the image reproducing method of the present invention is applied, in the n-frame skipping process, first, in the n-frame skipping process, the frame 2 before the skipping is performed. After decoding
All the motion vector data up to the skipped frame Frame (n + 1) are decoded, and then, based on the decoded motion vector, Frame0 to Frame
A block sequence that satisfies the above condition is detected for all blocks of me (n + 1).

Further, the control unit 2 adds a DCT coefficient to each of the detected block strings for each block string, performs an IDCT operation on the added DCT coefficient, and finally, The pixel values of the first frame of the column are added. This allows the conventional method to perform multiple I
Calculations requiring DCT operation can be processed by one IDCT operation.

Therefore, in general, when skipping reproduction of image data encoded by a communication medium encoding algorithm in which image data of an I picture is not inserted in a bit stream, a plurality of data not related to the MC are provided. Since the configuration is such that a plurality of IDCT operations on a block can be processed by one IDCT operation, high-speed skip reproduction can be realized. In addition, due to this, the processing capacity of the CPU is insufficient,
The problem that synchronization with audio data cannot be obtained can also be solved.

In the first and second embodiments described above, the case where the DCT coefficient subjected to the DCT operation at the time of encoding is decoded by performing the IDCT operation. Is not limited to DCT, and any other coding method may be used as long as the coding method does not deviate from the gist of the present invention, for example, the coefficient of each block has linearity at the time of inverse transform.

In the first and second embodiments described above, examples in which the effect of the present invention is particularly remarkable are as follows.
Although the description has been given of the case where the bit stream is decoded and reproduced by the encoding algorithm for the communication medium, the present invention is applied to the decoding of the bit stream by the other encoding algorithm using the forward prediction encoded image. It is also possible to apply when reproducing.

(Third Embodiment) Hereinafter, FIGS.
With reference to, the image encoding device 10 to which the image encoding method according to the present invention is applied will be described in detail. In addition, in the present embodiment, the intra image is the same as the conventional one, and therefore, the description will be expanded to the inter image. Decoding is performed in the same manner as in the related art.

First, the configuration will be described.

FIG. 8 is a block diagram showing a configuration of the image coding apparatus 10 according to the present embodiment. In FIG. 8, the image encoding device 10 includes a control unit 11, an input device 1
2, display device 13, communication interface unit 14, image codec unit 15, audio codec unit 16, RAM 17,
It comprises a storage device 18 and a storage medium 19, and each part except the storage medium 19 is connected by a bus 20.

The control unit 11 stores the application program designated from the various application programs corresponding to the image encoding device 10 stored in the storage device 18 and various instructions or data input from the input device 12 into the RAM 17. In accordance with the input instruction and the input data, various processes are executed in accordance with the application program stored in the RAM 17, and the processing results are stored in the work memory in the RAM 17 and displayed on the display device 13. I do. Then, the processing result stored in the work memory is stored in a storage destination in the storage device 18 which is instructed to be input from the input device 12.

At the time of the image encoding process, the control unit 11 multiplexes the image encoded data encoded by the image codec unit 15 and the audio encoded data encoded by the audio codec unit 16. Then, a bit stream of one column is generated and stored in the storage device 18 or output to the outside via the communication interface unit 14. On the other hand, at the time of image decoding processing, the control unit 11
Separates the image data and the audio data from the bit stream read from the storage device 18 or the bit stream input externally through the communication interface unit 14 and transfers them to the image codec unit 15 or the audio codec unit 16, respectively. I do.

The input device 12 includes, for example, cursor keys,
A keyboard with numeric input keys and various function keys, and a mouse as a pointing device,
A signal for pressing a key pressed on the keyboard is output to the control unit 11, and an operation signal from a mouse is output to the control unit 11.
Output to

The display device 13 is composed of a CRT, a liquid crystal display device and the like, and displays display data according to a display control signal input from the control unit 11. Also, the display device 1
3 performs image display according to the image data decoded by the image codec unit 15.

The communication interface unit 14 has a modem (M
ODEM: MOdulator / DEModulator, terminal adapter (TA), or router, etc., and controls communication with external devices via communication lines such as telephone lines, ISDN lines, or dedicated lines. Do.

The image codec 15 encodes image data stored in the storage medium 19 in the storage device 18, image data input from a video camera (not shown), and the like. That is, the image codec unit 15 outputs the immediately preceding frame, Frame (n-1), for Frame (n), which is the n-th frame of the input image data.
As described later, based on the result of calculating the ME with the reference source as the reference source, in case 1 as shown in FIG. 9, in the same manner as the conventional encoding process, the block unit in the procedure as shown in FIG. Encoding is performed, and in case 2, without generating a predicted image, encoding is performed in block units according to the procedure shown in FIG. 11 and the IDCT operation is omitted as appropriate.

[0119] The image codec unit 15
In the case of, as described later with reference to FIG.
DCT of input image P of input Frame (n-1)
Is calculated (DCT operation unit 15h), and the difference between the DCT coefficient H and the predicted DCT coefficient N of Frame (n-2) of the reference destination is calculated (subtraction unit 15i). And the calculated DC
The difference DCT coefficient I of the T coefficient is quantized (quantization unit 15
j), generate a coded block J. And this ca
In the case of se2, coefficients of the same frequency in the DCT block are calculated in order to process the blocks at the same position, and the linearity of the DCT can be used.
There is no need to perform DCT calculations. That is, the coded block J is inversely quantized (the inverse quantization unit 15k), and the prediction D of the reference destination Frame (n−2) remains in the form of the DCT coefficient K.
The CT coefficient N is added (adding unit 15m), and the predicted DCT coefficient M for the next frame is input to the Delay 15n.

The image codec 15 decodes encoded image data separated from encoded audio data by the control unit 11 according to a decoding algorithm corresponding to the above-described encoding algorithm.

The audio codec unit 16 encodes audio to be added to the encoded image data encoded by the image codec unit 15, and transfers the encoded audio data to the control unit 11. Further, the audio codec unit 16 performs a decoding process on the encoded audio data separated from the encoded image data by the control unit 11.

The RAM 17 has a work memory for storing designated application programs, input instructions, input data, processing results, and the like.

The storage device 18 has a storage medium 19 in which programs, data and the like are stored in advance, and this storage medium 19 is constituted by a magnetic or optical recording medium or a semiconductor memory. The storage medium 19 is fixedly provided on the storage device 18 or is detachably mounted. The storage medium 19 includes various application programs corresponding to the image encoding device 10 and a block encoding process 1. And an image encoding processing program including block encoding processing 2, data processed by each processing program, image encoded data, and the like.

The program, data, and the like stored in the storage medium 19 may be received and stored from another device connected via a communication line or the like, and may be stored via the communication line or the like. A storage device having the storage medium 19 may be provided on the other connected device side, and a program, data, and the like stored in the storage medium 19 may be used via a communication line.

Next, the operation will be described. FIG. 9 to FIG. 11 are diagrams for explaining the outline of the block-based image encoding process performed by the image codec unit 15 of the image encoding device 10 according to the present embodiment. Note that, in the following description, BKn, j is similar to the second embodiment.
It means the j-th block in the n-th frame. Further, as described above, in the present embodiment, since only the inter image can be applied, the description focuses on the features of the invention to the inter image.

The image codec unit 15 calculates the ME of Frame (n), which is the n-th frame of the input image data, using Frame (n−1), which is the immediately preceding frame, as a reference source. Based on this, processing is performed by classifying into two cases, case 1 and case 2 as shown in FIG.

That is, for each block BKn-1, j of Frame (n-1), Frame
In the case of a block referred to from a block BKn, x at another position of e (n), a frame such as case2 is used.
The case is classified into two cases, namely, the case of the block referred to from the block BKn, j at the same position in (n). Then, in case 1, encoding is performed in block units in the procedure shown in FIG. 10 as in the conventional encoding process. In case 2, a predicted image is generated without generating a predicted image.
Encoding in block units is performed by the procedure shown in FIG. 11, and the IDCT operation is omitted as appropriate.

FIG. 10 is a block diagram showing the flow of arithmetic processing after DCT after ME calculation in case 1 (similar to the conventional case). In FIG. 10, when encoding each block of Frame (n), the image codec unit 15 uses case1 of Frame (n−1).
Is processed in the following procedure for the block corresponding to.

First, a difference between the input image P of the input Frame (n-1) and the predicted image G of the reference frame (n-2) is calculated (subtraction unit 15a). Then, the DCT of the calculated difference image A is calculated (DCT calculation unit 15
b) Further, the calculated DCT coefficient B is quantized (quantization unit 15c) to generate a coded block C.

Then, as local encoding for generating a predicted image, the coded block C is inversely quantized (the inverse quantization unit 1).
5d) Further, an IDCT is calculated (IDCT operation unit 15).
e), the reference destination Frame subtracted by the subtraction unit 15a
The (n-2) predicted image G is added (adding unit 15f),
The predicted image F for the next frame is input to the predictor 15g. That is, in case 1, Frame
An IDCT operation is required to generate the (n-1) predicted image.

FIG. 11 is a block diagram showing the flow of arithmetic processing after DCT after ME calculation in case 2 (characteristic part of the present embodiment). In FIG. 11, the image codec unit 15 encodes each block of Frame (n) by using c of Frame (n−1).
The block corresponding to case 2 is processed according to the following procedure.

First, the DCT of the input image P of the input Frame (n-1) is calculated (DCT operation unit 15h).
DCT coefficient H and prediction D of Frame (n-2) of reference destination
The difference from the CT coefficient N is calculated (subtraction unit 15i). Then, the difference DCT coefficient I of the calculated DCT coefficient is quantized (a quantization unit 15j) to generate a coded block J.

In case 2, in order to process the block at the same position (ie, the same block as the previous frame without motion), the coefficient of the same frequency in the DCT block is calculated. , And there is no need to calculate the IDCT. That is, the coding block J is inversely quantized (inverse quantization unit 15k), and the reference Fr is kept in the form of the DCT coefficient K.
The predicted DCT coefficient N for ame (n−2) is added (adding unit 15m), and the predicted DCT coefficient M for the next frame is input to the predictor 15n. As a result, it is possible to omit the calculation of the IDCT which requires a large amount of calculation,
Speeding up of the entire image encoding process can be realized.

Next, while referring to the outline described above,
The operation of the image encoding device 10 according to the present embodiment will be described in detail with reference to FIGS. FIG. 12 is a flowchart illustrating an image encoding process performed by the image codec unit 15 of the image encoding device 10 according to the present embodiment.

In FIG. 12, image codec 1
5 first encodes the input first frame Frame0 (step S101). Then, n is set to 1 (step S102), and in steps S103 to S108, Frame (n) of the image data is sequentially encoded by incrementing the value of n.

When encoding the Frame (n), the image codec 15 first sets the previous frame Fra.
For example, the ME of each 16 × 16 block of Frame (n) with reference to me (n−1) is calculated (step S103).

Next, the image codec unit 15
For each block BKn, j of me (n), whether MC on the BKn, j side is 0, that is, if BKn, j is Fra
It is determined whether or not the block BKn-1, j at the same position of me (n-1) is referenced (step S104). Then, for BKn, j for which it is determined that MC is 0 (step S104; YES), block encoding processing 1 (see FIG. 13) described later is executed (step S10).
5) For BKn, j for which it is determined that MC is not 0 (step S104; NO), block encoding processing 2 (see FIG. 14) described later is executed (step S10).
6).

When the block encoding process 1 or the block encoding process 2 ends, the image codec unit 15 determines whether to end the image encoding based on whether the next frame has been input (step S107). If it is determined that the processing has ended (step S107; YES),
When a series of image encoding processes is completed and it is determined that the process is not completed (step S107; NO), the value of n is incremented (step S108), and a loop process of steps S103 to S107 for the next frame is performed. Move to.

FIG. 13 is a flowchart showing a block encoding process 1 executed by the image codec unit 15 as a process in the image encoding process shown in FIG. This FIG.
The flowchart shown in FIG. 11 shows the processing corresponding to FIG. 11 described above, and will be described below with reference to FIG. Hereinafter, the DCT coefficient of BK is represented as * BK.

In FIG. 13, image codec 1
5 is that, for BKn-1, j at the same position as BKn, j, the MC for the Frame (n-2) block is 0.
Whether BKn-1, j is Frame (n
It is determined whether or not the block BKn-2, j at the same position in (-2) is referred to (step S201). If it is determined that MC is not 0 (step S201; NO), a block code to be described later is used. Since the IDCT calculation has already been completed for the reference block BKn-2, x serving as the predicted image by the conversion process 2, the DCT of BKn-2, x is calculated to calculate * BKn-2, x Is calculated (step S202).

On the other hand, when it is determined that MC is 0 (step S201; YES), Frame (n-
Since BKn-2, j is held as a DCT coefficient during the encoding of 1), the DCT calculation is not performed. Then, regardless of the result of the determination in step S201, the processing described above causes the addition unit 15m shown in FIG.
Has * BK as a predicted DCT coefficient N based on the predicted image.
This means that n-2, j is prepared.

Then, the image codec unit 15
The DCT coefficient K (= * BKn-1, j- * BKn-2, z (z = j) input from the inverse quantization unit 15k to the addition unit 15m shown in FIG.
Or x)) and a predicted DCT coefficient N (= * BKn-2, z) based on the predicted image (adding unit 15m), thereby obtaining a predicted DCT coefficient M (= * BKn-1, j); That is, the DCT coefficient of BKn-1, j is calculated (step S203). For the input image P (= BKn, j of Frame (n)), the DCT operation unit 15 shown in FIG.
h, the DCT is calculated and the DCT coefficient H (= * B
Kn, j) (step S204), and the DCT
The difference DCT coefficient I (= * BKn, j- * BKn-1, j) between the DCT coefficient of BKn, j input from the operation unit 15h and the DCT coefficient of BKn-1, j input from the predictor 15n is calculated. After calculating (subtraction unit 15i) and quantizing the difference DCT coefficient I (quantization unit 15j), the encoded block J is transferred to the control unit 11, or the communication interface unit 1
4 to the communication path (step S205).

Further, the next frame Frame (n + 1)
As local decoding for predicting each block BKn + 1, j, the coded block J is inversely quantized by the inverse quantization unit 15k (step S206), and a series of block encoding processes 1 is completed. In the above processing, in order to process blocks at the same position on the time axis, coefficients of the same frequency in the DCT block are calculated,
The linearity of DCT can be used, and the calculation of IDCT can be omitted.

FIG. 14 is a flowchart showing a block encoding process 2 executed by the image codec 15 as a process in the image encoding process shown in FIG. This FIG.
The flowchart shown in FIG. 10 shows the processing corresponding to FIG. 10 described above, and will be described below with reference to FIG.

In FIG. 14, image codec unit 1
5 first locally decodes the reference block BKn-1, x of Frame (n-1) based on the MC in order to generate a predicted image when BKn, j is encoded. At this time, the image codec unit 15 converts the DCT coefficient D (= * BKn-1, x- * BK) calculated in the process of encoding BKn-1, x.
n-2, y) by the IDCT operation unit 1 shown in FIG.
5e to calculate the difference image E (= BKn-1, x−BK
n-2, y) is calculated (step S301).

Next, the image codec unit 15 outputs the BKn-
It is determined whether or not the MC of 1, x and the block BKn-2, y of Frame (n-2) referred to by BKn-1, x is 0, that is, whether x = y. (Step S30
2). If it is determined that MC is 0 (step S302; YES), BKn−1, x is encoded by the above-described block encoding process 1, and BKn−x is thus encoded.
2, y is the DCT coefficient form (predicted DCT coefficient M = * BKn-2,
y), it is held in the predictor 15n, so * BKn-2,
BKn-2, y is calculated by calculating the IDCT of y (step S303). When it is determined that MC is not 0 (step S302; NO), BKn-1, x is coded by the block coding process 2, and BKn-2, y is decoded. Image shape (predicted image F = B
Kn-2, y), which is stored in the predictor 15g,
There is no need to perform CT calculations.

The image codec unit 15 further includes a difference image E (= BKn-1, x-BKn-2, y) input from the IDCT operation unit 15e shown in FIG.
By adding Kn-2, y (adding unit 15f),
Calculate the predicted image F (= BKn-1, x) (step S3)
04).

Then, the image codec unit 15 outputs the input image P (= BKn, j of Frame (n)) and the predictor 15
The difference image A from BKn-1, x input from g (= BKn,
j -BKn-1, x) (subtraction unit 15a), the DCT of the difference image A is calculated by the DCT calculation unit 15b (step S305), and the calculated DCT coefficient B
Is quantized (quantization unit 15c), and then the encoded block C
Is transferred to the control unit 11 or output to the communication path via the communication interface unit 14 (step S30).
6).

Further, the next frame Frame (n + 1)
As local decoding for predicting each block BKn + 1, j, the coded block C is inversely quantized by the inverse quantization unit 15d (step S307), and a series of block encoding processes 2 ends.

The block encoding 1 as described above, or
The image encoding process is performed by appropriately using the block encoding process 2, but in the actual image encoding process, ME
It is known that, as a result of the calculation, blocks (MC = 0) in which MC refers to a block at the same position are continuously generated. Therefore, when the image encoding process shown in FIG. 12 is performed, many IDCT operations can be omitted.

As described above, according to the image encoding apparatus of the third embodiment to which the image encoding method of the present invention is applied, the image codec unit 15 in FIG. As shown, first, the input Frame
The DCT of the input image P of e (n-1) is calculated (DCT calculation unit 15h), and the DCT coefficient H and the reference Frame (n) are calculated.
The difference from the predicted DCT coefficient N of -2) is calculated (subtraction unit 15i). Then, the difference DCT between the calculated DCT coefficients is calculated.
The coefficient I is quantized (a quantization unit 15j) to generate a coded block J. In case 2, in order to process blocks at the same position, coefficients of the same frequency in the DCT block are calculated, so that the linearity of the DCT can be used and the IDCT is calculated. No need. That is, the coding block J is inversely quantized (the inverse quantization unit 15k), and the prediction DCT coefficient N of the reference frame (n−2) is added in the form of the DCT coefficient K (addition unit 15m). Predicted DCT for next frame
The coefficient M is input to the predictor 15n.

Therefore, based on the calculation result of ME,
For blocks satisfying the condition that the MC refers to the block at the same position in the previous frame, DCT
Can be used to calculate the difference between the DCT coefficient of the input image and the DCT coefficient of the predicted image instead of the difference between the input image and the predicted image. If you want to
The calculation of CT can be omitted, and the speed of the image encoding process can be increased.

Further, even if a high-performance processor or the like is not used as the image codec unit 15,
Since the image encoding process can be performed at a practical speed by an inexpensive processor, the manufacturing cost of the image encoding device 10 can be reduced.

Note that in the third embodiment,
Although the case where DCT is used as an operation for encoding image data has been described, image data may be encoded using other orthogonal transformation operations generally used for block encoding.

[0155]

According to the first aspect of the present invention, a predetermined decoding process can be executed only on blocks necessary for decoding a frame after skipping.
The amount of processing at the time of skip reproduction can be reduced, and high-speed skip reproduction can be realized. This also allows C
The problem that synchronization with audio data cannot be achieved due to insufficient processing capacity of the PU can also be solved.

According to the second aspect of the present invention, a plurality of blocks extending over a plurality of frames referencing only a block at the same position in a frame without reference to and reference to a block at a different position in the preceding and succeeding frames. , Decoding processing can be performed at once, so that the amount of processing at the time of skip reproduction can be reduced and high-speed skip reproduction can be realized. This also solves the problem that synchronization with audio data cannot be achieved due to insufficient processing capacity of the CPU.

According to the image coding apparatus of the invention described in claim 3 or the image coding method of the invention described in claim 4,
Based on the motion vector calculation result, for a block satisfying the condition that the motion vector refers to the block at the same position in the previous frame, for example, the linearity of a predetermined orthogonal transform operation such as DCT is determined. The difference between the orthogonal transform coefficient (for example, DCT coefficient) of the input image and the orthogonal transform coefficient of the predicted image can be calculated instead of the difference between the input image and the predicted image. Appear, the calculation amount is large, for example, I
The calculation of the inverse transform of the predetermined orthogonal transform operation such as DCT can be omitted, and the image encoding process can be speeded up. In addition, this makes it possible to perform image encoding processing at a practical speed with an inexpensive processor without using a high-performance processor or the like,
The manufacturing cost of the image encoding device can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an image reproducing apparatus 1 to which an image reproducing method according to the present invention is applied.

FIG. 2 is a view showing a direction in which a two-dimensional image is viewed as one-dimensional in order to briefly explain the principle of the image reproducing method of the present invention.

FIG. 3 is a diagram for explaining one-frame skip processing in a one-dimensional case in order to briefly explain the principle of the image reproducing method of the present invention.

FIG. 4 is a flowchart illustrating a procedure of n-frame skip processing by the image reproducing apparatus 1 to which the image reproducing method according to the first embodiment of the present invention is applied.

FIG. 5 is a diagram illustrating a decoding order when n = 3 in the procedure of FIG. 4;

FIG. 6 is a flowchart showing a procedure of n-frame skip processing by an image reproducing apparatus to which the image reproducing method according to the second embodiment of the present invention is applied.

FIG. 7 is a diagram illustrating a decoding order in a case where n = 4 in the procedure of FIG. 6;

FIG. 8 is a block diagram illustrating a configuration of an image encoding device 10 to which the image encoding method of the present invention has been applied.

FIG. 9 is a diagram showing two cases, case 1 and case 2, for classifying the processing methods in the encoding processing in the image encoding method of the present invention.

10 is a diagram illustrating a case 1 in case 1 shown in FIG.
It is a block diagram showing the flow of the arithmetic processing after DCT after calculation.

11 is a diagram illustrating a case 2 in case 2 shown in FIG.
It is a block diagram showing the flow of the arithmetic processing after DCT after calculation.

FIG. 12 is a flowchart illustrating an image encoding process performed by the image codec unit 15 of the image encoding device 10 to which the image encoding method of the present invention has been applied.

13 is a block encoding process 1 performed by the image codec unit 15 as a process in the image encoding process illustrated in FIG.
It is a flowchart which shows.

14 is a block encoding process 2 performed by the image codec unit 15 as a process in the image encoding process illustrated in FIG.
It is a flowchart which shows.

FIG. 15 shows M conventionally used for image encoding processing.
It is a figure for explaining C.

FIG. 16 is a diagram illustrating an encoding and decoding process of the MC + DCT hybrid scheme.

FIG. FIG. 263 is a diagram showing a bit stream structure of H.263 in frame units.

FIG. 18 is a diagram showing a conventional MPEG bit stream structure in frame units.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 image reproduction device 2 control unit 3 input device 4 RAM 5 display device 6 storage device 7 storage medium 8 bus 10 image encoding device 11 control unit 12 input device 13 display device 14 communication interface unit 15 image codec unit 15a, 15i subtraction unit 15b, 15h DCT operation unit 15c, 15j Quantization unit 15d, 15k Inverse quantization unit 15e IDCT operation unit 15f, 15m Addition unit 15g, 15n Delay 16 Audio codec unit 17 RAM 18 Storage device 19 Storage medium 20 Bus

Claims (4)

[Claims] An image data encoded by an image encoding method using a motion vector for motion-compensating each block of a current frame with reference to a block constituting a frame positioned temporally ahead is decoded. In the image reproducing method for reproducing, when an arbitrary number of frames are skipped and reproduced, the motion vectors are previously decoded and stored in a storage unit for all blocks to be skipped and all the blocks in the skipped frame, Based on the motion vector stored in the storage unit, the reference relation of each block is sequentially and retroactively calculated from the skipped frame toward the temporally previous frame, and the skipped frame having the motion vector is calculated. It is characterized in that the block is detected and a predetermined decoding process is performed on the detected block to reproduce an image. Image reproduction method to. 2. Decoding image data encoded by an image encoding method using a motion vector for motion-compensating each block of a current frame with reference to a block constituting a frame located temporally ahead. In the image reproducing method for reproducing, when an arbitrary number of frames are skipped and reproduced, the motion vectors are previously decoded and stored in a storage unit for all blocks to be skipped and all the blocks in the skipped frame, On the basis of the motion vector stored in the storage means, a block at a different position in a temporally preceding frame is not referred to, and a reference is not made from a block at a different position in a temporally located frame. A block is detected from all the skipped frames, and the detected block is located at the same position in the frame. A block sequence formed by being present over a plurality of frames is further detected, and a predetermined decoding process for all blocks over a plurality of frames constituting the block sequence is collectively executed at a time to reproduce an image. Characteristic image reproduction method. 3. A motion vector calculating means for calculating a motion vector for motion-compensating each block of a current frame of the image data with reference to a block constituting a frame positioned temporally before the image data; Encoding means for transforming the image data into orthogonal transform coefficients in block units by orthogonal transformation operation and encoding the image data; and inversely transforming the image data in block units encoded by the encoding means into the predetermined orthogonal transformation operation A local decoding means for performing local decoding by operation, wherein the motion vector calculated by the motion vector calculating means for each block of the current frame is within a frame located temporally ahead. And determining whether the block refers to a block at the same position. If it is determined that the vector does not refer to a block at the same position in the frame located earlier in time, the reference decoding block in the frame located earlier in time is replaced by the local decoding means. After performing local decoding to reconstruct the block of the reference image and calculating the difference between the block of the input image and the block of the reference destination, the encoding unit performs the predetermined orthogonal transformation operation to perform the When the motion vector is determined by the determination means to refer to the block at the same position in the frame located earlier in time, the motion vector is located earlier in the time. An orthogonal transform obtained by transforming a block of an input image by the encoding unit without performing local decoding processing by the local decoding unit on a reference block in a frame. Encoding the image data by calculating the difference between the number and the orthogonal transform coefficient of the reference block already calculated when encoding the temporally preceding frame. Image coding device. 4. An image encoding apparatus for encoding image data by an image encoding method using a motion vector for motion-compensating each block of a current frame with reference to a block constituting a frame located earlier in time. Determining whether the motion vector calculated for each block in the current frame refers to a block at the same position in the temporally preceding frame, wherein the motion vector is temporally If it is determined that the reference block does not refer to the block at the same position in the frame located earlier, the reference block in the frame located earlier in time is locally decoded and the block of the reference image is reproduced. After calculating the difference between the block of the input image and the block of the reference image, a predetermined orthogonal transform operation is performed to encode the image data, and If it is determined that the vector refers to a block at the same position in the frame located earlier in time, the local decoding process is performed on the reference block in the frame located earlier in time. Instead, the orthogonal transform coefficient calculated by performing the predetermined orthogonal transform operation on the block of the input image, and the orthogonality of the reference block already calculated when encoding the frame located earlier in time. An image coding method comprising calculating a difference from a transform coefficient and coding image data.