JP2005347841A - Image reproducing device and image reproducing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reproducing device and an image reproducing method capable of more smoothly reproduce a slow image by suppressing distortion at slow reproduction. <P>SOLUTION: The motion vector arithmetic section 16a of a motion compensation section 16 calculates a motion vector for an interpolation image for slow reproduction. The motion compensation section 16 shifts the regular decode image of a reference frame stored in an image memory 19 by the calculated motion vector and outputs the result to an adder 16. Further, the motion compensation section 16 outputs the calculated motion vector mv1 to the position conversion section 17b of a difference information processing section 17. A pixel conversion section 17a applies pixel conversion to the pixel value of each pixel of regular difference information supplied from an inverse DCT section 14 on the basis of slow control information S1, and the position conversion section 17b carries out position conversion. The adder 18 sums up the difference information outputted from the difference information processing section 17 to the image outputted from the motion compensation section 16 to generate an interpolation image for the slow reproduction. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image reproduction device and an image reproduction method.

  Some moving picture images use MPEG (Moving Picture Experts Group / Moving Picture Image Coding Experts Group) encoding methods such as MPEG2 and MPEG4.

  The MPEG encoding method is a method in which motion compensation prediction is performed for a portion having motion between frames forming an image, and difference information obtained between the frames is encoded by DCT processing and entropy encoding method. It is. In the MPEG encoding system, by performing such encoding, the redundancy is removed from the moving image such as the supplied moving image, and the amount of information is reduced.

  In order to decode the encoded data generated by this MPEG encoding method, a motion vector indicating the direction and distance of movement of the subject from the reference frame to the current frame is required. For this reason, motion vectors are included in encoded data generated by the MPEG encoding method.

Then, there is an image reproduction apparatus that uses the motion vector to reproduce an image slowly (see, for example, Patent Document 1). That is, this image reproducing apparatus calculates (N-1) motion vectors for interpolated images, where N is the slow playback magnification, and uses the calculated motion vectors to calculate (N-1) interpolated images. It is trying to generate. In addition, the image reproduction device generates an interpolation image based on the normal difference information corresponding to the normal decoded image to be normally decoded and the calculated motion vectors for (N−1) interpolation images. Like to do.
Japanese Patent Laid-Open No. 10-093920 (pages 3, 4 and FIG. 1)

  However, in the conventional image reproducing device, even if an interpolation image is generated in this way, distortion may occur between blocks for which motion compensation is performed, and this distortion may become significant. Is not always smooth.

  The present invention has been made in view of such a conventional problem, and an object of the present invention is to provide an image reproducing apparatus and an image reproducing method capable of smoothly reproducing a slow image.

In order to achieve this object, an image reproduction apparatus according to the first aspect of the present invention provides:
In an image reproduction apparatus for decoding an encoded data including difference information indicating a difference between a current image and an image subjected to motion compensation, and a motion vector for performing the motion compensation, and reproducing an image,
A decoding unit for decoding the encoded data;
An interpolation motion vector generation unit that generates an interpolation motion vector for an interpolation image used for slow reproduction from the normal motion vector, using the motion vector decoded by the decoding unit as a normal motion vector;
Using the difference information decoded by the decoding unit as normal difference information, based on the preset slow multiple and the interpolation motion vector acquired by the interpolation motion vector generation unit, performs image processing on the normal difference information, An interpolation difference information generating unit for generating interpolation difference information for generating the interpolation image;
An interpolated image that generates the interpolated image by adding the motion compensated interpolated image subjected to motion compensation based on the interpolated motion vector generated by the interpolated motion vector generating unit and the interpolated difference information generated by the interpolated differential information generating unit A generator,
Inserting the interpolated image generated by the interpolated image generation unit before and after the normal decoded image generated based on the normal difference information and the normal motion vector, and setting the time so as to be played slowly, the normal decoding An image output unit that sequentially outputs an image and the interpolated image is provided.

  The interpolation difference information generation unit is configured to perform pixel processing for each pixel value of the normal difference information based on the slow multiple so that the pixel value changes between the normal difference information before and after the image processing for the normal difference information. The interpolation difference information may be generated by performing value conversion.

  The interpolation difference information generation unit performs position conversion on the normal difference information based on the interpolation motion vector generated by the interpolation motion vector generation unit as image processing for the normal difference information, and generates the interpolation difference information. You may make it do.

  The interpolation difference information generating unit sets each of the nth interpolation difference information from the previous normal difference information among (N−1) sheets inserted between the preceding and following normal difference information, where N is a slow multiple. The n-th interpolation difference information may be generated by obtaining the pixel value of n by multiplying the pixel value of each pixel of the subsequent normal difference information by n / N.

  The interpolation difference information generation unit obtains a position conversion motion vector for performing position conversion on the normal difference information from the normal motion vector according to Equation 1, and based on the obtained position conversion motion vector, the normal difference The interpolation difference information may be generated by performing position conversion on the information.

  The interpolation motion vector generation unit sets the slow multiple to N, and among the (N−1) images inserted between the preceding and succeeding normal decoded images, interpolation for generating the nth interpolated image from the previous normal decoded image The motion vector may be generated by multiplying the normal motion vector by n / N.

An image reproduction method according to a second aspect of the present invention is:
Decoding encoded data including difference information indicating a difference between a current image and an image on which motion compensation has been performed, and a motion vector for performing motion compensation;
Generating an interpolated motion vector for an interpolated image used for slow reproduction from the normal motion vector, using the decoded motion vector as a normal motion vector;
Using the decoded difference information as normal difference information, image processing is performed on the normal difference information based on a preset slow multiple and the generated interpolation motion vector, and the interpolation difference information for generating the interpolation image is obtained. Generating step;
Adding the motion compensated interpolated image subjected to motion compensation based on the generated interpolated motion vector and the generated interpolated difference information to generate the interpolated image;
Inserting the generated interpolated image before and after the normal decoded image generated based on the normal difference information and the normal motion vector, and setting the time so as to be played back slowly, the normal decoded image and the interpolated image Are sequentially output.

  According to the present invention, a slow image can be smoothly reproduced.

Hereinafter, an image reproducing apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of an image playback apparatus according to this embodiment.
The image reproduction device 1 according to the present embodiment includes a code buffer 11, a variable length decoding unit 12, an inverse quantization unit 13, an inverse DCT unit 14, a control unit 15, a motion compensation unit 16, and difference information. A processing unit 17, an adder 18, an image memory 19, a display buffer 20, and an image output unit 21 are provided.

  For example, data encoded by an MPEG encoding method such as MPEG2 or MPEG4 is supplied to the image reproducing device 1. In the MPEG encoding method, motion compensation is performed based on the inter-frame difference of an image moved between the reference frame and the current frame.

  This motion compensation will be described with reference to FIG. Note that motion compensation includes forward prediction and backward prediction. Hereinafter, motion compensation will be described using the previous frame in forward prediction as a reference frame. Motion compensation is performed in units of blocks. The nth frame and the (n−1) th frame are defined as a current frame and a reference frame, respectively. The blocks An (i, j) and An-1 (i, j) are blocks at the pixel position (i, j) of the nth frame and the (n-1) th frame, respectively. In motion compensation, the most similar block in the set search range is searched around the block An-1 (i, j) located at the same position as the block An (i, j).

  If the most approximate block is An-1 (iu, jv) indicated by a broken line, the block An-1 (iu, jv) is used as a motion compensation prediction value. That is, with the motion vector V (u, v), the nth block based on the block An-1 (i, jv) and the motion vector V (u, v) of the (n-1) th frame. Block An (i, j) can be predicted.

  Thus, instead of predicting the block An (i, j) of the nth block based on the block An-1 (i, j) of the (n-1) th frame, the block An-1 (i, u, Predicting the block An (i, j) of the nth block based on j−v) and the motion vector V (u, v) is called motion compensation.

  The encoded data by the MPEG encoding method is generated by an encoding device 2 as shown in FIG. The encoding device 2 includes a subtractor 31, a DCT unit 32, a quantization unit 33, an inverse quantization unit 34, an inverse DCT unit 35, an adder 36, a frame memory 37, and a motion compensation unit 38. And a motion vector detection unit 39 and a variable length encoding unit 40.

  The subtractor 31 obtains a difference between the supplied original data (image data of the current frame) and the image data of the reference frame generated by the motion compensation unit 38. The DCT unit 32 performs DCT processing on the difference obtained by the subtractor 31 to generate a DCT coefficient. The quantization unit 33 performs quantization on the DCT coefficient generated by the DCT unit 32.

  The inverse quantization unit 34, the inverse DCT unit 35, the adder 36, the frame memory 37, and the motion compensation unit 38 are provided so that the original data supplied to the encoding device 2 can be correctly restored in the image reproduction device 1. It is a thing. First, the inverse quantization unit 34 performs inverse quantization on the data quantized by the quantization unit 33 to restore the DCT coefficients.

  The inverse DCT unit 35 performs inverse DCT processing on the DCT coefficient restored by the inverse quantization unit 34 and generates data corresponding to the difference obtained by the subtractor 31. The adder 36 adds the data obtained by the motion compensation unit 38 performing motion compensation and the data generated by the inverse DCT unit 35 to generate decoded image data.

  The frame memory 37 stores the decoded image data generated by the adder 36 as reference frame image data. Based on the supplied original data and the data stored in the frame memory 37, the motion vector detection unit 39 detects a motion vector mv indicating the direction and position of the image data of the current frame with respect to the image data of the reference frame.

  The motion compensation unit 38 performs motion compensation on the reference frame image stored in the frame memory 37 based on the motion vector mv detected by the motion vector detection unit 39, and supplies the motion compensated data to the subtractor 31. .

  The variable length encoding unit 40 encodes the data quantized by the quantization unit 33 into variable length code data according to the entropy encoding method. The variable length coding unit 40 uses, for example, a Huffman coding method or a run length coding method as an entropy coding method.

  The variable length encoding unit 41 encodes the motion vector mv detected by the motion vector detection unit 39 into a variable length code according to the same encoding scheme as that of the variable length encoding unit 40. The multiplexing unit 42 multiplexes the data encoded by the variable length encoding units 40 and 41, respectively. The encoding device 2 outputs the encoded data multiplexed by the multiplexing unit 42.

  Returning to FIG. 1, the code buffer 11 temporarily stores the encoded data output from the encoding device 2.

  The variable length decoding unit 12 decodes the encoded data temporarily held by the code buffer 11. The variable length decoding unit 12 corresponds to the variable length encoding units 40 and 41 of the encoding device 2 shown in FIG. 3 and restores data including the motion vector mv detected by the motion vector detection unit 39. The variable length decoding unit 12 supplies the motion vector mv to the motion compensation unit 16.

  The inverse quantization unit 13 performs inverse quantization on the data restored by the variable length decoding unit 12 to restore DCT coefficients.

  The inverse DCT unit 14 obtains normal difference information by performing inverse DCT processing on the DCT coefficients restored by the inverse quantization unit 13. The normal difference information is information corresponding to the difference obtained by the subtracter 31 of the encoding device 2.

  The control unit 15 controls each unit of the image reproduction device 1. The control unit 15 is supplied with the slow instruction information by the user. The slow instruction information is information for instructing whether or not to perform slow reproduction. The control unit 15 generates slow control information S1 for controlling each unit in accordance with the throw instruction information. The control unit 15 outputs the generated slow control information S1 to the motion compensation unit 16 and the difference image processing unit 17.

  If the slow instruction information indicates normal playback, the control unit 15 generates slow control information S1 that causes normal playback to be performed. If the slow instruction information indicates slow playback, the control unit 15 generates slow control information S1 indicating the insertion position on the time axis of the interpolated image so as to perform slow playback.

  For example, the normal decoded image k1 in the reference frame as shown in FIG. 4 (a) moves by the motion vector mv and changes to the normal decoded image k2 in the current frame as shown in FIG. 4 (b). And The normal decoded images k1 and k2 are images in which a cross mark is drawn in a 5 × 5 rectangle with a cell indicated by a broken line as 1. In this case, when slow instruction information for instructing slow reproduction is supplied by the user, the control unit 15 generates slow control information S1 for generating an interpolation image for slow reproduction.

  First, the control unit 15 calculates the position on the time axis of the interpolated image for interpolating the normal decoded image based on the supplied slow instruction information. For example, when the slow instruction information for generating a slow image of N (N is an integer satisfying N ≧ 1) times as a slow multiple is supplied, the control unit 15 sets the number of interpolation images to (N−1). Discriminate.

  Next, the control unit 15 calculates the insertion coefficient of the nth interpolated image between the reference frame and the current frame. When the number n of interpolated images is supplied, the control unit 15 calculates the insertion coefficient as n / N so that the insertion coefficient is set to a value from 0 to 1 according to a monotone function.

  Note that the image reproducing apparatus 1 performs slow reproduction in units of frames when the configuration of the code data according to the MPEG2 or MPEG4 encoding scheme is a frame configuration, and in units of fields in the case of a field configuration.

  When calculating the insertion coefficient of the n-th interpolation image, the control unit 15 supplies the slow compensation information S1 indicating the insertion coefficient of the n-th interpolation image to the motion compensation unit 16 and the difference information processing unit 17.

  The motion compensation unit 16 moves the normal decoded image k1 as shown in FIG. 5A stored in the image memory 19 by the motion vector mv supplied from the variable length decoding unit 12, and then moves to FIG. The normal motion compensated image p2 subjected to motion compensation as shown in FIG. The motion compensation unit 16 supplies the generated normal motion compensation image p2 to the adder 18.

  Further, the motion compensation unit 16 generates a motion compensated interpolation image based on the slow control information S1 supplied from the control unit 15. Therefore, the motion compensation unit 16 includes a motion vector calculation unit 16a.

  Based on the slow control information S1 supplied from the control unit 15, the motion vector calculation unit 16a calculates the motion vector mv using the motion vector mv as a normal motion vector, and the frame immediately before the current frame in the playback time The motion vector mv1 for the nth interpolated image g1n counted from the above is obtained.

動き補償部１６は、図５（ｃ）に示すように、参照フレームの正規復号画像ｋ１を、動きベクトル演算部１６ａが求めた動きベクトルｍｖ１分だけ移動して、動き補償補間画像ｐ１ｎを生成する。 The motion vector mv1 is a motion vector for converting the position of p1n as a motion compensated interpolated image for generating the interpolated image g1n, and the motion vector computing unit 16a receives the motion vector mv supplied from the variable length decoding unit 12. On the other hand, a motion vector mv1 as an interpolation motion vector for generating the nth interpolated image is obtained by performing the calculation shown in the following equation (2).

As shown in FIG. 5C, the motion compensation unit 16 moves the normal decoded image k1 of the reference frame by the motion vector mv1 obtained by the motion vector calculation unit 16a to generate a motion compensation interpolated image p1n. .

  In addition, the motion compensation unit 16 supplies the motion vector mv1 obtained by the motion vector calculation unit 16a to the position conversion unit 17b of the difference information processing unit 17.

  The difference information processing unit 17 is supplied with the slow control information S1 from the control unit 15 so as to perform slow reproduction, and generates the difference information q1n for generating the interpolated image g1n. Image processing is performed on the difference information q2. The difference information processing unit 17 includes a pixel value conversion unit 17a and a position conversion unit 17b.

  When the normal decoded image k2 is analyzed as shown in FIG. 6, the normal decoded image k2 shown in FIG. 6B is the same as the output image p2 of the motion compensation unit 16 shown in FIG. 6C and FIG. Is obtained by adding the normal difference information q2 as shown in FIG.

  An output image p2 illustrated in FIG. 6C is an image obtained by the motion compensation unit 16 translating the decoded image k1 of the reference frame as illustrated in FIG. 6A by the motion vector mv. The normal difference information q <b> 2 illustrated in FIG. 6D is normal difference information of the current frame supplied from the inverse DCT unit 14 to the difference information processing unit 17.

  The pixel value conversion unit 17a and the position conversion unit 17b perform image processing on the normal difference information q2 in order to generate difference information q1n for generating an interpolated image g1n.

  First, the pixel value conversion unit 17a converts the pixel value of each pixel of the normal difference information q2 shown in FIG. 6D based on the slow control information S1 supplied from the control unit 15. By performing such pixel value conversion, the pixel value conversion unit 17a generates difference information q1n as illustrated in FIG. 7A as n-th interpolation difference information from the previous normal difference information.

  Specifically, the pixel value conversion unit 17a obtains the pixel value of each pixel of the difference information q1n by multiplying the pixel value of each pixel of the normal difference information q2 by n / N. When the motion vector mv is small and the difference information is large (for example, when the brightness is greatly changed due to the flash when the movement of an object in the image is small), the pixel value conversion unit 17a functions effectively.

  The position conversion unit 17b performs position conversion of the difference information q1n generated by the pixel value conversion unit 17a based on the motion vector mv1 supplied from the motion compensation unit 16 during slow reproduction.

For this reason, the position conversion unit 17b uses the motion vector mv1 supplied from the motion compensation unit 16 and the slow control information s1 supplied from the control unit 15, and performs the calculation shown in the following Equation 3 to obtain the difference information. A motion vector mv2 for performing the position conversion of q1n is obtained.

  The position conversion unit 17b moves the difference information q1n shown in FIG. 7A output from the pixel value conversion unit 17a by the calculated motion vector mv2, and generates the difference information q1n as shown in FIG. 7B. To do. When the motion vector mv is large, it is effective for the difference information q1n to follow and move, and the position conversion unit 17b functions effectively.

  The difference information processing unit 17 outputs the difference information q1n generated by the position conversion unit 17b to the adder 18.

  The adder 18 adds the difference information output from the difference information processing unit 17 and the data of the reference frame image subjected to motion compensation by the motion compensation unit 16 to the original data supplied to the encoding device 2. A corresponding regular decoded image and an interpolation image for slow reproduction are generated. The adder 18 supplies the generated regular decoded image to the image memory 19 and the display buffer 20.

  The image memory 19 stores the normal decoded image generated by the adder 18. The display buffer 20 temporarily stores the normal decoded image and the interpolated image generated by the adder 18. At the time of slow reproduction, the display buffer 20 temporarily stores the last interpolated image to be inserted between the normal decoded image k1 of the reference frame supplied from the adder 18 and the normal decoded image k2 of the current frame. The control information S2 shown is output to the control unit 15.

  The image output unit 21 sets the time so that the image data temporarily stored in the display buffer 20 is slowly reproduced and sequentially outputs the data.

Next, the operation of the image reproduction device 1 according to this embodiment will be described.
When the encoded data including the image data generated by the encoding device 2 shown in FIG. 2 and the motion vector mv is supplied to the image reproducing device 1, the code buffer 11 holds the encoded data.

  The variable length decoding unit 12 decodes the data held in the code buffer 11 and restores the image data and the motion vector mv. The variable length decoding unit 12 supplies the restored motion vector mv to the difference information processing unit 17 and the motion compensation unit 16.

  The inverse quantization unit 13 performs inverse quantization on the data restored by the variable length decoding unit 12 to restore the DCT coefficients.

  The inverse DCT unit 14 obtains normal difference information by performing inverse DCT processing on the DCT coefficients restored by the inverse quantization unit 13. The inverse DCT unit 14 supplies the acquired normal difference information to the difference information processing unit 17.

  When the slow instruction information for instructing normal playback is supplied to the control unit 15, the control unit 15 generates slow control information S1 that causes normal playback to be performed, and this slow control information S1 is used as the difference information processing unit 17. To the pixel value conversion unit 17a and the motion compensation unit 16.

  In this case, the difference information processing unit 17 outputs the normal difference information supplied from the inverse DCT unit 14 to the adder 18 as it is. The motion compensation unit 16 performs motion compensation on the normal decoded image of the reference frame stored in the image memory 19 and outputs this image to the adder 18.

  The adder 18 adds the normal difference information output from the difference information processing unit 17 and the image output from the motion compensation unit 16 to generate a normal decoded image of the current frame. The adder 18 supplies the generated regular decoded image to the image memory 19 and the display buffer 20.

  The image memory 19 stores a new supplied normal decoded image instead of the normal decoded image already stored. The display buffer 20 temporarily stores the supplied normal decoded image, and the image output unit 21 outputs the normal decoded image temporarily stored in the display buffer 20.

  When the slow instruction information for instructing the slow reproduction is supplied to the control unit 15, the control unit 15 generates the slow control information S1 for performing the slow reproduction. If the slow instruction information indicates, for example, playback of 3 (= N) times slow images, the control unit 15 determines that the number of interpolation images is 2 (= N−1), and 2 The insertion position on the time axis of one interpolated image is calculated. Let this interpolated image be g11, g12.

  As shown in FIG. 8, the control unit 15 generates the slow control information S1 in order to generate the regular decoded images k1, k2, and the interpolated images g11, g12. First, the control unit 15 generates slow control information S1 = 1/3 for generating the interpolation image g11. The control unit 15 supplies the slow control information S <b> 1 to the difference information processing unit 17 and the motion compensation unit 16.

  It is assumed that the image memory 19 stores a normal decoded image k1 of a reference frame as shown in FIG. Further, the inverse DCT unit 14 supplies normal difference information q2 of the current frame as shown in FIG. 10A to the difference information processing unit 17.

  The motion vector calculation unit 16a of the motion compensation unit 16 performs the calculation of Equation 2 according to the slow control information S1 supplied from the control unit 15, thereby obtaining the motion vector mv supplied from the variable length decoding unit 12 by 1/3. The motion vector mv1 is obtained by scaling to twice.

  The motion compensation unit 16 moves the normal decoded image k1 shown in FIG. 9A by the motion vector mv1 obtained by the motion vector calculation unit 19a, and obtains a motion compensated interpolated image p11 as shown in FIG. 9B. Generate.

  The motion compensation unit 19 outputs the generated motion compensation interpolation image p11 to the adder 18. The motion vector calculation unit 16 a supplies the motion vector mv1 obtained by the motion vector calculation unit 16 a to the position conversion unit 17 b of the difference information processing unit 17.

  Based on the slow control information S1 supplied from the control unit 15, the pixel value conversion unit 17a of the difference information processing unit 17 multiplies the pixel value of each pixel of the normal difference information q2 illustrated in FIG. And difference information q11 as shown in FIG. 10B is generated. Then, the pixel value conversion unit 17a supplies the acquired difference information q11 to the position conversion unit 17b.

  The position conversion unit 17b obtains the motion vector mv2 by performing calculation according to Equation 3 based on the motion vector mv1 supplied from the motion compensation unit 16. The position conversion unit 17b moves the difference information q11 obtained by the pixel value conversion by the pixel value conversion unit 17a by the calculated motion vector mv2, and generates the difference information q11 as illustrated in FIG. For comparison, FIG. 11A shows the normal difference information q2 of the current frame supplied by the inverse DCT unit 14. The difference information processing unit 17 outputs the generated difference information q11 to the adder 18.

  The adder 18 adds the difference information q11 as shown in FIG. 11 (b) and the motion compensated interpolation image p11 as shown in FIG. 9 (b) to obtain an interpolation image g11 as shown in FIG. 12 (b). Is generated. The adder 18 supplies the generated interpolated image g11 to the display buffer 20. The display buffer 20 temporarily stores the interpolated image g11 supplied from the adder 18, and outputs control information S2 indicating that the generation of the decoded image g11 has been completed to the control unit 15.

  When the control information S2 indicating that the generation of the decoded image g11 is completed is supplied, the control unit 15 generates the slow control information S1 = 2/3 for generating the interpolation image g12. This slow control information S1 indicates the insertion position of the interpolation image g12. The control unit 15 supplies the slow control information S <b> 1 to the motion compensation unit 16 and the difference information processing unit 17.

  The motion vector calculation unit 16a of the motion compensation unit 16 performs the calculation of Equation 2 according to the slow control information S1 supplied from the control unit 15, thereby scaling the motion vector mv to 2/3 times to obtain the motion vector mv1. .

  The motion compensation unit 16 moves the normal decoded image k1 shown in FIG. 9A by the motion vector mv1 obtained by the motion vector calculation unit 19a, and obtains a motion compensated interpolation image p12 as shown in FIG. 9C. Generate.

  The motion compensation unit 16 outputs the generated motion compensation interpolation image p11 to the adder 18. In addition, the motion vector calculation unit 16a supplies the motion vector mv1 obtained by the motion vector calculation unit 16a to the position conversion unit 17b.

  Based on the slow control information S1 supplied from the control unit 15, the pixel value conversion unit 17a of the difference information processing unit 17 doubles the pixel value of each pixel of the normal difference information q2 illustrated in FIG. And difference information q12 as shown in FIG. 10C is generated. Then, the pixel value conversion unit 17a supplies the acquired difference information q12 to the position conversion unit 17b.

  The position conversion unit 17b obtains the motion vector mv2 by performing calculation according to Equation 3 based on the motion vector mv1 supplied from the motion compensation unit 16. The position conversion unit 17b moves the difference information q11 obtained by the pixel value conversion by the pixel value conversion unit 17a by the calculated motion vector mv2, and generates the difference information q12 as illustrated in FIG. The difference information processing unit 17 outputs the generated difference information q12 to the adder 18.

  The adder 18 adds the difference information q12 as shown in FIG. 11 (c) and the motion compensated interpolation image p12 as shown in FIG. 9 (c) to obtain an interpolation image g12 as shown in FIG. 12 (c). Is generated. The adder 18 supplies the generated interpolated image g12 to the display buffer 20. The display buffer 20 temporarily stores the interpolated image g12 supplied from the adder 18, and outputs control information S2 indicating that the generation of the interpolated image g12 has been completed to the control unit 15.

  When the control information S2 indicating that the generation of the interpolation image g12 has been completed is supplied, the control unit 15 generates the slow control information S1 = 3/3 for generating the normal decoded image k2. The control unit 15 supplies the generated slow control information S <b> 1 to the motion compensation unit 16 and the difference information processing unit 17.

  The motion vector calculation unit 16a of the motion compensation unit 16 performs the calculation of Equation 2 according to the slow control information S1 supplied from the control unit 15, thereby scaling the motion vector mv to 3/3 times to obtain the motion vector mv1. .

  The motion compensation unit 19 moves the normal decoded image k1 shown in FIG. 9A by the motion vector mv1 obtained by the motion vector calculation unit 19a, and obtains a normal motion compensation image p2 as shown in FIG. 9D. Generate.

  The motion compensation unit 19 outputs the generated normal motion compensation image p2 to the adder 18. In addition, the motion vector calculation unit 16a supplies the motion vector mv1 obtained by the motion vector calculation unit 16a to the position conversion unit 17b.

  Based on the slow control information S1 supplied from the control unit 15, the pixel value conversion unit 17a of the difference information processing unit 17 increases the pixel value of each pixel of the normal difference information q2 illustrated in FIG. And normal difference information q2 as shown in FIG. 10 (d) is generated. Then, the pixel value conversion unit 17a supplies the acquired normal difference information q2 to the position conversion unit 17b.

  Similarly, the position conversion unit 17b calculates the motion vector mv2 by performing the calculation according to Equation 3 based on the supplied motion vector mv1. The position conversion unit 17b moves the difference information q11 obtained by the pixel value conversion by the pixel value conversion unit 17a by the calculated motion vector mv2, and generates normal difference information q2 as illustrated in FIG. . The difference information processing unit 17 outputs the generated normal difference information q2 to the adder 18.

  The adder 18 adds the normal difference information q2 as shown in FIG. 11 (d) and the motion compensated interpolation image p2 as shown in FIG. 9 (d), and performs normal decoding as shown in FIG. 12 (d). An image k2 is generated. The adder 18 supplies the generated normal decoded image k2 to the display buffer 20. The display buffer 20 temporarily stores the normal decoded image k2 supplied from the adder 18 and outputs to the control unit 15 control information S2 indicating that the generation of the normal decoded image k2 has been completed.

  The display buffer 20 temporarily stores the interpolated image k2 supplied from the adder 18, and outputs to the control unit 15 control information S2 indicating that the generation of the normal decoded image k2 has been completed. The image memory 18 updates the already stored normal decoded image k1 with the normal decoded image k2.

  The image output unit 21 inserts the interpolated images g11 and g12 between the normal decoded image k1 and the normal decoded image k2, sets the time axis so that slow reproduction is performed, and the display buffer 20 temporarily holds the image. Output data.

  In this way, the image reproduction device 1 sequentially outputs the normal decoded image k1, the interpolated images g11 and g12, and the normal decoded image k2 generated as shown in FIGS. 12 (a) to 12 (d). Thereby, the position of the image and the pixel value are changed to perform slow reproduction, and a smoother slow image is reproduced.

  As described above, according to the present embodiment, the difference information processing unit 17 performs pixel value conversion and position conversion on the normal difference information q2 supplied from the inverse DCT unit 14.

  Accordingly, during slow playback, the position of the image changes smoothly and the pixel value also changes smoothly, so that distortion during slow playback can be suppressed, and the slow image can be played back more smoothly.

  In addition, the control unit 15 inserts the nth interpolated image between the previous frame and the current frame in the current frame playback time so as to take a value between 0 and 1 according to a monotone function. Since the slow image does not change suddenly, the slow playback can be performed more smoothly.

In carrying out the present invention, various forms are conceivable and the present invention is not limited to the above embodiment.
In the above embodiment, the difference information processing unit 17 is configured such that the position conversion unit 17b converts the position of the difference information after the pixel value conversion unit 17a performs the pixel value conversion on the difference information. However, the configuration of the difference information processing unit 17 is not limited to this. For example, the pixel value conversion unit 17a may be connected after the position conversion unit 17b and may perform pixel value conversion after performing position conversion on the normal difference information q2.

  Further, the difference information processing unit 17 may output the difference information processed by either one of the pixel value conversion unit 17a and the position conversion unit 17b.

  When only the pixel value conversion unit 17a converts the pixel value of each pixel of the difference information, the control unit 15 supplies n / N as the slow control information S1 to the pixel value conversion unit 17a.

  Further, the position conversion unit 17b sets the motion vector mv2 to 0 regardless of the motion vector mv1 supplied from the motion compensation unit 16. If it does in this way, the image reproduction apparatus 1 will output the normal decoded image k1, the interpolation images g11 and g12, and the normal decoded image k2 as shown to Fig.13 (a)-(d).

  When only the position conversion unit 17b converts the pixel value of each pixel of the difference information, the control unit 15 supplies the slow control information S1 = 1 to the pixel value conversion unit 17a.

  Further, the position conversion unit 17b moves the difference information q11, q12 with constant pixel value by the calculated motion vector mv2, and the difference information q11 with constant pixel value as shown in FIGS. , Q12 and normal difference information q2.

  The image reproduction device 1 outputs a normal decoded image k1, interpolation images g11 and g12, and a normal decoded image k2 as shown in FIGS. 15 (a) to 15 (d). In this way, slow playback can be performed.

  In the above-described embodiment, 3 × slow playback has been described as an example. However, it is also possible to perform slow playback of 2 times or 4 times or more.

  In the above-described embodiment, the N slow instruction information is designated by the user by designating a multiple N of slow reproduction. However, if N for designating a multiple of slow playback is constant, slow playback may be performed simply by instructing slow playback.

  The difference information may be in units of pixels or may indicate a representative value obtained by collecting several pixels, for example, four pixels.

  Further, instead of setting S1 = n / N for the slow control information, for example, S1 = (n * n) / (N * N) may be used.

  In the above embodiment, the position conversion unit 17b calculates the position conversion motion vector mv2 based on the motion vector mv1 calculated by the motion vector calculation unit 16a. However, the motion vector calculation unit 16a may obtain the motion vectors mv1 and mv2 and supply them to the position conversion unit 17b.

It is a block diagram which shows the structure of the image reproduction apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows the content of motion compensation. It is a block diagram which shows the structure of the encoding apparatus which produces | generates encoding data. It is explanatory drawing which shows the normal decoding image of a reference frame, and the normal decoding image of the present frame. It is explanatory drawing which shows operation | movement of the motion compensation part of FIG. It is explanatory drawing which shows the analysis content of the image of the present flame | frame. 2A and 2B are explanatory diagrams illustrating an operation of a difference information processing unit in FIG. 1, in which FIG. 1A illustrates an operation of a pixel value conversion unit, and FIG. 2B is an explanatory diagram illustrating an operation of a position conversion unit; It is explanatory drawing which shows operation | movement of the image reproduction apparatus of FIG. (A) is explanatory drawing which shows the normal decoded image of the reference frame which an image memory memorize | stores, (b)-(d) is explanatory drawing which shows a series of output images of the motion compensation part of FIG. (A) shows the normal difference information of the current frame that the inverse DCT unit of FIG. 1 supplies to the difference information processing unit, and (b) to (d) show the output information of the pixel value conversion unit of FIG. It is explanatory drawing. (A) shows the normal difference information of the current frame supplied to the difference information processing unit by the inverse DCT unit of FIG. 1, and (b) to (d) are explanatory diagrams showing the output information of the position conversion unit of FIG. It is. It is explanatory drawing which shows the output image of an image reproduction apparatus. It is explanatory drawing which shows a series of output images of the image reproduction apparatus when the position conversion part of FIG. 1 does not perform position conversion. It is explanatory drawing which shows a series of output information of the difference information processing part in case the pixel value conversion part of FIG. 1 does not perform pixel value conversion. It is explanatory drawing which shows a series of output images of the image reproduction apparatus when the pixel value conversion part of FIG. 1 does not perform pixel value conversion.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image reproduction apparatus, 2 ... Encoding apparatus, 15 ... Control part, 16 ... Motion compensation part, 16a ... Motion vector calculating part, 17 ... Difference information processing part, 17a ... Pixel value converter, 17b ... Position converter, 18 ... Adder, 19 ... Image memory, 20 ... Display buffer, 23 ... Image output unit

Claims (7)

In an image reproducing apparatus for decoding an encoded data including difference information indicating a difference between a current image and an image subjected to motion compensation, and a motion vector for performing the motion compensation, and reproducing an image,
A decoding unit for decoding the encoded data;
An interpolation motion vector generation unit that generates an interpolation motion vector for an interpolation image used for slow reproduction from the normal motion vector, using the motion vector decoded by the decoding unit as a normal motion vector;
Using the difference information decoded by the decoding unit as normal difference information, based on the preset slow multiple and the interpolation motion vector acquired by the interpolation motion vector generation unit, performs image processing on the normal difference information, An interpolation difference information generating unit for generating interpolation difference information for generating the interpolation image;
An interpolated image that generates the interpolated image by adding the motion compensated interpolated image subjected to motion compensation based on the interpolated motion vector generated by the interpolated motion vector generating unit and the interpolated difference information generated by the interpolated differential information generating unit A generator,
Inserting the interpolated image generated by the interpolated image generation unit before and after the normal decoded image generated based on the normal difference information and the normal motion vector, and setting the time so as to be played slowly, the normal decoding An image output unit that sequentially outputs the image and the interpolated image;
An image reproducing apparatus characterized by that. The interpolation difference information generation unit is configured to perform pixel processing for each pixel value of the normal difference information based on the slow multiple so that the pixel value changes between the normal difference information before and after the image processing for the normal difference information. Performing value conversion and generating the interpolation difference information;
The image reproducing apparatus according to claim 1, wherein: The interpolation difference information generation unit performs position conversion on the normal difference information based on the interpolation motion vector generated by the interpolation motion vector generation unit as image processing for the normal difference information, and generates the interpolation difference information. To
The image reproducing apparatus according to claim 1, wherein: The interpolation difference information generating unit sets each of the nth interpolation difference information from the previous normal difference information among (N−1) sheets inserted between the preceding and following normal difference information, where N is a slow multiple. The n-th interpolation difference information is generated by determining the pixel value of each of the subsequent normal difference information by multiplying the pixel value by n / N.
The image reproducing apparatus according to claim 2, wherein: The interpolation difference information generation unit obtains a position conversion motion vector for performing position conversion on the normal difference information from the normal motion vector according to Equation 1, and based on the obtained position conversion motion vector, the normal difference Performing position conversion on the information to generate the interpolation difference information;
The image reproduction apparatus according to claim 3, wherein

The interpolation motion vector generation unit sets the slow multiple to N, and among the (N−1) images inserted between the preceding and succeeding normal decoded images, interpolation for generating the nth interpolated image from the previous normal decoded image A motion vector is generated by multiplying the normal motion vector by n / N,
The image reproduction apparatus according to claim 1, wherein the image reproduction apparatus is an image reproduction apparatus. Decoding encoded data including difference information indicating a difference between a current image and an image subjected to motion compensation, and a motion vector for performing the motion compensation;
Using the decoded motion vector as a normal motion vector, generating an interpolation motion vector for an interpolated image used for slow reproduction from the normal motion vector;
Using the decoded difference information as normal difference information, image processing is performed on the normal difference information based on a preset slow multiple and the generated interpolation motion vector, and the interpolation difference information for generating the interpolation image is obtained. Generating step;
Adding the motion compensated interpolated image subjected to motion compensation based on the generated interpolated motion vector and the generated interpolation difference information to generate the interpolated image;
Inserting the generated interpolated image before and after the normal decoded image generated based on the normal difference information and the normal motion vector, and setting the time so as to be played back slowly, the normal decoded image and the interpolated image And sequentially outputting
An image reproduction method characterized by the above.

Images