JP5493362B2 - Movie playback apparatus and program - Google Patents

Description

  The present invention relates to a moving image playback apparatus and a program executed by a computer of the moving image playback apparatus.

  In recent years, when digital moving image data is transmitted / stored, it is compressed to several tenths using a digital moving image compression / decompression technique to reduce the load on the transmission rate and storage capacity. As such a digital moving image compression / decompression technique, various methods such as MPEG1 / MPEG2 / MPEG4 dedicated to moving images and Motion JPEG that handles still images as continuous frames are used.

  By the way, in the process of decoding the encoded moving image data and generating a plurality of image frames, the decoding time may be different for each image frame. That is, when moving image data is encoded by performing motion compensation, the load on the CPU differs for each image frame due to a change in the number of macroblock divisions during decoding. For this reason, an image frame in which decoding does not end by a predetermined reproduction display timing occurs, and frame dropping becomes remarkable.

  Therefore, in order to solve such a problem, the real-time recording / playback apparatus using the digital moving image compression / decompression process described in Patent Document 1 has a frame rate control unit so that the process can be made in time during compression. If there is not, interpolation processing is performed. If the processing is not in time for decompression, frame drop processing is performed. It is disclosed.

JP 2001-309303 A

  However, according to the technique disclosed in Patent Document 1 as described above, the interpolation process at the time of compression is substantially a thinning process. As a result, the frame rate is lowered, so that the frame rate is reduced by dropping frames. Only the real-time property of the processing is ensured by lowering, and there is a problem that the displayed moving image becomes a non-smooth image.

  Accordingly, an object of the present invention is to provide a moving image reproducing apparatus and a program that can smoothly reproduce a moving image.

In order to solve the above-mentioned problem, the invention according to claim 1 is characterized in that recording means for recording encoded moving image data in which a plurality of continuous image frames are encoded, and decoding the encoded moving image data to generate the plurality of pieces Decoding means for generating image frames, reproduction means for switching and reproducing the plurality of image frames generated by the decoding means at a predetermined frame rate, and a predetermined number of consecutive image frames among the plurality of image frames A counting means for counting the number of delays for which the decoding time required for generating each image frame by the decoding means is equal to or longer than the reproduction time of each image frame according to the predetermined frame rate by the reproduction means for the group; Number determination means for determining whether or not the number of delays counted by the counting means is equal to or greater than a predetermined number, and the delay by the number determination means When it is determined that the number of images exceeds a predetermined number, two image frames generated by decoding by the decoding unit are specified from the image frame group with a predetermined interval, and between the two image frames An interpolated image generating means for generating an interpolated image frame corresponding to one image frame to be reproduced, and an interpolated image frame generated by the interpolated image generating means is inserted between the two image frames to be reproduced by the reproducing means. And a playback control means for providing a moving image playback apparatus.

The interpolated image generating means generates the interpolated image frame when the reproducing means reproduces the plurality of image frames at a frame rate faster than the predetermined frame rate. The moving picture reproducing apparatus according to claim 1 is provided.

According to a third aspect of the present invention, the interpolated image generating means generates an intermediate image frame or a morphed image frame of the two image frames as the interpolated image. A playback device is provided.

According to a fourth aspect of the present invention, the interpolated image generating means weights the number according to the number of image frames between the two image frames and according to a relative reproduction position with respect to the two image frames. 4. The moving image reproducing apparatus according to claim 3, wherein the interpolated image frame is generated instead.

According to a fifth aspect of the present invention, there is provided imaging means for imaging one scene and generating a plurality of continuous image frames, and encoding moving image data by encoding the plurality of image frames generated by the imaging means. The video reproduction apparatus according to any one of claims 1 to 4, further comprising: an encoding unit that generates the video.

According to a sixth aspect of the present invention, there is provided a computer of a moving image reproducing apparatus provided with a recording unit that records encoded moving image data in which a plurality of continuous image frames are encoded. Decoding means for generating image frames, and for a predetermined number of consecutive image frame groups among the plurality of image frames, the decoding time required for generating each image frame by the decoding means Counting means for counting the number of delays equal to or longer than the playback time of each image frame corresponding to the predetermined frame rate by the playback means for switching and reproducing at a rate, and the delay number counted by the counting means is equal to or greater than a predetermined number Number determining means for determining whether or not the number of delays has been determined by the number determining means to be greater than or equal to a predetermined number An interpolated image frame corresponding to one image frame scheduled to be reproduced between the two image frames by identifying two image frames generated by decoding by the decoding means from the image frame group with a predetermined interval. Interpolated image generating means for generating the interpolated image frame, and a reproducing control means for inserting the interpolated image frame generated by the interpolated image generating means between the two image frames and causing the reproducing means to reproduce the program. Is to provide.

  According to the present invention, a moving image can be smoothly reproduced.

It is a block diagram which shows schematic structure of the moving image reproducing device of Embodiment 1 to which this invention is applied. 6 is a flowchart illustrating an example of an operation related to a moving image decoding process performed by the moving image reproducing apparatus in FIG. 1. It is a figure for demonstrating the moving image decoding process of FIG. It is a block diagram which shows schematic structure of the moving image reproduction apparatus of Embodiment 2 to which this invention is applied. It is a figure for demonstrating the moving image decoding process by the moving image reproducing device of FIG. 5 is a flowchart illustrating an example of an operation related to a moving image decoding process performed by the moving image reproducing device in FIG. 4. It is a figure for demonstrating the moving image decoding process of FIG. It is a block diagram which shows schematic structure of the moving image reproducing device of Embodiment 3 to which this invention is applied. It is a flowchart which shows an example of the operation | movement which concerns on the moving image reproduction process by the moving image reproducing device of FIG. It is a figure for demonstrating the moving image reproduction process of FIG.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
FIG. 1 is a block diagram showing a schematic configuration of a moving image playback apparatus 100 according to the first embodiment to which the present invention is applied.
The moving image reproducing apparatus 100 according to the first embodiment generates an interpolated image frame corresponding to one image frame scheduled to be reproduced between two image frames out of a plurality of image frames constituting moving image data in MPEG format, and performs decoding. When the decoding time required to generate one image frame by the unit 6a is equal to or longer than the reproduction time of each image frame corresponding to a predetermined reproduction frame rate by the display unit 5, the interpolated image frame is replaced with one image frame. To reproduce and display on the display unit 5.
Specifically, as shown in FIGS. 1 and 2, the moving image playback apparatus 100 includes a CPU 1, a memory 2, a recording medium 3, a display control unit 4, a display unit 5, a CODEC 6, and an operation input unit. 7.

The recording medium 3 is composed of, for example, a non-volatile memory (flash memory) or the like, and records JPEG format still image data or MPEG format moving image data encoded by the CODEC 6.
Here, the recording medium 3 constitutes recording means for recording encoded moving image data in which a plurality of continuous image frames are encoded in the MPEG format.

  The display control unit 4 performs control to read display image data temporarily stored in the memory 2 and display the display image data on the display unit 5. Specifically, the display control unit 4 includes a VRAM, a VRAM controller, a digital video encoder, and the like. The digital video encoder periodically reads the luminance signal Y and the color difference signals Cb and Cr read from the memory 2 and stored in the VRAM (not shown) under the control of the CPU 1 from the VRAM via the VRAM controller. And a video signal is generated based on these data and output to the display unit 5.

Based on the video signal from the display control unit 4, the display unit 5 converts a plurality of image frames generated by decoding the moving image data in the MPEG format recorded on the recording medium 3 by the CODEC 6 into predetermined playback frames. Play by switching at the rate.
Here, the display unit 5 constitutes reproduction means for switching and reproducing a plurality of image frames generated by decoding by the decoding unit 6a at a predetermined frame rate.

  The CODEC 6 performs decoding (decoding) according to a predetermined encoding method under the control of the CPU 1. Specifically, the CODEC 6 includes a decoding unit 6a, a time determination unit 6b, and a control unit 6c.

When the image is reproduced, the decoding unit 6a decodes the encoded data read from the recording medium 3 and stored in the memory 2 to generate display image data. Specifically, the decoding unit 6a decodes moving image data encoded in the MPEG format stored in the memory 2, and generates a plurality of image frames.
Here, the decoding unit 6a constitutes decoding means for decoding the encoded moving image data and generating a plurality of image frames.

  The time determination unit 6b serves as a time determination unit, for each of a plurality of image frames generated by the decoding unit 6a, the decoding time required to generate each image frame depends on a predetermined playback frame rate by the display unit 5. It is determined whether or not the reproduction time of each image frame has been reached.

The control unit 6 c includes a CPU that controls each unit of the CODEC 6.
In addition, when the decoding unit 6a decodes each of the plurality of image frames, the control unit 6c uses, as an interpolated image frame, a pixel value between the preceding frame decoded in advance by the decoding unit 6a and the immediately preceding image frame. Are averaged to generate an intermediate image frame. The control unit 6c may generate a morphing image frame between the preceding frame and the immediately preceding image frame as the interpolation image frame.
Here, the control unit 6c generates an interpolated image frame corresponding to one image frame to be reproduced between the two image frames from the two image frames generated by the decoding unit 6a among the plurality of image frames. The interpolated image generating means is configured.

In addition, when the time determination unit 6b determines that the decoding time of any one of the plurality of image frames is equal to or longer than the reproduction time, the control unit 6c converts the generated interpolated image frame into the one interpolated image frame. A display control signal to be reproduced and displayed on the display unit 5 is output to the display control unit 4 instead of the image frame.
When the display control signal output from the control unit 6c is input, the display control unit 4 reproduces and displays the interpolated image frame on the display unit 5 in accordance with the control signal.
Here, the control unit 6c and the display control unit 4 replace the generated interpolated image frame with the one image frame in the display unit 5 when the time determination unit 6b determines that the decoding time is equal to or longer than the reproduction time. Reproduction control means for reproducing and displaying is configured.

  The operation input unit 7 is for performing a predetermined operation of the moving image playback apparatus 100. Specifically, although not shown in the figure, the operation input unit 7 includes a menu button related to a display instruction of a menu screen related to function selection, display setting, etc., a selection determination button related to selection of an operation mode, etc., and a determination instruction. I have. And the operation input part 7 outputs a predetermined operation signal to CPU1 according to operation of these buttons.

  The memory 2 is composed of, for example, a flash memory and temporarily stores data processed by the CPU 1 and the CODEC 6.

  The CPU 1 controls each part of the moving image playback apparatus 100. Specifically, the CPU 1 performs various control operations according to various processing programs for the moving image playback apparatus 100.

"
Next, video decoding processing by the video playback device 100 will be described with reference to FIGS.
FIG. 2 is a flowchart showing an example of an operation related to the moving picture decoding process, and FIG. 3 is a diagram for explaining the moving picture decoding process. In FIG. 3, a frame displayed as an interpolation image frame on the display unit 5 is schematically represented by dots.

The video decoding process is executed when a playback mode is selected based on a predetermined operation of the operation input unit 7 by the user, and moving image data encoded in the MPEG format on the recording medium 3 is selected as a playback display target. It is processing.
As shown in FIG. 2, when the control unit 6c of the CODEC 6 reads MPEG format moving image data recorded on the recording medium 3, it outputs a predetermined control signal to the decoding unit 6a, The first image frame (first frame) is decoded (step S1). Further, the control unit 6c transfers the decoded head frame to the memory 2 and temporarily stores it.

Next, the control unit 6c determines whether or not there is a next encoded frame that forms the moving image data in the MPEG format and continues to the first frame (step S2).
If it is determined that the next encoded frame exists (step S2; YES), the control unit 6c determines whether the encoded frame has not been decoded yet (whether it has not been decoded) (step S3). .

If it is determined that the image has not been decoded yet (step 3; YES), the control unit 6c determines whether or not there is an image frame (decoded preceding frame) that is decoded prior to the encoded frame. (Step S4).
Here, when it is determined that there is no decoded preceding frame (step S4; NO), the control unit 6c outputs a predetermined control signal to the decoding unit 6a, and images at predetermined intervals ahead are determined. Frame decoding is started (step S5). Then, when the decoding of the image frame (preceding frame) ahead by a predetermined interval by the decoding unit 6a is completed (step S6), a state in which a previously decoded preceding frame is prepared is obtained.

Thereafter, the control unit 6c outputs a predetermined control signal to the decoding unit 6a to start decoding of the next encoded frame continuous to the first frame (step S7), and in parallel with the decoding process, the preceding frame And an interpolated image frame between the previous image frame (first frame) (step S8).
If it is determined in step S4 that a decoded preceding frame exists (step S4; YES), the control unit 6c shifts the process to step S7.

Subsequently, the control unit 6c causes the time determination unit 6b to determine whether or not the decoding of the next encoded frame has been completed within the reproduction time (reproduction frame rate time) of each image frame corresponding to a predetermined reproduction frame rate. (Step S9).
If it is determined that decoding of the next encoded frame is completed within the playback frame rate time (step S9; YES), the control unit 6c uses the decoded image frame (decoded image) as the next playback image frame. ) Is adopted (step S10). If it is determined in step S3 that the next encoded frame is not undecoded (step S3; YES), for example, if the decoded frame has already been decoded as in the preceding frame, the control unit 6c performs processing. To step S7.
On the other hand, when it is determined that the decoding of the next encoded frame has not been completed within the playback frame rate time (step S9; NO), the decoding is played back as in “NO.3 frame” (see FIG. 3), for example. If it is not completed within the frame rate time, the control unit 6c adopts the interpolated image frame (interpolated image; “NO.3 (interpolated) frame”) as the next reproduced image frame and is executing The decoding process (decoding process of “NO.3 frame”) is stopped (step S11).

Thereafter, the control unit 6c shifts the process to step S2, and repeats the above process until it is determined in step S2 that the next encoded frame does not exist (step S2; NO). All the plurality of image frames constituting the moving image data in the MPEG format are decoded.
Thereby, the moving image decoding process is terminated.

As described above, according to the moving image reproducing apparatus 100 of the first embodiment, interpolation corresponding to one image frame scheduled to be reproduced between two image frames among a plurality of image frames constituting moving image data in MPEG format. The image frame is generated in parallel with the decoding process of the image frame by the decoding unit 6a. Specifically, an intermediate image frame is generated by averaging the pixel values between the preceding frame decoded in advance by the decoding unit 6a and the immediately preceding image frame.
When the decoding time required to generate one image frame by the decoding unit 6a becomes equal to or longer than the reproduction time (reproduction frame rate time) of each image frame corresponding to a predetermined reproduction frame rate by the display unit 5, interpolation is performed. The image frame is reproduced and displayed on the display unit 5 instead of one image frame.
Therefore, for each of a plurality of image frames, even when an image frame that does not finish decoding by the predetermined playback display timing occurs, the interpolated image frame is reproduced instead of the image frame. By displaying, a plurality of image frames can be reproduced and displayed at a constant reproduction frame rate, and the reproduction of moving images can be made smooth.

[Embodiment 2]
Hereinafter, the moving image reproducing apparatus 200 according to the second embodiment will be described with reference to FIGS.
As shown in FIGS. 4 to 7, the moving image reproduction apparatus 200 according to the second embodiment is configured to process each image by the decoding unit 6a for a predetermined number of image frames among a plurality of image frames constituting moving image data in MPEG format. When the decoding time required for frame generation is equal to or greater than the predetermined number of delay decodes that are equal to or greater than the reproduction time of each image frame corresponding to a predetermined reproduction frame rate by the display unit 5, the subsequent image frame group Two image frames are identified from each other with a predetermined interval, an interpolated image frame corresponding to one image frame to be reproduced is generated between the two image frames, and the interpolated image frame is inserted between the two image frames. Inserted and reproduced and displayed on the display unit 5.
Note that the video playback device 200 of the second embodiment has substantially the same configuration as the video playback device 100 of the first embodiment except that the configuration of the CODEC 206 is different, and the description thereof is omitted.

FIG. 4 is a block diagram showing a schematic configuration of a moving image playback apparatus 200 according to the second embodiment to which the present invention is applied.
As shown in FIG. 4, the CODEC 206 includes a counting unit 6e, a number determining unit 6f, and a control unit 6g in addition to the decoding unit 6a and the time determining unit 6b.

The counting unit 6e includes a counter, and when the decoding unit 6a of the CODEC 206 decodes a fixed number of consecutive image frames among a plurality of image frames, the decoding time required for generating each image frame by the decoding unit 6a However, the number of delay decodes that have reached the playback time of each image frame corresponding to a predetermined playback frame rate by the display unit 5 is counted.
Here, the counting unit 6e constitutes counting means for counting the number of delayed decodes in which the decoding time required for generating each image frame by the decoding unit 6a is equal to or longer than the reproduction time of each image frame for a predetermined number of image frame groups. doing.

  The number determination unit 6f, as number determination means, determines whether or not the number of delayed decodes counted by the counting unit 6e is equal to or greater than a predetermined number.

When the number determination unit 6f determines that the number of delayed decodes is equal to or greater than a predetermined number as an interpolation image generation unit, the control unit 6g generates two decoded images generated by the decoding unit 6a from the image frame group. An image frame is specified at a predetermined interval, and an interpolated image frame corresponding to one image frame to be reproduced is generated between the two image frames.
Specifically, the control unit 6g changes the weighting according to the number of image frames between two image frames as an interpolated image frame and the relative reproduction position with respect to the two image frames. Generate a morph image frame. For example, as illustrated in FIG. 5, the control unit 6g transmits two image frames (“NO.XX1 decode frame” and “NO.XX4 (preceding) decode frame”) to be reproduced between the two image frames. Two image frames (“NO.XX2 morphing frame” and “NO.XX3 morphing frame”) are generated. At this time, for “NO.XX2 morphing frame”, the weighting is changed so that the weight value of “NO.XX1 decoding frame” is larger than that of “NO.XX4 (preceding) decoding frame”. For “.XX3 morphing frame”, the weighting is changed so that the weight value of “NO.XX4 (preceding) decoding frame” is larger than that of “NO.XX1 decoding frame”.

  Further, the control unit 6g, when a fast-forward reproduction is instructed based on a predetermined operation of the operation input unit 7 by the user, that is, the display unit 5 displays a plurality of image frames at a frame rate faster than a predetermined reproduction frame rate. When replaying with, an interpolation image frame is generated.

Further, the control unit 6 g outputs a display control signal for inserting the generated interpolated image frame between the two image frames and causing the display unit 5 to reproduce and display the display control signal.
When the display control signal output from the control unit 6g is input, the display control unit 4 reproduces and displays the interpolated image frame on the display unit 5 in accordance with the control signal.
Here, the control unit 6g and the display control unit 4 constitute a reproduction control unit that inserts the generated interpolated image frame between the two image frames and reproduces and displays them on the display unit 5.

Next, video decoding processing by the video playback device 200 will be described with reference to FIGS. 6 and 7.
FIG. 6 is a flowchart illustrating an example of an operation related to the moving image decoding process, and FIG. 7 is a diagram for describing the moving image decoding process. In FIG. 7, a frame displayed as an interpolation image frame on the display unit 5 is schematically represented by dots.

The following video decoding process is a process executed when fast-forward playback is instructed based on a predetermined operation of the operation input unit 7 by the user.
As shown in FIG. 6, first, the CODEC 206 decodes a fixed number of continuous image frame groups among a plurality of image frames constituting MPEG moving image data recorded on the recording medium 3.
Specifically, when the control unit 6g of the CODEC 206 reads MPEG format moving image data recorded on the recording medium 3, the control unit 6g outputs a predetermined control signal to the decoding unit 6a to configure the moving image data. Of the plurality of image frames, the first image frame (first frame) of a predetermined number of consecutive image frame groups is decoded (step S21). Further, the control unit 6g transfers the decoded leading frame to the memory 2 and temporarily stores it.

Next, the control unit 6g determines whether or not there is a next encoded frame that forms the moving image data in the MPEG format and continues to the first frame (step S22).
If it is determined that the next encoded frame exists (step S22; YES), the control unit 6g outputs a predetermined control signal to the decoding unit 6a to start decoding the next encoded frame ( Step S23). When the decoding of the next encoded frame by the decoding unit 6a is completed (step S24), the control unit 6g performs the decoding within the reproduction time (reproduction frame rate time) of each image frame corresponding to a predetermined reproduction frame rate. The time determination unit 6b determines whether or not the process is completed (step S25).

When it is determined in step S25 that the decoding has not been completed within the playback frame rate time (step S25; NO), the control unit 6g causes the counting unit 6e to send a decoding time determined by the display unit 5 to a predetermined time. The number of delay decodes indicating that the playback time of each image frame corresponding to the playback frame rate is equal to or longer is counted (step S26).
Thereafter, the control unit 6g determines whether or not a decoding process has been performed on a fixed number of consecutive image frame groups among the plurality of image frames (step S27). Even when it is determined in step S25 that the decoding is completed within the playback frame rate time (step S25; YES), the control unit 6g shifts the process to step S27.

  If it is determined in step S27 that the predetermined number of image frame groups are not decoded (step S27; NO), the control unit 6g shifts the process to step S22, and performs the above process in step S27. The process is repeated until it is determined that a certain number of image frame groups have been decoded (step S27; YES). Thereby, the CODEC 6 decodes all the fixed number of image frame groups.

When it is determined in step S27 that the decoding process has been performed for a certain number of image frame groups (step S27; YES), the control unit 6g causes the number determination unit 6f to perform the delay decoding counted by the counting unit 6e. It is determined whether or not the number is equal to or greater than a predetermined number (step S28).
Here, when it is determined that the number of delay decodes is not equal to or greater than the predetermined number (step S28; NO), the control unit 6g moves the process to step S22, and the above process is performed in step S22. This is repeated until it is determined that no encoded frame exists (step S22; NO).

  On the other hand, when it is determined in step S28 that the number of delayed decodes is equal to or greater than the predetermined number (step S28; YES), the control unit 6g determines whether there is a next encoded frame in a certain number of image frame groups. Is determined (step S29).

If it is determined in step S29 that the next encoded frame exists (step S29; YES), the control unit 6g determines whether it is the timing to insert the interpolated image frame (step S30). In the video decoding process of the second embodiment, every time the image frame decoding process is performed twice, one interpolation image frame (for example, “NO.XX3 (interpolation) frame”) is generated and inserted. It shall be (see FIG. 7).
Here, when it is determined that it is not the timing to insert the interpolated image frame (step S30; NO), the control unit 6g outputs a predetermined control signal to the decoding unit 6a to decode the next encoded frame. Start (step S31). When the decoding of the next encoded frame by the decoding unit 6a is completed (step S32), the control unit 6g shifts the process to step S29.

On the other hand, if it is determined in step S30 that it is the timing to insert the interpolated image frame (step S30; YES), that is, the image frame is decoded twice to execute the interpolated image frame (for example, “NO.XX3”). When it is time to generate and insert one (interpolation) frame "), the control unit 6g generates an interpolated image frame between the immediately preceding image frame and the preceding frame (step S33).
Specifically, after the control unit 6g outputs a predetermined control signal to the decoding unit 6a and decodes an image frame (previous frame; “NO.XX4 frame”) ahead of a predetermined interval, Then, an interpolated image frame between the preceding frame and the immediately preceding image frame (“NO.XX2 frame”) is generated.
After generating the interpolated image frame, the control unit 6g shifts the process to step S29.

The above processing is repeated until it is determined in step S29 that the next encoded frame does not exist (step S29; NO).
Thereby, the moving image decoding process is terminated.

As described above, according to the moving image playback device 200 of the second embodiment, the decoding unit 6a generates each image frame for a predetermined number of image frames among a plurality of image frames constituting MPEG moving image data. When the number of delay decodes indicating that the decoding time required for the display unit 5 is equal to or greater than the reproduction time of each image frame corresponding to the predetermined reproduction frame rate by the display unit 5 is equal to or greater than the predetermined number, Two image frames are identified from each other with a predetermined interval, an interpolated image frame corresponding to one image frame to be reproduced is generated between the two image frames, and the interpolated image frame is inserted between the two image frames. Inserted and reproduced and displayed on the display unit 5.
That is, in the case of fast-forward playback of a moving image, there is a possibility that an image frame that does not finish decoding by the predetermined playback display timing may occur. Therefore, a predetermined number of image frame groups among a plurality of image frames are decoded in advance. The number of delay decodes for which the decoding time by the unit 6a is equal to or greater than the playback frame rate time is calculated, and the delay degree of the decoding processing of the entire moving image data is taken into account from the number of delay decodes, and the subsequent image frame group Two image frames are sequentially identified with a predetermined interval. Then, by interpolating interpolated image frames sequentially between the two image frames for playback and display, a plurality of image frames can be played back and displayed at a constant playback frame rate, thereby smoothing the playback of moving images. be able to.

  In addition, when generating an interpolated image frame, a morphing image frame is generated with a number corresponding to the number of image frames between the two image frames and with different weights according to the relative reproduction positions with respect to the two image frames. Therefore, even when a plurality of interpolated image frames are inserted between two image frames, the interpolated image frame can be a natural image that changes along the flow of time, and the moving image is more It can be played smoothly.

[Embodiment 3]
Below, the moving image reproducing device 300 of Embodiment 3 will be described with reference to FIGS.
As shown in FIGS. 8 to 10, the moving image reproducing apparatus 300 according to the third embodiment has two consecutive image frames when the display unit 5 performs slow reproduction of a plurality of image frames constituting moving image data in MPEG format. Interpolated image frames are generated, and the interpolated image frames are inserted between the two image frames and reproduced on the display unit 5.
Note that the video playback device 300 of the third embodiment has substantially the same configuration as the video playback devices 100 and 200 of the first and second embodiments except that the configuration of the CODEC 306 is different, and the description thereof is omitted.

FIG. 8 is a block diagram showing a schematic configuration of a moving image playback apparatus 300 according to the third embodiment to which the present invention is applied.
As shown in FIG. 8, the CODEC 306 includes a control unit 6h in addition to the decoding unit 6a.

The control unit 6h serves as an interpolated image generation unit when performing slow playback (playback at a frame rate lower than a predetermined frame rate) of a plurality of image frames constituting MPEG moving image data. Interpolated image frame (for example, “NO.2-3 frame”; see FIG. 10 (b)) for interpolating frames (for example, “NO.2 frame” and “NO.3 frame”; see FIG. 10 (b)) Is generated. That is, as shown in FIGS. 10 (a) and 10 (b), even in the case of slow playback, the playback frame rate of the image frame itself does not change. Although an empty time is generated between the frames (see FIG. 10A), generating and displaying the interpolated image frame prevents the empty time from being generated between the image frames (FIG. 10B). reference).
For example, as in the case of slow playback at a playback speed of 1/3, a plurality of frames between two image frames (for example, “NO.4 frame” and “NO.5 frame”; see FIG. 10B). When the interpolated image frames (for example, “NO. 4-5a frame” and “NO. 4-5b frame”; see FIG. 10B) are reproduced and displayed, the control unit 6h is configured to display the relative relative to the two image frames. An interpolated image frame is generated with different weights depending on the correct reproduction position.

Further, the control unit 6 h outputs a display control signal for inserting the generated interpolated image frame between the two image frames to reproduce and display on the display unit 5 to the display control unit 4.
When the display control signal output from the control unit 6h is input, the display control unit 4 reproduces and displays the interpolated image frame on the display unit 5 in accordance with the instruction signal.
Here, the control unit 6h and the display control unit 4 constitute a reproduction control unit that inserts the generated interpolated image frame between two image frames and reproduces and displays them on the display unit 5.

Next, the moving image reproduction processing by the moving image reproduction device 300 will be described with reference to FIGS. 9 and 10.
FIG. 9 is a flowchart illustrating an example of an operation related to the moving image reproduction process. FIG. 10A is a diagram for explaining normal moving image reproduction processing, and FIG. 10B is a diagram for explaining moving image reproduction processing according to the third embodiment. In FIG. 10B, a frame displayed as an interpolation image frame on the display unit 5 is schematically represented by dots.

The moving image reproduction process described below is a process that is executed when a reproduction display of a moving image is instructed based on a predetermined operation of the operation input unit 7 by the user. Generate and play back.
As shown in FIG. 6, when the control unit 6h of the CODEC 306 first reads MPEG format moving image data recorded on the recording medium 3, the control unit 6h outputs a predetermined control signal to the decoding unit 6a, and Of the plurality of image frames constituting the data, the first image frame (first frame) is decoded (step S41). The control unit 6h transfers the decoded first frame to the memory 2 and temporarily stores it.

Next, the control unit 6h determines whether or not there is a next encoded frame that is included in the moving image data in the MPEG format and continues to the first frame (step S42).
If it is determined that the next encoded frame exists (step S42; YES), the control unit 6h outputs a predetermined control signal to the decoding unit 6a to start decoding the next encoded frame ( Step S43). When the decoding of the next encoded frame by the decoding unit 6a is completed (step S44), the control unit 6h determines whether or not the display timing of the next image frame has been reached (step S45). In addition, when it is determined in step S42 that the next encoded frame does not exist (step S42; NO), the control unit 6h shifts the process to step S45.

If it is determined in step S45 that the display timing of the next image frame has been reached (step S45; YES), the control unit 6h is still in the plurality of image frames constituting the moving image data in the MPEG format. It is determined whether or not there is an undisplayed image frame (undisplayed frame) (step S46).
Here, if it is determined that an undisplayed frame exists (step S46; YES), the control unit 6h shifts the process to step S42, and if the above-described process does not exist in step S46. Repeat until it is determined (step S46; NO).
As a result, the CODEC 6 decodes all MPEG moving image data.

On the other hand, when it is determined in step S45 that the display timing of the next image frame has not been reached (step S45; NO), the control unit 6h decodes the decoded image frame (decoded frame) that matches the display timing. Is determined (step S47).
If it is determined that there is a decoded frame that matches the next display timing (step S47; YES), the control unit 6h employs a decoded image frame (decoded image) as the next reproduced image frame. Then, the display control signal of the decoded image is output to the display control unit 4, and the decoded image is reproduced and displayed on the display unit 5 in accordance with the next display timing (step S48).
Thereafter, the control unit 6h shifts the process to step S46, and repeats the above process until it is determined in step S46 that no undisplayed frame exists (step S46; NO).

On the other hand, when it is determined in step S47 that there is no decoded frame that coincides with the next display timing (step S47; NO), that is, for example, based on a predetermined operation of the operation input unit 7 by the user, a slow reproduction instruction 10h, the control unit 6h displays two consecutive image frames (for example, “NO.2 frame” and “NO.3 frame” already decoded by the decoding unit 6a; FIG. 10B). An interpolation image frame (for example, “NO. 2-3 frame”) is generated from the reference (step S49). At this time, a plurality of interpolated image frames reproduced between two image frames (for example, “NO.4 frame” and “NO.5 frame”; see FIG. 10B) according to the slow reproduction speed. (For example, “NO.4-5a frame” and “NO.4-5b frame”; see FIG. 10B) may be generated.
Then, the control unit 6h outputs a display control signal for the interpolation image to the display control unit 4, and reproduces and displays the interpolation image on the display unit 5 in accordance with the next display timing (step S50).
Thereafter, the control unit 6h shifts the process to step S46, and repeats the above process until it is determined in step S46 that no undisplayed frame exists (step S46; NO).
Thereby, the moving image reproduction process is terminated.

As described above, according to the moving image playback apparatus 300 of the third embodiment, when a plurality of image frames constituting MPEG moving image data is played back slowly on the display unit 5, two consecutive image frames are interpolated. An interpolated image frame is generated, and the interpolated image frame is inserted between the two image frames and reproduced on the display unit 5.
Therefore, when a moving image is played back slowly, an interpolated image frame is inserted between two image frames and played back, thereby preventing an empty time from being generated between the image frames. Image frames can be reproduced and displayed at a constant reproduction frame rate, and moving images can be reproduced smoothly.

Note that the present invention is not limited to the first to third embodiments, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, in Embodiments 1 to 3 described above, both the decoding of the image frame and the generation of the interpolated image frame are performed in the CODECs 6, 206, and 306. However, each process is performed by a separate functional unit. May be. That is, for example, by generating the interpolated image frame with the CPU 1 of the moving image reproducing apparatuses 100, 200, and 300, it is possible to reduce the burden on the CODEC 6 and reproduce and display the image frame at a frame rate higher than the performance of the CODEC 6. .

  Further, the configurations of the moving image playback apparatuses 100, 200, and 300 are merely examples, and are not limited to those illustrated in the above embodiment. That is, a moving image playback apparatus that includes at least recording means for recording encoded moving image data, and that generates an interpolated image frame when a predetermined condition is satisfied, and reproduces and displays the interpolated image frame. Any configuration may be used. For example, an imaging unit that captures one scene and generates a plurality of continuous image frames, and an encoded moving image that encodes the plurality of image frames generated by the imaging unit. A configuration including an encoding unit that generates data may be used.

In addition, in the above embodiment, the functions of the decoding unit, the reproducing unit, the interpolated image generating unit, the time determining unit, the reproduction controlling unit, the counting unit, and the number determining unit are controlled by the CODEC 6, However, the present invention is not limited to this, and may be realized by executing a predetermined program or the like by the CPU 1.
That is, a program memory (not shown) that stores a program stores a program including a decoding process routine, an interpolated image generation process routine, a time determination process routine, a reproduction control process routine, a count process routine, and a number determination process routine. The CPU 1 may function as a decoding unit, a reproducing unit, an interpolated image generating unit, a time determining unit, a reproduction controlling unit, a counting unit, and a number determining unit by a predetermined processing routine.

100 video playback device 1 CPU
4 Display control unit 5 Display unit 6, 206, 306 CODEC
6a Decoding unit 6b Time determination unit 6c, 6g, 6h Control unit 6e Count unit 6f Number determination unit

Claims (6)

Recording means for recording encoded moving image data in which a plurality of continuous image frames are encoded;
Decoding means for decoding the encoded moving image data to generate the plurality of image frames;
Reproduction means for switching and reproducing the plurality of image frames generated by the decoding means at a predetermined frame rate;
Of the plurality of image frames, for a predetermined number of consecutive image frame groups, the decoding time required for generation of each image frame by the decoding unit is the reproduction of each image frame according to the predetermined frame rate by the reproduction unit. Counting means for counting the number of delays over time;
Number determination means for determining whether or not the number of delays counted by the counting means is equal to or greater than a predetermined number;
When it is determined by the number determination means that the number of delays is equal to or greater than a predetermined number, two image frames decoded and generated by the decoding means are identified from the image frame group with a predetermined interval. Interpolated image generating means for generating an interpolated image frame corresponding to one image frame to be reproduced between the two image frames;
Playback control means for inserting the interpolated image frame generated by the interpolated image generating means between the two image frames and causing the playback means to play back;
A moving picture reproducing apparatus comprising: The interpolation image generation means includes
2. The moving image reproducing apparatus according to claim 1 , wherein the interpolating image frame is generated when the reproducing unit reproduces the plurality of image frames at a frame rate faster than the predetermined frame rate. The interpolation image generation means includes
The moving image reproduction apparatus according to claim 1 , wherein an intermediate image frame or a morphing image frame of the two image frames is generated as the interpolation image. The interpolation image generation means includes
Claims wherein the number corresponding to the number of image frames between two image frames, and, and generating the interpolation image frame instead of the weighted according to the relative playback position with respect to the two image frames 4. The moving image playback device according to 3 . Imaging means for imaging one scene and generating a plurality of continuous image frames;
Encoding means for encoding the plurality of image frames generated by the imaging means to generate encoded moving image data;
5. The moving picture reproducing apparatus according to claim 1 , further comprising: A computer of a moving image reproducing apparatus comprising recording means for recording encoded moving image data in which a plurality of continuous image frames are encoded,
Decoding means for decoding the encoded moving image data to generate the plurality of image frames;
Reproduction means for reproducing a predetermined number of consecutive image frame groups among the plurality of image frames by switching the plurality of image frames at a predetermined frame rate for a decoding time required to generate each image frame by the decoding means. Counting means for counting the number of delays equal to or longer than the reproduction time of each image frame according to the predetermined frame rate by
Number determination means for determining whether or not the number of delays counted by the counting means is equal to or greater than a predetermined number;
When it is determined by the number determination means that the number of delays is equal to or greater than a predetermined number, two image frames decoded and generated by the decoding means are identified from the image frame group with a predetermined interval. Interpolated image generating means for generating an interpolated image frame corresponding to one image frame to be reproduced between the two image frames,
Reproduction control means for inserting the interpolated image frame generated by the interpolation image generating means between the two image frames and causing the reproduction means to reproduce the reproduction control means;
A program characterized by functioning as

Images