JPH1013794A - Moving image reproducing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image reproducing device from which an image is smoothly reproduced where a reproduction time of reproduction display of a moving image is in matching with a substantial reproduction time independently of a load state of a CPU. SOLUTION: A decoding processing section 103 reads compressed image data from a compression data file 101 to conduct decoding processing of compressed image data. An image data buffer 104 stores a plurality of frames of the image data decoded by the decoding processing section 103. A display control section 105 displays the frames in the image data buffer 104 onto a display device 107. A timer section 106 counts a timing of display in the display control section 105. An interleave control section 102 interleaves data by the decoding processing. The decoding processing section 103 and the display control section 105 are in existence independently and while the display is updated at a prescribed interval, when the decoding is not in time, the frames are interleaved and when the decoding is faster, the decoding is stopped.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture reproducing apparatus for reproducing moving pictures, and more particularly to a moving picture reproducing apparatus for reproducing moving pictures smoothly.

[0002]

2. Description of the Related Art When a large amount of data such as a moving image is handled, the interval at which images are reproduced and displayed is not constant unless sufficient CPU performance and bus performance are secured. For that reason,
An unnatural and smooth display is obtained in which the image display is suddenly stopped depending on the load status of the bus or the CPU, or the image suddenly starts to move from a state close to stationary.

[0003] As a moving image reproducing apparatus for smoothing reproduction display, there is a method described in Japanese Patent Application Laid-Open No. 7-32714 (hereinafter referred to as Prior Art 1).

[0004] A moving image reproducing apparatus disclosed in Prior Art 1 for smoothing reproduction display will be described below with reference to FIG.

[0005] A moving image reproducing apparatus shown in FIG.
, ROM 802, RAM 803, and external storage device 8
04, an operating device 805, a management table 806, a display control circuit 807, a frame buffer 808, a display device 809, a data processing device 810, a line control circuit 811, a line 812, and a bus 813. .

[0006] The data processing device 810 decodes the image data and adds information on the generation time of each frame to the management table 806. The display control circuit 807 obtains the difference in the generation time between each frame, compares this value with the reference value α to obtain the display level HL, and registers it in the management table 806. The display device 809 checks the display level HL of the frame being displayed by the management table 806, and determines a time interval until the next frame is displayed according to the value of the display level. Further, the number of frames stored in the frame buffer 808 is checked, and the value of the display level HL is adjusted according to the number of frames to adjust the display interval.

As described above, the display time of each frame is set and controlled in accordance with the time required for processing for displaying a moving image and the number of undisplayed images stored in the frame buffer 808. If the decoding takes a long time, the frame display interval is made longer, and if the reverse is the case, the frame display interval is made shorter. In this way, moving image data can be smoothly reproduced regardless of the load on the CPU 801.

A method of thinning out images in order to control the reproduction time is described in JP-A-6-121282 (hereinafter referred to as prior art 2).

Referring to FIG. 10, a description will be given of a moving image reproducing apparatus which realizes a method of thinning out images disclosed in Prior Art 2.

The illustrated moving picture reproducing apparatus includes an image data storage unit 901, an image buffer 902, a similarity calculation unit 90
3, a maximum similarity detection unit 904, a buffer display controller 905, and an image display 904.

An image data storage unit 901 stores a time series of image data continuously recorded at a fixed time interval and outputs it as a time series image signal a. Image buffer 9
In step 02, the time-series image signal a is input, and image data for a certain time is temporarily held until the buffer display controller 905 reads or skips the image data.
The similarity calculation unit 903 receives the image signal b, calculates the similarity between the respective temporally adjacent image data, and outputs a similarity signal c indicating the similarity. The similarity between two pieces of image data is determined, for example, by summing the sum of squares of the three primary color values of each corresponding image between the image data for all pixels, and determining that the smaller the value is, the more similar the image data is. The maximum similarity detection unit 904 receives the similarity signal c, selects (n−m) images having the highest similarity with the adjacent image from the top, and thins out those images by the buffer display controller 905. The thinning control signal d is output. Buffer display controller 905
Displays a thinned image in accordance with the thinning control signal d.

As described above, fast-forward reproduction without discontinuous movement is realized by thinning out portions having a high degree of similarity.

[0013]

The method for smoothing reproduction disclosed in the prior art 1 has the following problems. First, when the frame display interval is lengthened or shortened, the reproduced image looks smooth, but the overall reproduction time becomes undefined and does not match the original reproduction time. Secondly, when synchronized with the audio, if the audio is reproduced in synchronization with the reproduction time of the moving image, the audio may be fast-forwarded or slowed down, or the audio may be interrupted, making it difficult to hear.

If the time required for the image processing is too long, it is necessary to perform the image processing as soon as possible by thinning out frames to be subjected to the image processing. However, the conventional image thinning method has a drawback that the reproduced image cannot be seen smoothly because the thinning is performed mechanically regardless of the image of the frame to be thinned or the image processing method. On the other hand, prior art 2
In the method of calculating the similarity between image data in order to smooth this, it takes time to calculate,
Although the load on the CPU is large for calculating the similarity, the load on the CPU is further increased for thinning out the images, and there is a problem that the number of reproduced frames is reduced.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a moving image generating apparatus which can reproduce a moving image smoothly with the reproduction time coincident with the original reproduction time.

[0016]

SUMMARY OF THE INVENTION A moving image reproducing apparatus according to the present invention converts an input data into displayable image data, an image data buffer for storing a plurality of image data as a plurality of frames, and Image processing means for storing the image data as the frame in the image data buffer, and reading and displaying the image data from the image data buffer at regular intervals, regardless of the time required for image processing in the image processing means. Display control means for displaying a moving image by updating the image.

[0017]

Embodiments of the present invention will be described below in detail with reference to the drawings.

Referring to FIG. 1, a moving picture reproducing apparatus according to an embodiment of the present invention comprises a compressed data file 101 for storing compressed image data,
A decoding processing unit 103 that reads compressed image data from the storage unit and performs decoding processing of the compressed image data; a plurality of image data buffers 104 that store decoded image data decoded by the decoding processing unit 103 as frames; Display control unit 104 that displays frames in buffer 104
A timer unit 106 for measuring display timing in the display control unit 104, a thinning control unit 102 for thinning out decoding processing,
And a display device 107.

The decryption processing unit 103 stores the compressed data file 1
01, the compressed image data is read, and the decoded frames are sequentially stored in the image data buffer 104. The image data buffer 104 is a ring buffer that can store a plurality of decoded frames, and instructs the timer unit 106 to start displaying when a certain number of frames above are stored. The timer unit 106 notifies the display control unit 105 of a display request signal at regular intervals (in units of 33 milliseconds in the case of display at 30 frames / second). The display control unit 105 transfers the decoded frame from the image data buffer 104 to the display device 107 for display.

At this time, when the number of frames remaining in the image data buffer 104 is equal to or less than a certain fixed number (first threshold value) N, the number of thinned frames is indicated in order to omit decoding processing. Increments a counter (not shown). Conversely, when the number of frames remaining in the image data buffer 104 is equal to or more than a certain number (second threshold value) M, the counter is cleared,
Cancel the thinning specification. The thinning control unit 102 checks the counter, and if there is a thinning request and the frame to be decoded can be thinned, the decoding processing unit 10
For 3, skip reading of compressed image data is specified. When the skipping of the compressed image data is specified, the decoding processing unit 103 skips the frames of the compressed image data by one frame to thin out the frames.

Next, the operation of the display control unit 105 shown in FIG. 1 will be described with reference to FIG.

The display control unit 105 is called from the timer unit 106 at a certain display interval. This interval is the frame rate, and is usually in units of (1/30) seconds at the frame rate of a television (TV). The display control unit 105 checks the number of remaining frames in the image data buffer 104 (step 201). Here, if the number of remaining frames is equal to or smaller than the first threshold value N, the display control unit 105 increments the number-of-thinned-frames counter to request the thinning-out (step 202), and ends the processing. On the other hand, if the number of remaining frames in the image data buffer 104 is equal to or larger than the second threshold value M, the display control unit 105 sets a thinning-out frame number counter (not shown) to 0 in order not to thin out (step 203 ), And proceed to step 204. If the number of remaining frames in the image data buffer 104 is in the range N to M, the display control unit 105 immediately proceeds to step 204. In step 204, the display control unit 105 transfers one frame in the image data buffer 104 to the display device 107. And the display control unit 1
05 displays it on the display device 107 (step 2
05).

Here, the relationship between the first threshold N for issuing a thinning request, the second threshold M for clearing a thinning request, and the number K of frames that can be stored in the image data buffer 104 is as follows. .

0 <N <M ≦ K Here, when the number of frames is equal to or smaller than the first threshold value N, the image data is not transferred to the display device 107. The same is true for forwarding.

Next, the operation of the thinning control unit 102 shown in FIG. 1 will be described with reference to FIG.

The decimation control unit 102 includes a decryption processing unit 103
Is called every time one frame is processed. The thinning control unit 301 checks the number of thinned frames set by the display control unit 105 (step 3).
01) If the number of thinned frames is 0, no thinning processing is performed. When the number of thinned frames is larger than 0,
The thinning control unit 102 checks the next compressed image data (step 302). Here, if thinning is possible,
The thinning control unit 102 decrements the thinning frame number counter (step 303) and designates skipping of compressed image data (step 304).

The determination as to whether or not thinning is possible can be made based on, for example, whether or not compressed image data is referred to by other compressed image data. For example, MPEG (Moving Pic
In the case of the turecording experts group method, the configuration of the compressed image data is as shown in FIG. As shown in FIG. 4, the compressed image data includes three types of compression schemes called an I picture (intra coded image), a P picture (predicted coded image), and a B picture (bidirectional predicted coded image). . The I picture is a compression method that can decode a one-frame image using only the compressed image data. The P picture is a compression method for decoding by referring to an I picture or a P picture frame existing before the frame. B
A picture is a compression method for decoding with reference to an I picture or P picture frame existing before the frame and an I picture or P picture existing after the frame. In this case, even if the B picture is thinned, no trouble occurs because other compressed image data is not referred to. However, when the P picture or the I picture is thinned, the P picture and the B picture which refer to the frame are not referenced. The picture cannot be decoded. Therefore, in the case of the MPEG system, B pictures can be thinned out, but P pictures and I pictures cannot be thinned out.

As another method, it is also possible to provide a level for a thinning request when specifying thinning. The thinning request level is, for example, a low level when the counter value is 1, a medium level when the counter value is 2, and a high level when the counter value is 3 or more, depending on the value of the thinning frame number counter. Then, at the low level, only the B picture can be thinned, at the middle level, the B picture and the P picture can be thinned, and at the high level, all the pictures can be thinned.

Next, the operation of the decoding processing unit 103 shown in FIG. 1 will be described with reference to FIG.

The decoding processing unit 103 is called when one frame is decoded, and checks the number of remaining frames in the image data buffer 104 (step 401).
If the number of remaining frames in the image data buffer 104 is equal to or greater than the third threshold value L, the decoding processing unit 103 waits for the start of image decoding processing (step 402). If the number of remaining frames in the image data buffer 104 is less than the third threshold value L, the decoding processing unit 103 checks whether to skip the compressed image data set by the thinning control unit 102 (step 403). If skipping is specified, the decoding processing unit 103 skips one frame of the compressed image data (step 404). If no skipping is specified, the decoding processing unit 103 performs a normal decoding process (step 405), and stores the decoded frame in the image data buffer 104 (step 406).

The relationship between the first threshold value N for issuing a thinning request, the third threshold value L for stopping the decoding process, and the number K of frames that can be stored in the image data buffer 104 is as follows. .

0 <N <L ≦ K In the flowchart shown in FIG. 5, when thinning out the frames, the compressed image data is simply skipped. However, as another method, a frame interpolation unit (FIG. (Not shown), and the skipped frame may be created by interpolating from the preceding and following frames. As a method of the interpolation, for example, there is a method of averaging the three primary color values of the corresponding pixels of the preceding and succeeding frames for all the pixels to form one frame.

[0033]

Next, an embodiment of the present invention will be described with reference to the drawings.

FIGS. 6 to 8 show the decoding processing time, the change of the image data buffer, and the temporal transition of the display frame. For the sake of explanation, a first threshold N = 2 for issuing a thinning request and a second threshold M = for clearing a thinning request
6, a description will be given assuming that the third threshold value L at which decoding processing is stopped is L = 6, and the number of frames that can be stored in the image data buffer 104 is 8. The display process is started when the data of four frames has accumulated in the image data buffer 104 after the start of the decoding process. Also, the numbers in the figure mean frame numbers.

FIG. 6 is a diagram showing display timing when the time required for the decoding process differs for each frame.

In this example, when four frames of image data are stored in the image data buffer 104 (at time T).
9) The display is started, and the image of frame number 1 is displayed. A plurality of frames are stored in the image data buffer 104, and the display is updated at a constant interval α. However, since the image data buffer 104 has room, there is a frame having a long decoding time such as the frame number 8. However, the display is updated at regular intervals.

FIG. 7 is a diagram showing the display timing when the display interval α is shorter than the time required for the decoding process.

In this example, when four frames of image data are stored in the image data buffer 104 (at time T).
9) The display is started, and the image of frame number 1 is displayed. After time α (time T11), the image of the frame number 2 is displayed. At this time, the number of remaining frames in the image data buffer 104 is equal to the first threshold value N at which the thinning request is issued.
= 2 or less, the display control unit 105 issues a thinning request to the thinning control unit 102. When decoding the next compressed image data (compressed image data of frame number 6), the thinning control unit 102 that has received the request checks the compressed data to be decoded and determines that the compressed image data can be thinned. , The compressed image data is skipped, and the same frame as the previous frame is written in the image data buffer 104.

Further, at time T15, an image of frame number 4 is displayed.
Is the threshold N at which the number of remaining frames in the system issues a thinning request.
= 2 or less, the display control unit 105 issues a thinning request to the thinning control unit 102. When decoding the next compressed image data (compressed image data of frame number 8), the thinning control unit 102 that has received the request checks the compressed image data to be decoded, and determines that this compressed image data cannot be thinned. In this case, the compressed image data of frame number 8 is decoded as usual.

Further, when decoding the compressed image data of the frame number 9, the thinning correction control unit 102 which has received the request checks the compressed image data to be decoded and determines that this compressed image data can be thinned. , The compressed image data is skipped, and the same frame (frame of frame number 8) as the previous image data is stored in the image data buffer 104.
Write to. The thinning process is performed in this manner, and the display is updated at regular intervals.

FIG. 8 is a diagram showing the display timing when the display interval α is longer than the time required for the decoding process.

In this example, when four frames of image data are stored in the image data buffer 104 (at time T).
9) The display is started, and the image of the first frame is displayed. Thereafter, images are sequentially displayed at intervals of time α, and when the compressed image data of frame number 9 is decoded, the number of remaining frames in the image data buffer 104 stops decoding processing. Therefore, the decoding process is stopped until the number of remaining frames in the image data buffer 104 becomes less than the third threshold value L = 6 for stopping the decoding process. When the frame of the frame number 4 is displayed at the time T21, the decoding process is restarted because the remaining frames in the image data buffer 104 become less than the third threshold value L = 6 at which the decoding process is stopped. Thus, the decoding processing time is adjusted, and the display is updated at an integral interval.

The present invention is not limited to the above embodiment,
Various changes and modifications are possible without departing from the spirit of the present invention. For example, in the above embodiment, since the compressed image data is stored in the compressed data file 101, the decoding processing unit 103 reads the compressed image data from the compressed data file 101 and performs the decoding processing of the compressed image data. Instead of the decoding processing unit 103, an image processing unit that generally converts input data into displayable image data and stores the converted image data as a frame in an image data buffer may be used.

[0044]

As described above, according to the present invention, the display can be updated at regular intervals and the frames to be thinned out can be selected by making the decoding processing unit and the display processing unit independent, so that smooth reproduction is possible. Becomes

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving image reproducing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart for explaining an operation of a display control unit used in the moving image reproducing device of FIG. 1;

FIG. 3 is a flowchart illustrating an operation of a thinning-out control unit used in the moving image reproducing apparatus of FIG. 1;

FIG. 4 is a diagram showing a configuration of MPEG-format compressed image data.

FIG. 5 is a flowchart for explaining an operation of a signal processing unit used in the moving image reproducing device of FIG. 1;

FIG. 6 is a diagram showing display timing according to the present invention when the time required for the decoding process differs for each frame.

FIG. 7 is a diagram showing the display timing of the present invention when the display interval is shorter than the time required for the decoding process.

FIG. 8 is a diagram showing the display timing of the present invention when the display interval is longer than the time required for the decoding process.

FIG. 9 is a block diagram showing a configuration of a moving image reproducing apparatus disclosed in Japanese Patent Application Laid-Open No. 7-32714 (hereinafter referred to as Prior Art 1) for smoothing reproduction display.

FIG. 10 is a block diagram illustrating a configuration of a moving image reproducing apparatus that realizes a method for thinning out images disclosed in Japanese Patent Application Laid-Open No. 6-121282 (hereinafter, referred to as Prior Art 2).

[Explanation of symbols]

 Reference Signs List 101 compressed data file 102 thinning control unit 103 decoding processing unit 104 image data buffer 105 display control unit 106 timer unit 107 display device

Claims (5)

[Claims] An image data buffer for storing a plurality of image data as a plurality of frames; converting input data into displayable image data; and storing the converted image data in the image data buffer. An image processing unit (103) for storing the image data as the frame, and reading out the image data from the image data buffer at regular intervals, regardless of the time required for the image processing in the image processing unit, and updating the display image. A moving image reproducing apparatus, comprising: a display control unit (105) for displaying a moving image. 2. The moving image reproducing apparatus according to claim 1, wherein the image processing means adjusts a time at which image processing is started according to the number of remaining frames in the image data buffer. 3. The moving image reproducing apparatus according to claim 1, further comprising a thinning control unit for thinning out image processing by said image processing unit according to the number of remaining frames in said image data buffer. . 4. The input data is compressed image data, the image processing means decodes the compressed image data as the image processing, and the thinning control means sets a remaining frame in the image data buffer. Setting the image thinning request level in accordance with the number and the number of frames repeatedly read, thinning out the image processing in the image processing means in accordance with the image thinning request level and the type of the compressed image data; 4. The moving image according to claim 3, wherein the control unit reads the same image data immediately before the image data thinned by the thinning control unit from the image data buffer and updates the display image. Image reproduction device. 5. The moving picture reproducing apparatus according to claim 3, further comprising a frame interpolating means for interpolating a frame skipped by skipping image processing from previous and subsequent frames to create the interpolated frame. .