JP4834636B2 - Video playback apparatus and video playback method - Google Patents

Description

  The present invention relates to a video playback apparatus and a video playback method for playing back a compressed video signal with high image quality.

  In recent DVD recorders and HDD recorders, video / audio signals are recorded on a recording medium in a state of being compressed and encoded (encoded) by the MPEG method or the like, and are decoded and decoded for viewing. During special playback such as fast-forward / slow playback, the amount of decoding processing varies depending on the playback speed, so processing must be performed within the processing capability of the apparatus.

  Japanese Patent Laid-Open No. 2004-228867 aims to reduce the processing amount when decoding compressed moving image data by fast-forwarding, and moving image data excluding pictures to be output when reproduced by fast-forwarding, or moving images excluding output pictures and their reference pictures A technique for deleting and decoding data is disclosed.

  On the other hand, a so-called “super-resolution processing” technique for generating a high-resolution image from a plurality of images is known. In this method, a plurality of frames are accurately aligned to synthesize one image, and high definition is realized by utilizing a minute shift (phase difference) of pixels (see, for example, Patent Document 2). ).

Japanese Patent Laid-Open No. 2005-192013 JP-A-8-336046

  If a super-resolution processing technique as shown in Patent Document 2 is used in the reproduction of the DVD recorder or the like, the user can view a higher quality image. However, when applied during high-speed special playback such as fast forward / rewind, the decoding capability of the apparatus becomes an obstacle. This is because, in order to apply super-resolution to an I picture that is displayed by fast-forwarding or rewinding, it is necessary to decode a plurality of frames that are not displayed immediately before the I-picture, which greatly increases the amount of processing. Because it does. Decoding even frames that are not used for display leads to an increase in power consumption of the processing circuit.

  Although the technique of Patent Document 1 reduces the decoding process, it is sufficient to decode only the picture to be output (I picture) or the picture to be referred to at the time of decoding, so the load on the apparatus does not increase so much. . On the other hand, when super-resolution processing is applied, the problem of insufficient decoding processing capability becomes serious.

  An object of the present invention is to provide a high-resolution video while suppressing an increase in decoding processing load during special reproduction.

  The present invention is a video playback device that plays back a compressed video signal, a frame selection unit that selects a frame to be decoded from a video signal read from a recording medium, a decoding unit that decodes a selected frame, and a decoding A super-resolution processing unit that creates a high-quality frame by super-resolution processing using a plurality of frames, and a control unit that controls the frame selection unit and the super-resolution processing unit. The frame to be decoded is selected according to the control, and the application of the super-resolution processing to the decoded frame is controlled.

  Here, the control unit decodes all frames and applies super-resolution processing when the playback mode is normal playback, slow playback, or frame-by-frame playback, and the playback mode is fast-forward playback or rewind playback. In this case, only the frame to be displayed is decoded and the super-resolution processing is not applied.

  Here, the control unit decodes all frames and applies super-resolution processing when the playback mode is variable speed playback and the playback speed V is less than or equal to a threshold Vth determined by the decoding processing capability of the device. When the reproduction speed V exceeds the threshold value Vth, only the frame to be displayed is decoded and the super-resolution processing is not applied.

  Here, when the playback mode is playback of a digest scene, the control unit decodes a frame immediately after the start of the scene but does not apply super-resolution processing.

  The present invention is a video playback method for playing back a compressed video signal, the step of selecting a frame to be decoded from the video signal read from the recording medium, the step of decoding the selected frame, and a plurality of decoded frames And a step of creating a high-quality frame by super-resolution processing using the selected frame, selecting a frame to be decoded according to the playback mode, and controlling application of the super-resolution processing to the decoded frame.

  According to the present invention, it is possible to stably perform decoding processing during special reproduction and provide a high-resolution video.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of a video reproducing apparatus according to the present invention. Compressed video data (stream) is recorded and stored in a recording medium 11 such as a hard disk or an optical disk. Here, it is assumed that the video data is encoded by the MPEG-2 system. The frame selection unit 12 selects a frame to be decoded (decoded) from the read video data according to the reproduction mode. The video decoding unit 13 decodes the selected frame. The super-resolution processing unit 15 creates a high-resolution image using the plurality of decoded images. The switch 14 selects whether or not the super-resolution processing unit 15 executes processing on the decoded image from the video decoding unit 13. The output unit 16 outputs an image signal that has been subjected to super-resolution processing or a decoded image that is not subjected to super-resolution processing. The control unit 17 controls the frame selection unit 12, the switch 14, and the super-resolution processing unit 15 according to the designated playback mode. For normal playback, super-resolution processing is executed for all frames, but for special playback (trick playback), select the frame required for display and perform decoding and super-resolution processing according to the processing capability. Execute. As described above, in the video playback apparatus according to the present embodiment, application of super-resolution processing is limited according to the playback mode. In the configuration of FIG. 1, the processing for the audio signal is omitted. Although only the playback function portion is shown, it goes without saying that the recording / playback apparatus may be provided with a recording function.

  FIG. 2 is a diagram showing application of decoding and super-resolution processing to each reproduction mode in the present embodiment. Here, the playback mode will be described as being divided into groups 1, 2, and 3. The compressed video data is a combination of frames of I picture (intra-frame encoded image), P picture (inter-frame forward prediction encoded image), and B picture (bidirectional predictive encoded image) according to the MPEG-2 system. Composed.

  Group 1: During normal playback, all frames are decoded and super-resolution processing is applied to all frames. The same processing is performed for slow playback and frame advance playback.

  Group 2: In the case of fast forward playback and rewind playback, only the frame to be displayed (I frame) is decoded. In this case, if all the frames are decoded, the amount of processing becomes very large and the capacity of the decoding unit is insufficient. Therefore, the frames are skipped for decoding. As for the super-resolution processing, the frame used for image composition is lost, so the super-resolution processing is not applied.

  Group 3: In the case of variable speed reproduction, the reproduction speed V is compared with a threshold value Vth to determine application. This threshold value Vth is determined from the upper limit value of the processing capability (the number of processing frames per unit time) of the decoding unit. If V ≦ Vth, all frames are decoded, and super-resolution processing is applied to the display frame. If V> Vth, only the display frame (I frame) is decoded, and the super-resolution processing is not applied. That is, in variable speed playback, super-resolution processing is not performed because a part of the frame is skipped and decoded and displayed at higher speeds.

  However, there are cases where super-resolution processing can be applied even when V> Vth. That is, super-resolution processing can be performed if the number of frames to be skipped is small and decoding is possible including preceding frames used for display frame super-resolution processing.

  FIG. 3 is a diagram showing the relationship between the reproduction speed and the decoding processing amount, where the horizontal axis represents the reproduction speed V and the vertical axis represents the decoding processing amount C. The processing amount for the normal playback speed Vo is indicated by Co, and the playback speed threshold for the upper limit Cth of the processing capacity is indicated by Vth. In the area (1), since the reproduction speed V is equal to or less than the threshold value Vth and the processing amount is equal to or less than the upper limit value Cth, decoding can be performed for all frames (super resolution processing is applied). In the region (2), since the playback speed V exceeds the threshold value Vth and the processing amount exceeds the upper limit value Cth, some frames are skipped and decoded (super-resolution processing is not applied). In the area (3), when the playback speed V is further high, the skip of frames is increased to suppress the processing amount to the upper limit value Cth or less.

  As described above, even if some frames are skipped and decoded in the region (2), it is possible to apply the super-resolution processing if the preceding frame used for the super-resolution processing is decoded. . This corresponds to a case where the number N of images used for the super-resolution processing is small (for example, two), and the reproduction speed threshold value Vth ′ is larger than the above value Vth.

FIG. 4 is a flowchart showing the reproducing method of this embodiment.
When the reproduction mode is instructed by the user (S101), a predetermined amount of data is read from the recording medium 11 (S102). The frame selection unit 12 selects a frame to be decoded from the read data according to the reproduction mode (S103). The video decoding unit 13 performs a decoding process on the selected frame (S104). Next, the playback mode is determined (S105). If the playback mode is group 1 (normal / slow playback, etc.), super-resolution processing is performed (S107) and video is output (S108). If the playback mode is group 2 (fast forward / rewind), the video is output without performing super-resolution processing (S108). If the playback mode is group 3 (variable speed playback), the playback speed V is compared with a threshold value Vth (S106). If V ≦ Vth, super-resolution processing is performed on the display frame (S107), and video is output (S108). If V> Vth, video is output without performing super-resolution processing (S108). All the target data is reproduced and the process is terminated (S109).

  FIG. 5 is a diagram for explaining the principle of super-resolution processing performed by the super-resolution processing unit 15. First, a plurality (two in this case) of input low resolution image frames 51 and 52 are aligned with decimal precision. Positioning also takes into account movements such as rotation and enlargement / reduction. In this way, one composite image 53 whose sampling position is deviated from the integer pixel position is generated, and then a high-definition image 54 is generated by re-sampling at the target resolution. For resampling, determine the pixel value while convolving the influence of surrounding pixels by interpolating and increasing the number of pixels (sampling points) using a low-pass filter, or by applying an inverse function of the point spread function. To do. The preceding frame used for image composition is called an auxiliary frame (auxiliary image). Here, the frame 51 corresponds to an auxiliary frame.

  Hereinafter, the decoding process and the super-resolution process for the frame sequence in each reproduction mode will be described in detail. Note that the super-resolution processing here is a case where a high-resolution image is synthesized from the self frame and the immediately preceding two auxiliary frames.

  FIG. 6 is a diagram showing frame processing during normal playback. In the figure, I, P, and B represent I picture, P picture, and B picture frames, respectively, and the numbers are given in the order of frame input. (A) is a frame to be decoded (solid line frame), (b) is a frame display order, (c) is a frame to be subjected to super-resolution processing (dots are displayed in the frame) Indicates.

  In the case of normal playback, decoding processing is performed on all frames. Then, super-resolution processing is performed on all frames. For example, in the I1 frame super-resolution processing, the immediately preceding frames B3 and B4 are used as auxiliary images. The frame after the super-resolution processing is output according to the display rate.

  In the case of normal reproduction processing, one frame may be decoded every time the display frame is updated. Further, in slow playback and frame advance playback, the display frame update interval becomes longer, so that the decoding processing amount is reduced and there is no problem of insufficient processing capability.

  FIG. 7 is a diagram showing frame processing during fast-forward playback. In this case, the decoding process of (a) is performed only for the I frame (I1, I13,...) To be displayed, and the super-resolution process of (c) is not performed. Only the decoded I frames (I1, I13,...) Are displayed. The playback speed in this case is 12 times the normal playback speed.

  Here, it is assumed that super-resolution processing is applied in fast-forward playback. When the I1 frame is synthesized in the super-resolution processing of (c), the auxiliary images of the immediately preceding frames B3 and B4 are used. Therefore, in the decoding process of (a), it is necessary to decode the frames B3 and B4 (indicated by a one-dot chain line). As a result, for the display frame update interval (from I1 to I13), three frames I1, B3, and B4 are decoded. To decode B3 and B4, I1 is used as a reference image. Since the preceding I or P frame needs to be decoded, the amount of processing increases to four times or more that during normal reproduction. If the processing capability of the apparatus for this is insufficient, the reproduction process will fail.

  Thus, the reproduction operation can be stably executed by disabling super-resolution processing. In the case of rewind playback, the playback direction is just reversed, and the other operations are the same.

  FIG. 8 is a diagram showing frame processing during variable speed reproduction. Here, a case where the reproduction speed is twice the normal reproduction is shown. The decoding process (a) is performed on all frames, and the super-resolution process (c) is performed on a frame to be displayed, that is, every other frame (B3, I1, B7,...).

  In this case, two new frames may be decoded for the display frame update interval. That is, the amount of processing increases to twice that of normal reproduction, but if the processing capability of the apparatus is sufficient, there will be no failure even if decoding processing and super-resolution processing are performed.

  According to the above embodiment, since application / non-application of super-resolution processing is controlled according to the reproduction mode, it is possible to prevent the reproduction operation from failing due to an excessive decoding processing amount. Also, during variable speed reproduction, application / non-application control is performed by comparing the reproduction speed with a threshold determined by the processing capability, so that high-resolution images can be efficiently provided according to the capability of the apparatus.

Next, an example in which the video playback apparatus of the present invention is applied to video digest playback will be described.
FIG. 9 is a diagram for explaining video digest playback. Important scenes (called digest scenes, highlight scenes, etc.) 91, 92, 93 are extracted from the recorded series of video / audio information. In order to extract a digest scene, for example, a scene with a large excitement in a program or the like is automatically detected based on, for example, a change in luminance of a video signal or the level of an audio level. A label or the like indicating a digest scene is pasted on the extracted scene. In digest playback, the digest scenes 91, 92, and 93 are sequentially played back so that the user can view only important scenes in a short time. Super-resolution processing can also be applied to this digest reproduction.

  FIG. 10 is a diagram illustrating an example of frame processing for applying super-resolution processing in digest reproduction. When super-resolution processing is applied to all the frames constituting the digest scene, the burden of decoding processing increases. The present embodiment reduces this burden. (A) shows a frame for decoding, (b) shows a frame display order, and (c) shows a frame for super-resolution processing. Here, it is assumed that the digest scene starts from the frame I1. In addition, in order to perform the super-resolution processing, the last two auxiliary frames are used.

  To perform the super-resolution processing of the scene start portion I1 in (c), the decoded images of the frames B3 and B4 are required as auxiliary images. In addition, in order to perform the super-resolution processing of the frame B6, the decoded images of the frames B4 and I1 are required as auxiliary images. In order to decode B3 and B4, it is necessary to decode I1 and the preceding I or P frame as a reference image. Therefore, a decoding process for four frames or more is required for the frame I1, and a decoding process for three frames or more is required for the frame B6. Thereafter, a decoding process for one frame is sufficient. As described above, the amount of decoding processing at the start of the scene increases, and if the processing capability of the apparatus is insufficient, the reproduced image is disturbed.

  In this embodiment, in order to prevent this, the super-resolution processing is not applied to the frame immediately after the start of the scene, although the decoding processing is performed. Specifically, a frame without super-resolution processing is output for two frames immediately after the start of the scene, and a frame to which super-resolution processing is applied is output from the third frame B7. The non-application period of the super-resolution processing depends on the number N of frames used for the super-resolution processing. Preferably, the number of frames for which super-resolution processing is stopped may be (N-1) frames immediately after the start of the scene. As a result, the decoding process does not increase, and the digest scene reproduction operation can be stabilized.

The block diagram which shows one Example of the video reproduction apparatus concerning this invention. The figure which shows application of the decoding and super-resolution process with respect to each reproduction | regeneration mode. The figure which shows the relationship between reproduction speed and the amount of decoding processes. The flowchart which shows the reproduction | regenerating method of a present Example. The figure explaining the principle of a super-resolution process. The figure which shows the frame process at the time of normal reproduction | regeneration. The figure which shows the frame process at the time of fast-forward reproduction | regeneration. The figure which shows the frame process at the time of variable speed reproduction | regeneration. The figure explaining video digest reproduction | regeneration. The figure which shows the frame process in digest reproduction | regeneration.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Recording medium 12 ... Frame selection part 13 ... Video decoding part 14 ... Switch 15 ... Super-resolution processing part 16 ... Output part 17 ... Control part.

Claims (6)

In a video playback device that plays back a compressed video signal,
A frame selector for selecting a frame to be decoded from the video signal read from the recording medium;
A decoding unit for decoding the selected frame;
A super-resolution processing unit that creates a high-quality frame by super-resolution processing using a plurality of decoded frames;
A control unit that controls the frame selection unit and the super-resolution processing unit;
When the playback mode is variable speed playback and the playback speed V is equal to or lower than a threshold Vth determined by the decoding processing capability of the device, the control unit decodes all frames and applies super-resolution processing.
A video playback device characterized in that when the playback speed V exceeds a threshold value Vth, only the frame to be displayed is decoded and the super-resolution processing is not applied. In a video playback device that plays back a compressed video signal,
A frame selector for selecting a frame to be decoded from the video signal read from the recording medium;
A decoding unit for decoding the selected frame;
A super-resolution processing unit that creates a high-quality frame by super-resolution processing using a plurality of decoded frames;
A control unit that controls the frame selection unit and the super-resolution processing unit;
When the playback mode is playback of a digest scene, the control unit decodes a frame immediately after the start of the scene but does not apply super-resolution processing. The video playback device according to claim 2,
A video reproducing apparatus characterized in that when N is the number of frames used for the super-resolution processing, the super-resolution processing is not applied to (N-1) frames immediately after the start of the scene when reproducing the digest scene. In a video playback method for playing back a compressed video signal,
Selecting a frame to be decoded from the video signal read from the recording medium;
Decoding the selected frame; and
Using a plurality of decoded frames to create a high quality frame by super-resolution processing,
When the playback mode is variable speed playback and the playback speed V is less than or equal to the threshold Vth determined by the decoding processing capability, all frames are decoded and super-resolution processing is applied,
A video playback method characterized by decoding only a frame to be displayed and not applying super-resolution processing when the playback speed V exceeds a threshold value Vth. In a video playback method for playing back a compressed video signal,
Selecting a frame to be decoded from the video signal read from the recording medium;
Decoding the selected frame; and
Using a plurality of decoded frames to create a high quality frame by super-resolution processing,
A video playback method characterized in that when the playback mode is playback of a digest scene, a frame immediately after the start of the scene is decoded but super-resolution processing is not applied. The video reproduction method according to claim 5, wherein
A video playback method characterized in that when N is the number of frames used for the super-resolution processing, the super-resolution processing is not applied to (N-1) frames immediately after the start of the scene in the playback of the digest scene.

Images