JP2009239335A - Video interpolation device and video interpolation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video interpolation device and a video interpolation method which can improve video quality inexpensively by detecting content change of source video data without using any dedicated signal line and controlling frame interpolating operation. <P>SOLUTION: Source source image data of 15 FPS received by an antenna 11 or read out of a video recording medium are coded by a decoder 15 and then input as input video data of 60 Hz to a frame interpolation unit 20. In the frame interpolation unit 20, a frame comparison portion 23 determines whether contents of successive frames match each other, a counter 24 counts the number of times of matching, and an interpolating operation control portion 25 controls generation of an interpolation frame by an interpolation processing portion 22 according to whether every four frames have mismatching. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video interpolation apparatus and video interpolation method for increasing the frame rate of video data and performing interpolation between frames.

  In recent years, it has become necessary to display after performing processing to increase the frame rate of video data. For example, with the digitization of television broadcasting, digital broadcasting with a small amount of data for display on mobile phone terminals or the like, so-called one-segment broadcasting, is being carried out. The frame rate of this one-segment broadcasting is 15 FPS (frames / second). It is. On the other hand, since the current television broadcasting is 30 FPS, in order to display the one-segment broadcasting smoothly, it is necessary to increase the frame rate and interpolate between the frames.

  As techniques for increasing the frame rate for video data in this way, for example, Patent Literature 1 and Patent Literature 2 are disclosed.

Japanese Patent Laid-Open No. 2003-69691 JP 2001-313896 A

  In order to perform frame interpolation, it is necessary to increase the frame rate by repeatedly using the frame of the original video data, and then create an interpolation frame indicating the intermediate state content from the frame whose content has changed.

  At this time, the problem is how to select two frames to be used for creating an interpolation frame. When the frame rate is increased from the original video data, if a repeated frame is created exactly a predetermined number of times, it is sufficient to select the frame every predetermined number of times. However, if the number of repetitions varies depending on the time required for decoding, etc. Interpolation errors such as creating an interpolated frame from the content frame or skipping the intervening frames to perform frame interpolation occur, and the video quality deteriorates.

  In order to solve this problem, it is necessary to detect the content change (frame transition) in the original video data, but if a content change signal is transmitted from a decoder or the like, a dedicated signal line and signal processing system Is required, leading to increased costs.

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems, and detects the content change in the original video data without using a dedicated signal line, and performs frame interpolation. An object of the present invention is to provide a video interpolation apparatus and a video interpolation method that improve the video quality at low cost by controlling the operation.

  In order to solve the above-described problems and achieve the object, the video interpolating apparatus and the video interpolating method according to the present invention output the original video data at a frame rate higher than the original frame rate by repeating each frame a predetermined number of times. When the input video data is received and the content of the repeated frame in the input video data is interpolated from the frames located before and after the frame, the frame content is updated by comparing the content of successive frames of the input image data. The timing is detected, and the operation related to the interpolation is controlled based on the detection result.

  According to the present invention, the video interpolation device and the video interpolation method detect the content change in the original video data by the frame comparison of the input video data without using a dedicated signal line or the like, and control the frame interpolation operation. There is an effect that it is possible to obtain a video interpolation device and a video interpolation method that improve video quality at low cost.

  A video interpolation apparatus and video interpolation method according to the present invention will be described below with reference to the drawings.

  The video interpolating apparatus according to the present invention is used inside a receiving apparatus compatible with Japanese terrestrial digital television broadcasting. In digital terrestrial television broadcasting, the bandwidth for one channel is divided into 13 segments, 12 of which are used for fixed services (for home use), and the remaining 1 segment is used for mobile services (mobile receiver terminals). Assigned for use. The fixed service employs the MPEG2 system as a data compression system, and the mobile service employs the H.264 data compression system. The H.264 system (one of the video compression systems also known as MPEG-4 Part 10) (or the MPEG4 system) is employed. Encoded digital data for fixed services is encoded by the MPEG2 system and is high quality, but the bit rate is high, so noise resistance is weak, and good when the distance from the broadcast station is long. Reception may be difficult. In contrast, encoded digital data for mobile services (so-called one-segment broadcasting) is H.264. H.264 format (or MPEG4 format) encoded with a lower bit rate than the MPEG2 format, so it has high noise resistance and can be received well even when the distance from the broadcasting station is longer. It becomes possible.

  In this one-segment broadcasting, the data rate is reduced by setting the frame rate to 15 FPS (frames / second) and half of the current television broadcasting of 30 FPS. Therefore, in order to display the one-segment broadcasting smoothly, the frame rate is increased on the receiving device side and interpolation between frames is performed.

  FIG. 1 is a schematic configuration diagram illustrating a schematic configuration of a one-segment receiver 1 that is an embodiment of the present invention. The one-seg receiving apparatus 1 shown in the figure includes a frame interpolation unit 20 in addition to an antenna 11, a tuner 12, a video recording medium 13, a switching unit 14, a decoder 15, a frame memory 16, and a display unit 17.

  In the one-segment broadcasting reception apparatus 1, the tuner 12 selects a reception wave of the antenna 11. The video recording medium 13 holds arbitrary video data such as past one-segment broadcast recording data.

  The switching unit 14 selects any one of the one-segment broadcasting received by the tuner 12 and the video data held in the video recording medium 13 and outputs the selected data to the decoder 15. The decoder 15 performs processing for decoding the video data selected by the switching unit 14 and inputting the decoded video data to the frame interpolation unit 20.

  Here, the input video data that the decoder 15 inputs to the frame interpolation unit 20 is about 60 Hz, that is, 60 FPS. However, the original video data (received one-segment broadcasting) is 15 FPS, and the frame content is updated only 15 times per second. Therefore, the decoder 15 repeatedly outputs each frame of the original video data four times, thereby creating input video data of 60 FPS for the output rate although the content update is substantially 15 FPS.

  The frame interpolation unit 20 stores the input video data input from the decoder 15 in the frame memory, and creates an interpolation frame indicating the intermediate state content from the frame whose content has changed, thereby improving the substantial frame rate. Then, a process of displaying on the display unit 17 is performed.

  The operation of the frame interpolation unit 20 will be further described with reference to FIG. As shown in FIG. 2, since the input video data is substantially 15 FPS video data input at 60 Hz, the same data is input every four frames during normal operation. In FIG. 2, input frames fi1 to fi4, fi5 to fi8, and fi9 to fi12 are frames having the same contents.

  Therefore, the frames fi1 and fi5 are extracted, and an interpolation frame indicating an intermediate state between the frames fi1 and fi5 is created and used as the contents of the frames fi3 and fi4. Similarly, the interpolated frame indicating the intermediate state between the frames fi5 and fi9 is the contents of the frames fi7 and fi8, the frames fi9 and fi13 are extracted, and an interpolated frame indicating the intermediate state between the frames fi9 and fi13 is created. The contents of the frames fi11 and fi2.

  In this manner, by extracting one image every four frames and creating an interpolation frame, a smooth video in which the inward direction is substantially updated at 30 FPS can be obtained.

  However, as shown in the figure, if the number of repetitions varies depending on the time required for decoding, an interpolation error such as creating an interpolated frame from frames with the same content or skipping intervening frames. Occurs and the video quality deteriorates.

  In the figure, when an error occurs, input frames fi1 to fi6, fi7 to fi8, and fi9 to fi12 are frames having the same contents. Here, if an interpolated frame is created from an image extracted every four frames as in the normal case, the interpolated frame created from the frames fi1 and fi5 has the same contents as the original frame, and the interpolation does not function. . In addition, the interpolated frame created from the frames fi5 and fi9 is created by skipping the second frame of the original video data, and the video may be deteriorated.

  Therefore, the frame interpolation unit 20 compares the data of the preceding and succeeding frames as shown in FIG. 3, and detects the data mismatch, that is, the update of the frame contents, and controls the operation of the frame interpolation.

  As illustrated in FIG. 1, the frame interpolation unit 20 includes a frame capturing unit 21, an interpolation processing unit 22, a frame comparison unit 23, a counter 24, and an interpolation operation control unit 25. The frame capturing unit 21 is a processing unit that receives an input video frame input from the decoder 15 and stores it in the frame memory 16. The interpolation processing unit 22 reads out the data stored in the frame memory 16 and performs interpolation. This is a processing unit that is displayed on the display unit 17.

  The frame comparison unit 23 is a processing unit that compares each frame of the input video data received by the frame capture unit 21 with the previous frame, determines whether the contents match, and outputs the result.

  Although any method can be used for the comparison by the frame comparison unit 23, here, the method of comparing the sum of the pixel values of the frame (checksum) and the value of each pixel of the frame are compared between the frames. A method (verify) will be described.

  FIG. 4 is an explanatory diagram for explaining frame comparison by checksum. First, a buffer memory for even frames and a buffer memory for odd frames are prepared. When valid data is obtained for a certain frame, the data is added and the total value is held. The valid data is similarly added to the next frame, and when the valid data is completed, the total value of the data is compared with the total value of all the frames. As a result, if the total values match, it is determined that the contents are the same as all frames, and if they do not match, it is determined that the frame contents have been updated.

  Here, as the data to be added, for example, luminance information and color information may be used alone, or luminance information and color information may be used in combination.

  FIG. 5 is an explanatory diagram for explaining frame comparison by verification. In the method shown in the figure, the first frame is simply stored in the frame memory, and when the next frame is received, the value of each pixel (dot) is set to the value of the dot at the same position in the previously stored frame. Compare. Then, when all the pixel values match the previous frame, it is determined that the frame contents are the same, and when there are dots having different pixel values, it is determined that the frame contents are updated.

  Returning to FIG. 1, the counter 24 is added when “match” is output as a result of the frame comparison unit 23, and is reset when “mismatch” is output. The interpolation operation control unit 25 controls the operation of the interpolation processing unit 22 based on the value when the counter 24 is reset, that is, the timing when the frame contents are updated. Specific control contents by the interpolation operation control unit 25 include on / off control of the interpolation operation and control of the capturing timing of the frame used for the interpolation.

  FIG. 6 is an explanatory diagram for explaining the control contents when the interpolation operation control unit 25 performs on / off control of the interpolation operation. In the example shown in the figure, the frame fi1 to the frame fi5 match, and the mismatch determination is output in the frame fi6 (the determination is output after the input of the frame fi6 is completed, so the output timing of the determination result is the frame This is the last of fi6). Therefore, the counter value at the time of mismatch detection is “5”.

  When the input video data is normally updated at 15 FPS, the counter value when the mismatch is detected always satisfies 4n-1 (n is a natural number), so the interpolation operation control that detects the mismatch with the counter value "5". The unit 25 considers that the input video data has not been updated normally at 15 FPS, and stops the interpolation operation.

  Thereafter, the coincidence determination is output until the frame fi9, and the mismatch determination is output at the frame fi10. At this time, the counter value is “3”, which satisfies 4n−1. Therefore, the interpolation operation control unit 25 assumes that the update of the input video data is performed at normal 15 FPS and starts the interpolation operation.

  FIG. 7 is an explanatory diagram for explaining the control contents when the interpolation operation control unit 25 controls the capture timing of frames used for interpolation. In the example shown in the figure, the frames fi1 to fi5 match, and a mismatch determination is output at frame fi6. Therefore, the counter value at the time of mismatch detection is “5”.

  When the input video data is normally updated at 15 FPS, the counter value when the mismatch is detected always satisfies 4n-1 (n is a natural number), so the interpolation operation control that detects the mismatch with the counter value "5". The unit 25 changes the frame capture timing used for interpolation.

  Specifically, since frames for interpolation are captured every four frames starting from the frame fi6 in which the mismatch is detected, the frames fi10, fi14, fi18,... Are used as the frames for interpolation. Note that since the input of the frame fi6 has already been completed and the capture is not in time, the next frame fi7 is used as a substitute.

  Next, the processing operation of the frame interpolation unit 20 will be described with reference to FIG. The flowchart shown in the figure is executed each time the frame interpolation unit 20 receives a frame input from the decoder 15.

  First, the frame comparison unit 23 determines whether or not the frame input to the frame capture unit 21 is the first frame (step S101), and if it is the first frame (step S102, Yes), it is for comparison. Is stored (step S102), and the process is terminated.

  On the other hand, if it is after the second frame (step S101, No), the frame comparison unit 23 stores the comparison data (step S103), compares the frames before and after (step S104), and the frames before and after match. Whether or not (step S105).

  As a result, if they match (step S105, Yes), the counter 24 is incremented (step S106) and the process is terminated. If they do not match (step S105, No), the interpolation operation control unit 25 It is determined whether or not the value of the counter 24 satisfies 4n−1 (step S107).

  When the value of the counter satisfies 4n−1 (step S107, Yes), the interpolation operation control unit 25 determines whether the interpolation process is being executed (step S108). (Step S108, Yes) The counter is reset (Step S110), and the process ends. If the interpolation process is not being executed (No at step S108), the interpolation process is started (step S109), the counter is reset (step S110), and the process is terminated.

  When the value of the counter does not satisfy 4n−1 (No at Step S107), the interpolation operation control unit 25 determines whether or not the interpolation process is being performed (Step S111), and the interpolation process must be being performed. (No in step S111), the counter is reset (step S110), and the process is terminated. If the interpolation process is being executed (step S111, Yes), the interpolation process is stopped (step S112), the counter is reset (step S110), and the process ends.

  Here, the processing operation in the case of performing the on / off control of the interpolation operation has been described, but the interpolation frame capturing timing control can be executed by the same processing.

  As described above, the one-seg receiving apparatus 1 according to the present embodiment generates 60 Hz input video data by repeating each frame based on 15 FPS one-seg video data, and then the frame contents before and after the input video data are changed. Since the frame content change timing is acquired by determining whether or not they match and the interpolation operation is controlled, the video quality is improved at low cost without providing a dedicated signal line or the like from the decoder 15 to the frame interpolation unit 20. can do.

  As described above, the video interpolating apparatus and video interpolating method according to the present invention are useful for improving the quality by increasing the frame rate of video data, and are particularly suitable for controlling the interpolation operation in a low-cost apparatus.

It is a schematic block diagram which shows the schematic structure of the one-segment receiver which is an Example of this invention. It is explanatory drawing explaining frame interpolation. It is explanatory drawing explaining the interpolation operation control based on the comparison of the frame content. It is explanatory drawing explaining the frame comparison by a checksum. It is explanatory drawing explaining the flame | frame comparison by verification. It is explanatory drawing explaining the on / off control of the interpolation by a frame comparison result. It is explanatory drawing explaining the taking-in timing control of the flame | frame for interpolation by a frame comparison result. It is a flowchart explaining the process simultaneous of a frame interpolation part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 One-segment receiver 11 Antenna 12 Tuner 13 Video recording medium 14 Switching part 15 Decoder 16 Frame memory 17 Display part 20 Frame interpolation part 21 Frame capture part 22 Interpolation process part 23 Frame comparison part 24 Counter 25 Interpolation operation control part

Claims (8)

Receiving input video data output at a second frame rate higher than the first frame rate by repeating each frame a predetermined number of times for the original video data whose frame is updated at the first frame rate, A video interpolation device that interpolates and outputs the contents of a repeated frame in input video data from frames positioned before and after the frame,
Comparison means for comparing the contents of successive frames of the input image data;
Update timing detection means for detecting timing at which the frame contents are updated;
Interpolation operation control means for controlling the operation related to the interpolation based on the detection result by the update timing detection means,
A video interpolating apparatus comprising:   2. The video interpolating apparatus according to claim 1, wherein the update timing detecting means counts the number of times a frame having the same content is output and detects whether or not the frame content is updated at a predetermined timing.   The video interpolation device according to claim 1, wherein the interpolation operation control unit starts the interpolation processing when a frame is updated at a predetermined timing.   The video interpolation device according to claim 1, wherein the interpolation operation control unit stops the interpolation processing when a frame update is performed at a timing other than a predetermined timing.   The interpolation operation control means changes the frame to be subjected to the interpolation processing and the predetermined timing when a frame update is performed at a timing other than the predetermined timing. The video interpolating device according to one.   6. The video interpolating apparatus according to claim 1, wherein the comparison unit calculates a sum of pixel values of frames and compares the total value between frames.   The comparison means compares the value of each pixel of a frame between frames, and determines that the contents of the compared frames match when all pixel values match between frames. The video interpolating device according to any one of claims 6 to 6. Receiving input video data output at a second frame rate higher than the first frame rate by repeating each frame a predetermined number of times for the original video data whose frame is updated at the first frame rate, A video interpolation method for interpolating and outputting the contents of repeated frames in input video data from frames positioned before and after the frame,
A comparison step of comparing the contents of successive frames of the input image data;
An update timing detection step for detecting a timing at which the frame content is updated;
An interpolation operation control step of controlling an operation related to the interpolation based on a detection result of the update timing detection step;
A video interpolation method characterized by comprising:

Images