JP4951487B2 - Video processing apparatus and video display apparatus using the same - Google Patents

Description

  The present invention relates to a video processing technique for generating an interpolation frame from a video frame included in a video signal and performing frame rate conversion.

  For example, in a video processing device such as a television display device, the number of frames per unit time of a video signal (frame rate) is reduced in order to improve the performance of moving images by reducing blurring feeling (afterimage feeling) and rattling in moving image display. There may be a case where a frame rate conversion process is performed to increase the resolution in the time direction to increase. Such frame rate conversion processing typically detects an object motion vector from a difference in video data between a plurality of (typically two) frames included in a video signal, and creates an interpolation frame. It includes processing for insertion into the frame sequence of the video signal. As a conventional technique for obtaining a motion vector for creating an interpolation frame with high accuracy, for example, those described in Patent Document 1 and Non-Patent Document 1 are known.

Japanese Unexamined Patent Publication No. 2002-27414 (paragraph 0009, FIG. 9) Toshiba Review Vol.59 No.12 (2004)

  However, the above prior art does not take into consideration an image in which the entire image moves in a certain direction, that is, the screen pans. Here, “pan” means that the entire screen moves not only in the horizontal (left and right) direction, but also in the vertical (up and down) direction and other directions. In the case of a video scene in which the entire screen is panned (hereinafter, this video scene is referred to as “pan video”), the motion vector of an object in the video may not be detected well at the end of the screen. This will be described with reference to FIG.

  FIG. 13 shows a state of a pan video in which the entire screen moves in the direction of the arrow 135. In the pan video, an object 134 in the video that does not appear in the frame 131 in the video signal suddenly appears at the end of the frame 132 that is temporally continuous with the frame (after one vertical cycle with respect to the frame 131). An appearing phenomenon occurs. At this time, for example, even if an attempt is made to detect the movement of the object 134 within the range of the block 133, the object 134 exists in the frame 132 but the object 134 does not exist in the frame 131. As a result, the motion of the object 134 cannot be detected at the edge of the screen. For this reason, even if an interpolated frame is created for such a pan video and the frame rate conversion process is performed, particularly in the area including the edge of the screen, it is video-related to the (original) frame in the video signal. There is a possibility that no or low interpolation frame is created, and there is a possibility that the image quality is deteriorated in the video after the frame rate conversion. With regard to this problem, a method of solving the problem by increasing the number of frames to be referred to when generating a frame is also conceivable, but it is essential to increase the amount of processing calculation, the circuit scale, and the memory used.

  The present invention has been made in view of the above-described problems of the prior art, and provides a technique capable of performing frame rate conversion processing with high image quality even when an input video is a pan video.

  The present invention is characterized by the structures described in the claims. That is, the present invention creates a frame by performing interpolation using video (object) motion vectors detected from a plurality of frames in an input video signal, and inserts this into a frame sequence of the input video signal. In the case where the input video is a pan video, for the region including the edge of the video, interpolation processing is performed using a global vector indicating the overall direction of motion of the screen instead of the motion vector. It is a feature.

  According to the present invention, it is possible to perform frame rate conversion processing with high image quality even when the input video is a pan video.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus according to the present invention. Here, a television display device will be described as an example of the video processing device. Of course, the present invention can also be applied to video processing devices other than television display devices. In FIG. 1, a video signal is input to the input terminal 11. This input video signal is assumed to be a digital video signal reproduced by, for example, a DVD, a Blu-ray disc or an HDD player, or a digital TV signal transmitted by digital TV broadcasting. When the input video signal is in an analog format, it is converted into a digital signal by an A / D converter (not shown) and input to the input terminal 11.

  When the definition of the input video signal does not match the resolution of the display unit 16, for example, when the definition of the input video signal is 640 × 480 and the resolution of the display unit 16 is 1920 × 1080, the resolution conversion unit 12 inputs the input video signal. Is enlarged / reduced to a format matching the resolution of the display unit 16. The signal subjected to the enlargement / reduction process by the pixel interpolation process in the resolution converter 12 is supplied to a frame rate converter (FRC) 13. When the definition of the input video signal matches the resolution of the display unit 16, the input video signal is supplied to the FRC 13 without going through the resolution conversion unit 12. Note that the resolution conversion processing unit 12 may be provided in the subsequent stage of the FRC 13 and perform the above-described enlargement / reduction processing on the output signal from the FRC 13.

  The FRC 13 is connected to an image memory 14 that stores one or a plurality of frames of video data of an input video signal (output signal from the resolution conversion processing unit 12), and writes video data of the input video signal to the image memory or By reading, video data of two frames that are temporally continuous is obtained. The FRC 13 calculates a difference between the video data of the two frames, detects a motion vector of the video (object) between the two frames, creates an interpolation pixel or an interpolation block using the detected motion vector, and performs interpolation. Generate a frame. The interpolated frame is inserted into the frame sequence of the input video signal (output signal from the resolution conversion processing unit 12) to convert the frame rate. Details of the frame rate conversion processing will be described later.

The FRC 13 in this embodiment performs frame rate conversion in accordance with the FRC conversion mode signal 17 input to the external terminal 18. The FRC conversion mode signal 17 is a signal for switching the frame rate conversion processing mode in the FRC 13, and this frame rate conversion processing mode includes, for example, the following in this embodiment.
(1) A mode for converting a video signal having a frame rate (vertical frequency) of 60 Hz into a frame rate of 120 Hz.
(2) A mode for converting a video signal having a frame rate of 50 Hz into a frame rate of 100 Hz.
(3) A mode for converting a video signal having a frame rate of 24 Hz (cinema format) into a frame rate of 60 Hz.
(4) A mode in which a telecine signal having a frame rate of 60 Hz in 2-3 pull-down format is converted to a frame rate of 60 Hz in non-pull-down format.

  In the mode (1), the frame rate is converted to 120 Hz by inserting an interpolation frame every other frame of the input video signal having a frame rate of 60 Hz. In the mode (2), the frame rate is converted to 100 Hz by inserting an interpolation frame every other frame of the input video signal having a frame rate of 50 Hz. In the mode (3), one interpolation frame is inserted after a certain frame of the input video signal having a frame rate of 24 Hz, and two interpolation frames are inserted into the next frame temporally adjacent to the certain frame. Convert the frame rate to 60 Hz. In the mode (4), for example, three or four frames out of a total of five frames, that is, two frames in which the same video in the video signal is continuous and three frames in which another same video is continuous are displayed. This mode replaces the interpolation frame. In this mode (4), the frame rate of the input signal does not change, but the number of frames constituting a moving image per unit time (1 second) increases (increases from 24 to 60) by the interpolation frame. The mode is also a frame rate conversion process. In the above description, four modes are selected by the FRC conversion mode signal 17, but it is obvious that other modes (for example, a mode for converting the frame rate from 60 Hz to 180 Hz) may be provided. is there.

  The video signal subjected to the frame rate conversion processing by the FRC 13 is supplied to the timing controller unit 15. The timing controller unit 15 creates a timing signal (horizontal and vertical synchronization timing signal) that is optimal for displaying the video signal subjected to the frame rate conversion processing from the FRC 13 on the display unit 16. The created timing signal and the video signal subjected to the frame rate conversion processing are supplied to a display unit 16 constituted by a flat panel such as an LCD panel, PDP, or organic EL, for example, and frame rate conversion is performed in synchronization with the timing signal. The processed video is displayed.

  Next, details of the FRC 13 will be described with reference to FIG. The FRC 13 according to the present embodiment includes a motion vector detection unit 24, a pan determination / global vector calculation unit 26 (hereinafter simply referred to as “pan determination / GV calculation unit”), an interpolation frame generation unit 29, and the image memory. A memory interface (memory I / F) 23 that communicates video data and read / write commands is provided.

  The FRC 13 is sequentially supplied with frames of input video signals (output signals from the resolution conversion processing unit 12) and the FRC conversion mode signal 17 described above. Here, it is assumed that the frame currently input to the FRC 13 is the current frame 21. The video data of the current frame 21 is input to the motion vector detection unit 24 and temporarily stored in the image memory 14 described above via the memory I / F 23. Since the image memory 14 has a function of delaying the video data by one frame in this example, the output from the image memory 14 read through the memory I / F 23 is one frame from the current frame 21. This is the video data of the previous frame 22 before the period. The motion vector detection unit 24 obtains a difference in video data between the two frames from the input video data of the current frame 21 and the previous frame 22 in accordance with the conversion mode specified by the FRC conversion mode signal 17. Alternatively, the motion vector 25 is detected and output in units of blocks composed of a predetermined number of pixels.

  An example of motion vector detection in the motion vector detection unit 24 will be described with reference to FIG. FIG. 3 shows an example in which a motion vector is obtained not in units of blocks but in units of pixels, and the motion vector in units of pixels is present in the current frame 21 at a point-symmetrical position with respect to an interpolation pixel 30 in the interpolation frame 20. And a pair of pixels on the previous frame 22 is obtained and calculated.

  In FIG. 3, t indicates the frame time direction. Here, for the sake of convenience, the coordinates of a certain interpolation pixel 30 in the interpolation frame 30 are set to (0, 0).

  First, the motion vector detection unit 24 sets search windows 31 and 32 indicating the search range of motion vectors for each of the current frame 21 and the previous frame 22. The search window 31 of the current frame 21 has, for example, a size of 5 pixels in the vertical direction and 11 pixels in the horizontal direction centering on the pixel 33 of the current frame 21 that is in the same spatial position as the interpolation pixel 30. Similarly, the search window 32 of the previous frame 22 has, for example, a size of 5 pixels in the vertical direction and 11 pixels in the horizontal direction centering on the pixel 34 of the current frame 21 at the same spatial position as the interpolation pixel 30. Note that the coordinates of the pixels 33 and 34 are also (0, 0) for convenience here.

  Next, a straight line passing through the search window 31 of the current frame 21 and the search window 32 of the previous frame 22 is set around the interpolation pixel 30. For example, if the coordinates of the pixel at the lower left corner of the search window 21 are (−5, −2), the pixel in the search window W4 on the straight line connecting this pixel and the interpolation pixel 30 is the upper right pixel. The coordinates are (5, 2). This straight line is set for all the pixels in the search windows 31 and 32. In this example, since the number of pixels of the search windows 31 and 32 is 11 × 5 = 55, 55 straight lines are set as straight lines passing through the interpolation pixel 30.

  Subsequently, the difference between the pixels in the search window 31 and the pixels in the search window 32 through which each straight line passes is calculated for each of the 55 straight lines. Here, the difference between the luminance signals of the respective pixels is obtained. A straight line having a pair of pixels having the smallest difference is set as the motion vector of the interpolation pixel 30. In the example of FIG. 3, the pair 35 of the pixel 35 (coordinate is (−5, −2)) in the search window 31 and the pixel 36 (coordinate is (5,2)) in the search window 32 has the smallest difference. And Accordingly, a straight line connecting the pixel 35, the interpolation pixel 30, and the pixel 36 is set as the motion vector 25 of the interpolation pixel 30 (or pixel 35, pixel 36). That is, it is estimated that the pixel 35 of the current frame 21 moves to the pixel 36 in the previous frame 22 through a pixel that is positionally equal to the interpolation pixel 30 of the interpolation frame 20 according to the direction indicated by the motion vector 25. The

  In this way, the motion vector detection unit 24 detects the motion vector 25 for all the interpolation pixels. In the above example, the motion vector is detected for each pixel, but may be detected for each block. For example, each square of the search windows 31 and 32 is a block composed of N pixels in the horizontal direction and N pixels in the vertical direction (N is, for example, 4, 8 or 16), and there is a difference between the search windows 31 and 32. The motion vector may be detected by a so-called block matching method for obtaining a pair of blocks that is the minimum.

  The detected motion vector 25 is input to the interpolation frame generation unit 29 together with the FRC conversion mode 17, the current frame 21 and the previous frame signal 22, and is also input to the pan determination / GV calculation unit 26. The interpolation frame generation unit 29 performs interpolation processing for each pixel (or block) using the motion vector 25, the current frame 21, and the previous frame 22 in accordance with the conversion mode specified by the FRC conversion mode signal 17. This interpolation processing is processing for calculating, for example, the pixel value of the interpolation pixel or the interpolation block constituting the interpolation frame as the average value of the pixel value pair 36. That is, the interpolated frame generation unit 29 uses the video data of the pair 36 pixels (pixel 35 and pixel 36) indicated by the motion vector 25 from the motion vector detection unit 24 as video of the current frame signal 21 and the previous frame signal 22. The pixel values of the interpolation pixel or the interpolation block are calculated by extracting from the data and adding the video data together and dividing by a predetermined coefficient (the coefficient is 2 in the simple average). Depending on the FRC conversion mode signal 17, for example, the coefficient may be changed.

  One interpolation frame is created by performing the above interpolation processing for all the pixels or blocks constituting the interpolation frame 30.

  On the other hand, the pan determination / GV calculation unit 26 uses the detected motion vector 25 to calculate, for example, information on the appearance frequency (vector histogram) of one motion vector per current frame 21 (or previous frame 22), It is determined whether the input video is a pan video, and the global vector 28 is calculated. Here, the global vector is a motion vector indicating the motion direction of the entire video in a pan video where the video is moving as a whole. An example of the creation of the vector histogram, determination of pan video, and calculation of the global vector 28 will be described with reference to FIG.

  FIG. 4 shows an example of a vector histogram. For example, the X axis corresponds to the horizontal direction of the search window 31 shown in FIG. 3, and the Y axis corresponds to the vertical direction of the search window. That is, in this example, the search range of the global vector is horizontal ± 5 pixels and vertical ± 2 pixels. As shown in FIG. 4, within the search range (corresponding to the search window 31 or 32) of the current frame 21, the pixels (coordinates) through which the motion vector 25 has passed are counted over the entire current frame 21. Accumulation (count) of the frequency that the motion vector has passed for each pixel (coordinate) is obtained. The pan determination / GV calculation unit 26 compares the count value for each coordinate with a predetermined threshold value (for example, 30000), and when there is a coordinate having a count number equal to or greater than the threshold value in the vector histogram corresponding to a certain current frame 21. That is, when a motion vector pointing in the same direction in a certain video exists more than a predetermined threshold, the video is determined to be a pan video. A determination enable signal 27 is output from the pan determination / GV calculation unit 26 according to the determination result. The determination enable signal 27 not only indicates a determination result of whether or not the video is a pan video, but also indicates a period of a peripheral region where a motion vector cannot be detected well in the pan video, as will be described later. For example, “Hi”, otherwise “Low”.

  Further, among the coordinates having a count number equal to or greater than a predetermined threshold, the coordinate having the largest count number, that is, the motion vector having the highest appearance frequency in the vector histogram is set as the global vector 28. In the example of FIG. 4, the motion vector having the maximum count number of the coordinates (−3, 0) surrounded by the circle 41, that is, the highest appearance frequency, is set and output as the global vector 28.

  Note that, as described above, in a pan video, a motion vector is detected as 0 in the peripheral area including the top, bottom, left, and right edges of the video, or is likely to be detected erroneously. The motion vector detected in the region of 50 pixels from the screen edge) may be excluded when the vector histogram is created.

  In an actual video, there is a case where a vector histogram in which movement is concentrated in one direction as shown in FIG. 4 cannot be obtained. For example, when the background is scrolling and the object of interest is moving in the direction opposite to the scrolling direction, the histogram has two peaks as shown by reference numerals 51 and 52 as shown in FIG. An existing vector histogram is obtained. Here, reference numeral 51 is, for example, a histogram indicating the movement of the background, and reference numeral 52 is a histogram indicating the movement of the object of interest. In this case, since the background and the object of interest are moving differently, as shown in FIG. 4, there is a case where a pan video cannot be detected satisfactorily only by making a determination using only one histogram peak. In that case, for example, two different histogram peaks (coordinates having a high motion vector count) are used, and when one of them is equal to or greater than a predetermined threshold and the other is equal to or less than the threshold, it is determined that the image is a pan video. May be.

  The determination enable signal 27 and the global vector 28 obtained by the pan determination / GV calculation unit 26 are input to the interpolation frame generation unit 29. In the interpolation frame generation unit 29, during the period when the determination enable signal 27 is “Hi”, the motion vector detection unit 24 detects that the input video is a pan video and the motion vector cannot be detected well in the peripheral region. Instead of the motion vector 25, the input global vector 28 is used for interpolation processing. On the other hand, during a period when the determination enable signal 27 is “Low”, a motion vector as shown in FIG. 3 is detected as a region where a motion vector can be detected satisfactorily, and interpolation processing is performed based on this motion vector. Thus, in the case of pan video, interpolation processing is performed using the detected motion vector 25 in the central region of the video, and interpolation processing is performed using the global vector 28 in the peripheral region including the video edge. In addition, the content of processing in the interpolation frame generation unit 24 is controlled to be switched. The configuration for generating the determination enable signal 27 and the details of the interpolation processing to the surrounding area will be described with reference to FIGS.

  The failure of the peripheral area including the edge of the screen when generating the interpolation frame is basically noticeable when the entire screen is panned greatly. That is, in a pan video with a large amount of motion, a correct motion vector cannot be detected for the surrounding area by the pixel-by-pixel matching or the block matching method as shown in FIG. Therefore, the motion vector of the peripheral region in the pan video is detected as 0 or a vector different from the original motion, depending on the algorithm for motion detection.

  For example, in the case of an image in which the entire screen is panned in the direction of an arrow 66 pointing in the horizontal direction as in the image shown in FIG. On the other hand, in a region 65 (region in the center of the screen) 65 between the peripheral regions 61 and 62 near the center of the screen (region in the center of the screen), it is easier to detect a correct motion vector than in the peripheral regions 61 and 62. For this reason, since the interpolation process is relatively correct (highly related to the input original video) in the center area of the screen, the boundary between the peripheral areas 61 and 62 and the central area 65 is visually determined. It is recognized and the viewer feels image quality degradation. Here, a region in which the motion vector is detected as 0 (or an error) like the peripheral regions 61 and 62 is referred to as Zero-BAND. In addition, the above phenomenon is recognized in the same way even in a video panning in the vertical direction. In this case, Zero-BAND is generated at the upper and lower ends of the screen.

  The width of Zero-BAND is determined by the size of the motion vector search range (that is, the search window 31 or 32 shown in FIG. 3) and the pan speed in the pan video. Here, the panning speed is the number of pixels that the screen moves per unit time (one frame (one vertical cycle)). How to determine the width of this Zero-BAND will be described below with reference to FIG.

  FIG. 7 shows a state in which a Zero-BAND 72 shown in gray is generated in a peripheral region of the central portion 71 in a pan video moving in the horizontal direction. It is assumed that Zero-BAND 72 does not occur in other areas. Here, assuming that the pan direction is only the horizontal direction, in FIG. 7, the horizontal width of the motion vector search range 31 (32) (the width based on the center of the search range 31) is ± Xd, and the horizontal pan Xv (≧ Xd), and the distance from the left or right end of the screen to an interpolation pixel (interpolation position) 30 is Lx. Here, the pan speed Xv is, for example, the number of pixels in which the video has moved in a unit time (one frame period), and the global vector 28 is used.

  When searching for a motion vector using the pixel-by-pixel matching or the block matching method as shown in FIG. 3, the width Z_ht of the horizontal Zero-BAND 72 appearing at both ends of the screen is expressed by the following equation (1).

  Here, MIN (Lx, Xv) means that the smaller one of Lx and Xv is selected. As is clear from Equation 1, the zero-band 72 in the horizontal direction does not exist when the distance Lx from the screen edge to the interpolation position 30 is greater than or equal to the width Xd of the motion vector search range 31 (32), and the distance Lx is the width. When it is smaller than Xd, the distance Lx or the horizontal pan speed Xv is smaller. In addition, since the presence of the Zero-BAND 72 is conspicuous in the pan video, the Zero-BAND 72 does not occur when the pan speed Xv is 0. Furthermore, since a motion exceeding the width Xd of the motion vector search range 31 (32) cannot be detected, the maximum value of the Zero-BAND 72 in the horizontal direction is the width Xd (= Xv (max)).

  Also in the pan video moving in the vertical direction, the width Z_vt of the zero-band 72 in the vertical direction can be obtained in the same manner as described above. That is, if the vertical width of the motion vector search range 73 is ± Yd, the vertical panning speed is Yv (≧ Yd), and the distance from the screen edge to the interpolation position 30 is Ly, the following equation 2 is obtained. Is done. The global speed 28 obtained from the histogram shown in FIG. 4 is used as the panning speed Xy.

Here, MIN (Ly, Yv) means that the smaller one of Ly and Yv is selected. As apparent from Equation 2, the zero-band 72 in the vertical direction does not exist when the distance Ly from the screen edge to the interpolation position 30 is equal to or larger than the width Yd of the motion vector search range 31 (32), and the distance Ly is the width. When it is smaller than Yd, the distance Ly or the vertical pan speed Yv is smaller. In addition, since the presence of the Zero-BAND 72 is conspicuous in the pan video, the Zero-BAND 72 in the vertical direction does not occur when the pan speed Yv is 0. Furthermore, since a motion exceeding the width Yd of the motion vector search range 31 (32) cannot be detected, the maximum value of the Zero-BAND 72 in the vertical direction is the width Yd (= Xy (max)).

  The pan determination / GV calculation unit 26 performs the above-described calculation using the above equations 1 and 2, and the determination enable signal 27h corresponding to the horizontal zero-band 72 and the determination enable corresponding to the vertical zero-band 72. A signal 27v is generated. As shown in FIG. 7, the Hi period of the determination enable signal 27h matches the width Z_ht of the Zero-BAND 72 in the horizontal direction, and the Hi period of the determination enable signal 27v is the width of the Zero-BAND 72 in the vertical direction. It matches Z_vt. The determination enable signals 27h and 28v obtained in this way are supplied to the interpolation frame generation unit 29. As described above, the interpolation frame generation unit 29 performs the interpolation process using the global vector when the determination enable signals 27h and 28v are “Hi”, and uses the detected motion vector 25 when the determination enable signals 27h and 28v are “Low”. The interpolation process is switched according to the level of the determination enable signal so that the interpolation process is performed.

  Here, Zero-BAND may not stand out depending on the amount of pan movement. Therefore, the predetermined threshold TH_Z is set in advance, and the pan determination / GV calculation unit 26 determines the above only when the pan speed Xv (or Yv) is larger than the threshold TH_Z (that is, when the pan movement amount is large). The enable signal 27 and the global vector 28 may be output.

  Next, interpolation processing and interpolation frame generation in the interpolation frame generation unit 29 using the determination enable signal 27 and the global vector 28 will be described with reference to FIG. FIG. 8 shows a state in which one interpolation frame 20 is inserted between the previous frame 21 and the current frame 22 in a pan video in which the global vector 28 is directed from the right to the left. In this example, it is assumed that the object 84 does not appear in the previous frame 21 but first appears in the current frame 22, that is, the object 84 enters from the right side of the screen between the two frames of the previous frame 21 and the current frame 22. Furthermore, it is assumed that the object 85 appears in the previous frame 21 but disappears in the frame 22, that is, the object 85 exits from the left side of the screen between the previous frame 21 and the current frame 22.

  As shown in FIG. 8, the interpolation frame generation unit 29 uses the motion vector 25 from the motion vector detection unit 24 in the Low period (period other than Zero-BAND 72) of the determination enable signal 27. By calculating, for example, the average of the pixel data of the indicated previous frame 21 and the current frame 22, an interpolation pixel subjected to motion correction is generated.

  On the other hand, during the period when the determination enable signal 27 corresponding to the width of the Zero-BAND 72 in the horizontal direction is Hi, an interpolation pixel is generated using the global vector 28, and in accordance with the direction of the global vector 28 and the position of the Zero-BAND 72. Different interpolation processing.

  For example, as shown in FIG. 8, since the global vector 28 is in the right-to-left direction toward the screen, the object 84 that enters from the right side of the screen, that is, the area of the zero-band 72 on the right side of the screen, Interpolated pixels are generated as follows.

  First, a pixel P1 on the current frame 22 side that is in a line-symmetrical position with respect to a certain interpolation pixel Pa (spatially same position as the interpolation pixel) is determined. Subsequently, an image P2 located from the pixel P1 in the direction indicated by the global vector 28 and shifted by the length (distance) of the global vector 28 is specified. Then, the interpolation pixel data is interpolated from the pixel data of the pixel P2. In other words, this interpolation processing generates a new peripheral interpolation vector 86 by synthesizing the global vector 28 with a straight line connecting a certain interpolation pixel and the pixel P1 that is in a line-symmetric position with this, and this peripheral interpolation vector. The interpolation pixel is generated using the pixel P2 of the current frame 22 indicated by 86 as it is. In this interpolation process, the previous frame 21 is not used. Such an interpolation process is referred to herein as current frame one-sided interpolation.

  On the other hand, for the object 85 disappearing from the left side of the screen, that is, the area of Zero-BAND 72 on the left side of the screen, interpolation pixels are generated as follows.

  First, a pixel P3 on the previous frame 21 side that is in a line symmetry with a certain interpolation pixel Pb is determined. Subsequently, an image P4 is specified that is located in the direction of the vector 28 'in the opposite direction (directly opposite) from the global vector 28 and shifted by the length (distance) of the global vector 28 from the pixel P3. Then, the interpolation pixel data is interpolated from the pixel data of the pixel P4. That is, this interpolation processing is performed by synthesizing a new line of peripheral interpolation by synthesizing the global vector 28 in a direction opposite to the direction of the global vector 28 and a straight line connecting a certain interpolated pixel and the pixel P1 that is in line symmetry with the interpolated pixel. A vector 87 is generated, and an interpolation pixel is generated using the pixel P4 of the previous frame 21 indicated by the peripheral interpolation vector 87 as it is. In this interpolation process, the current frame 22 is not used. Such an interpolation process is herein referred to as previous frame one-sided interpolation. Here, in the interpolation processing for the area of the zero-band 72 on the left side of the screen, since interpolation is performed from the previous frame 21 in the direction opposite to the temporal flow of motion, the vector in the direction opposite to the direction indicated by the global vector 28 is used. Is used.

  Contrary to the example of FIG. 8, when the global vector 28 is panned from the left to the right, that is, when the object 84 enters from the left side of the screen and the object 85 exits from the right side of the screen, The processing opposite to the example may be performed. That is, the previous frame one-sided interpolation is performed for the area of Zero-BAND 72 on the right side of the screen, and the current frame one-sided interpolation is performed for the area of Zero-BAND 72 on the left side of the screen. Furthermore, in the case of a panning image in which the entire screen moves from bottom to top, that is, an image in which the object 84 enters from the bottom of the screen and the object 85 exits from the top of the screen, the area of the zero-band 72 at the bottom of the screen Performs one-side interpolation of the current frame and performs one-side interpolation of the previous frame for the area of Zero-BAND 72 on the upper side of the screen. The opposite is true for panning where the entire screen moves from top to bottom.

  In the above example, when the current frame 21 or the previous frame 22 is interpolated, the size of the global vector 28 is used as it is, but the length of the global vector 28 is adjusted as necessary (for example, the length is halved). May be used.

  As described above, in this embodiment, when the video moves from one side to the other side (or up and down) as a whole, that is, when the global vector 28 is directed from one side to the other, the peripheral region on one side (Zero-BAND72 ) For the current frame forward interpolation, and backward interpolation for the previous frame for the other peripheral region. When the global vector 28 is oblique, the global vector 28 is divided into a horizontal component and a vertical component, and interpolation processing is separately performed for the upper and lower peripheral regions and the left and right peripheral regions using the respective component vectors. Also good.

  As described above, according to the present embodiment, in the case of a pan video, the interpolation processing of the surrounding town area is replaced with the global vector indicating the motion of the entire video instead of the motion vector obtained by the matching processing described above. Since this is performed, it is possible to reduce the failure of the video in the peripheral area. Therefore, according to the present embodiment, frame rate conversion with high image quality can be performed even for panned video. In this embodiment, since the interpolation processing is switched according to the direction of the global vector and the position of the peripheral region, it is possible to perform interpolation processing with higher accuracy.

  Next, a second embodiment according to the present invention will be described with reference to FIGS. FIG. 9 is a block diagram showing a second embodiment of the FRC 13, and the same components as those shown in FIG. 2 are denoted by the same reference numerals and description thereof is omitted.

  In this embodiment, as shown in FIG. 9, in the embodiment of FIG. 1, a still image area detecting unit 91 that detects a moving image area and a still image area from an input image signal and outputs information on the boundary position is provided. It is provided. Here, it is assumed that the still image area is an area 101 where characters and data by data broadcasting or the like are displayed, as shown on the right side of FIG. It is assumed that a panning video 102 is displayed on the left side of the still picture area 101 and the still picture area 101 and the moving picture 102 are separated by a boundary 103. That is, the present embodiment is used when frame rate conversion processing is performed on a video obtained by combining a moving image and a still video.

  In such a video, although the moving image 102 is a pan video, there is a possibility that the video is not detected as a pan video because the still video area 101 having a motion vector of 0 exists. In addition, when the pan video is detected, the interpolation process is performed with the global vector even though the video displayed in the still and still video area 101 is a still image. Incorrect interpolation may occur.

  In order to solve this problem, in this embodiment, the still image area detecting unit 91 detects the boundary 103 between the still image area 101 and the moving image 102 and generates a determination enable signal based on the detected boundary 103. . For example, in the case of a video as shown in FIG. 10, the still and still image area 101 is greatly different in luminance from the moving image 102 (for example, the still and still image area 101 has a higher luminance than the moving image 102), and the luminance difference ( Horizontal edge) becomes prominent. The still image area detection unit 91 detects the horizontal position where the edge exists in units of lines, and supplies the detected position to the pan determination / GV calculation unit 26 as information on the boundary 103. The pan determination / GV calculation unit 26 generates a determination enable signal 104 as shown in FIG. 10 based on the information on the boundary 103 from the still image area detection unit 91 and outputs the determination enable signal 104 to the interpolation frame generation unit 29. As shown in FIG. 10, the determination enable signal 92 is Hi in the peripheral area including both ends of the moving image 102 within the range of the moving image 102, and is Low on the still and still image area 101 side from the boundary 103. It has become. That is, the pan detection / GV detection unit 26 generates and outputs a determination enable signal 104 indicating Zero-BAND with respect to the peripheral area of the video excluding the characters and the data display portion that are still images based on the information of the boundary 103. .

  As a result, the interpolation frame generation unit 29 performs the same interpolation process using the global vector as in the first embodiment for the peripheral area including the left and right ends of the moving image 102 corresponding to the period in which the determination enable signal 104 is Hi. During the Low period, interpolation processing is performed using the motion vector detected by the motion vector detection unit 24. Since the motion vector in the still image area 101 is 0, the interpolation pixel or block corresponding to the still image area 101 in the interpolation frame has the same spatial position as the interpolation pixel or block in the previous frame. Create interpolation data from

  Further, if a line memory (not shown) is provided in the still video area detection unit 91 and video data for one or more lines of video is stored in the line memory, a luminance difference between the plurality of lines can be detected. That is, according to such a configuration, it is possible to detect a luminance difference between lines, that is, a vertical boundary (vertical edge), and generate a determination enable signal indicating the zero-band of the moving image 102 in the vertical direction. Can do.

  As described above, in this embodiment, even when the video is a video in which a still video is combined with the video, and the video is a pan video, interpolation processing can be performed on the peripheral area of the video using a global vector. Furthermore, interpolation processing can be performed so that motion correction is not performed on still images. Therefore, according to the present embodiment, it is possible to perform a high-quality frame rate conversion process even for a composite video including a pan video as described above.

  Next, a third embodiment according to the present invention will be described with reference to FIGS. The present embodiment is characterized in that global vectors are individually detected in the peripheral area of the video, for example, the left and right sides or the upper and lower sides of the video. That is, in this embodiment, two systems corresponding to the left and right sides or the upper side and the lower side of the video are provided for the zero-band determination enable signal 27 and the output of the global vector 28 in the pan determination / GV calculation unit 26, respectively. Is.

  As described above, the motion vector is detected as 0 or an incorrect vector at the end of the video (hereinafter, such detection is referred to as error detection). For this reason, in this embodiment, a global vector is set by detecting a motion vector from a peripheral region excluding a video end portion where error detection is likely to occur. FIG. 11 shows the concept of global vector setting according to the present embodiment. FIG. 11 shows an example in which global vectors are set on both the left and right sides in the horizontal direction, and the first peripheral areas 110 and 111 close to the video edge are areas where error detection is likely to occur. Each of the first peripheral areas 110 and 111 has a width of, for example, 50 pixels from the video end. On the other hand, the second peripheral regions 112 and 113 adjacent to the video central region 116 side of the first peripheral regions 110 and 111 are regions that are considered to have a lower frequency of error detection than the first peripheral region. It shall have a width of ~ 300 pixels.

  The pan determination / GV calculation unit 26 creates a vector histogram as shown in FIG. 4 for each of the second peripheral regions 112 and 113 using the motion vector detected as in the first embodiment. At this time, the pan determination / GV calculation unit 26 detects an error in the first peripheral regions 110 and 111, and excludes the detected motion vector from the creation of the vector histogram. Then, in each of the second peripheral regions 112 and 113, the pan determination / GV calculation unit 26 sets motion vectors that are equal to or higher than the predetermined threshold and have the highest appearance frequency as the global vectors 114 and 115, as described above. . Here, the global vector 114 is a global vector corresponding to the second peripheral area 112 located on the left side of the video, and the global vector 115 is a global vector corresponding to the second peripheral area 113 located on the right side of the video.

  Then, the pan determination / GV calculation unit 26 outputs the two global vectors 114 and 115 and the two determination enable signals obtained by using the global vectors to the interpolation frame generation unit 24. The interpolation frame generation unit 24 performs interpolation processing individually for each of the second peripheral regions 112 and 113 using the method shown in FIG. 8 using the two sets of global vectors and the determination enable signal. .

  As described above, in this embodiment, the motion vectors detected in the first peripheral regions 110 and 111 are not used for setting the global vector, and the motion vectors detected in the second peripheral regions 112 and 113 are used as the right side of the screen. A global vector is set separately for the left side. As a result, this embodiment can set the global vector more accurately, for example, even in a pan video where the right half of the video moves from left to right as a whole and the left half of the video moves from right to left. Accurate interpolation processing corresponding to various movements can be performed.

  Although FIG. 11 shows an example in which global vectors are set on both the left and right sides of the video, global vectors may be set on the top and bottom. In this case, a first peripheral area and a second peripheral area are set at the top and bottom of the video shown in FIG. Further, a first peripheral region and a second peripheral region are provided at four locations on the top, bottom, left, and right of the video, respectively, and four global vectors and determination enable signal outputs from the pan determination / GV calculation unit 26 are provided. Alternatively, a global vector may be set individually to perform interpolation processing.

  Furthermore, by applying the third embodiment, a global vector 114 set individually in each of the second peripheral regions 112 and 113 in the pan video using the global vector 117 set from the motion vector of the entire video. And the reliability of 115 may be confirmed. That is, in this application example, the difference between the global vector 114 or 115 in the peripheral area and the global vector 117 in the video central area 116 is detected, and the reliability of the global vector in the peripheral area is determined according to this difference. For example, since the global vector 114 indicates the same direction as the global vector 117, it is determined that the reliability is high, and since the global vector 115 indicates the opposite direction to the global vector 117, it is determined that the reliability is low.

  For the region on the left side of the video corresponding to the global vector 114 determined to have high reliability, the above-described previous frame one-side interpolation is performed using the global vector 114 or 117. On the other hand, with respect to the region on the left side of the video corresponding to the global vector 115 determined to have low reliability, interpolation processing is performed with the motion vector 25 detected by the motion vector detection unit 24 on the left side or with the motion vector set to 0. . Interpolation processing when the motion vector is 0 is processing that performs interpolation from a previous or current frame pixel that is in a line-symmetric position with a certain interpolation pixel or block (which is spatially the same position as the interpolation pixel). is there. Alternatively, an interpolation process in which these are mixed is performed. Since the movement on the left side of the image in FIG. 11 is the opposite movement to the image center area 116 in FIG. 11, the left side of the image is considered to have a low possibility of the occurrence of Zero-BAND due to panning. It is.

  The reliability is not determined only by whether or not the global vector 117 in the screen center region 116 is opposite. For example, if the motion direction of the video is divided into four regions (regions 1 to 4) as shown in FIG. 12, and the global vector 114 of a certain peripheral region belongs to the same region as the global vector 117 of the screen center region 116, the reliability May be determined to be high.

  As described above, according to the present embodiment, it is possible to perform interpolation processing with higher accuracy by determining the reliability of the global vector.

1 is a block diagram illustrating a configuration example of a video processing apparatus to which the present invention is applied. The figure which shows the example of 1 structure of FRC13 which concerns on 1st Example of this invention. The figure which shows an example of a motion vector detection and interpolation frame production | generation. The graph which shows an example of the vector histogram for global vector setting. The graph which shows the other example of a vector histogram. The figure which shows an example of Zero-BAND. The figure which shows the relationship between Zero-BAND and the determination enable signal 27. FIG. The figure which shows an example of the interpolation process using a global vector. The figure which shows the example of 1 structure of FRC13 which concerns on 2nd Example of this invention. The figure which shows an example of the image | video by which the still image and the moving image were synthesize | combined. The figure which shows the concept of 3rd Example which concerns on this invention. The figure regarding determination of the reliability of a global vector. The figure which shows an example of the detection of the motion vector by a matching process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Input terminal, 12 ... Resolution conversion part, 13 ... Frame rate conversion (FRC: Frame Rate Conversion) part, 14 ... Image memory, 15 ... Timing control part, 16 ... Display unit: 17 ... FRC conversion mode signal, 20 ... interpolation frame, 21 ... current frame signal, 22 ... previous frame signal, 23 ... memory interface, 24 ... motion vector detection unit , 25 ... motion vector, 26 ... pan determination / GV calculation unit, 27 ... determination enable signal, 28 ... global vector, 29 ... interpolation frame generation unit.

Claims (17)

In a video processing apparatus configured to insert an interpolated frame into a video signal having a plurality of video frames to perform frame rate conversion,
A first motion vector acquisition unit that acquires a first motion vector indicating a motion of a video for each pixel or predetermined block using at least two frames included in the video signal;
An interpolation frame generation unit for generating an interpolation frame by creating an interpolation pixel / block using the first motion vector acquired by the motion vector acquisition unit;
A second motion vector acquisition unit that sets, as a second motion vector, a first motion vector acquired by the first motion vector acquisition unit that is equal to or higher than a predetermined appearance frequency,
The interpolation frame generation unit creates interpolation pixels / blocks in a peripheral region including an end of the video using the second motion vector set by the second motion vector acquisition unit. A video processing apparatus characterized by that. 2. The video processing device according to claim 1, wherein the second motion vector acquisition unit obtains a first motion vector having a maximum appearance frequency among the first motion vectors, and a first motion vector having the maximum appearance frequency is obtained. If the number of motion vectors is equal to or greater than a predetermined number, the first motion vector is set as the second motion vector,
When the first motion vector having the maximum appearance frequency is equal to or greater than a predetermined number, the interpolation frame generation unit uses the second motion vector to perform the interpolation pixel / block in the peripheral region including the edge of the video A video processing apparatus characterized by creating an image. In a video processing apparatus configured to insert an interpolated frame into a video signal having a plurality of video frames to perform frame rate conversion,
A first motion vector acquisition unit that acquires a motion vector of a video for each pixel or predetermined block using at least two frames included in the video signal;
In the motion vector acquired by the first motion vector acquisition unit, when a predetermined number or more of the same first motion vector exists, the same first motion vector is set as the second motion vector. A second motion vector acquisition unit;
An interpolation pixel / block of a first area including a central portion of the video is created using the first motion vector, and an interpolation pixel of a second area including an edge of the video is generated using the second motion vector. An interpolation frame generation unit that generates a block by creating a block;
A video processing apparatus comprising: In a video processing apparatus configured to insert an interpolated frame into a video signal having a plurality of video frames to perform frame rate conversion,
A motion vector detection unit that detects a motion vector of an object using at least two frames included in the video signal;
A determination unit for determining whether the video is moving as a whole;
A global vector calculation unit that sets a global vector indicating the overall motion of the video;
When it is determined by the determination unit that the video is moving as a whole, interpolation processing of a peripheral region including an end of the video is performed using the global vector obtained by the global vector calculation unit, An interpolation frame generation unit that generates an interpolation frame by performing interpolation processing of a region other than a peripheral region using the motion vector acquired by the motion vector acquisition unit;
A video processing apparatus comprising:   5. The video processing device according to claim 4, wherein when the motion vector detected by the motion vector detection unit includes a predetermined number or more of the same motion vector, the determination unit moves the video as a whole. A video processing device characterized in that it is determined that the image is present.   6. The video processing apparatus according to claim 5, wherein the global vector calculation unit includes the same motion vector when a predetermined number or more of the same motion vectors exist in the motion vector detected by the motion vector detection unit. Is set as the global vector.   5. The video processing device according to claim 4, wherein when the motion vector having the highest appearance frequency among the motion vectors detected by the motion vector detection unit is a predetermined number or more, the determination unit A video processing apparatus characterized in that it is determined to be moving.   The video processing apparatus according to claim 7, wherein the global vector calculation unit performs a motion when a motion vector having the highest appearance frequency among the motion vectors detected by the motion vector detection unit exceeds a predetermined number. A video processing apparatus characterized in that a vector is set as the global vector. In a video processing apparatus for processing a video signal having a plurality of video frames,
A motion vector acquisition unit that acquires a motion vector indicating a motion of a video for each pixel or predetermined block using at least two frames included in the video signal;
An interpolation frame generation unit that performs an interpolation process from the video frame using the motion vector acquired by the motion vector acquisition unit, and generates an interpolation frame to be inserted into a frame sequence of the image signal;
A pan determining unit that determines whether the entire image is panned up and down or left and right;
A global vector calculation unit that calculates a movement in the pan direction as a global vector when the pan determination unit determines that the entire image is panned,
The interpolation frame generation unit, when the pan determination unit determines that the entire video is panned, performs an interpolation process on a region including an end portion of the video by the global vector calculation unit. An image processing apparatus characterized by using a vector.   The video processing device according to claim 9, wherein the interpolation frame generation unit starts from a current frame with respect to a frame having an object entering the screen temporally for a region including an end portion of the video. An image processing apparatus characterized in that an interpolation process using the global vector is performed on a frame having an object that exits from the screen from the previous frame.   In the video processing device according to claim 10, for each of the top, bottom, left, and right of the region including the video edge portion, an object that enters the screen in time and an object that exits from the screen in time are detected, An image processing apparatus that performs interpolation processing using the global vector for each of the top, bottom, left, and right of the screen in accordance with the detection result.   The video processing apparatus according to claim 9, wherein the global vector calculation unit calculates a dominant motion on the entire screen as the global vector.   The video processing apparatus according to claim 9, wherein the global vector calculation unit calculates a dominant motion at an end portion of the screen as the global vector.   10. The video processing device according to claim 9, wherein the global vector calculation unit calculates, as the global vector, a mixture of a dominant motion of the entire screen and a dominant motion at an end portion of the screen. A video processing apparatus characterized by the above.   10. The video processing device according to claim 9, wherein the global vector calculation unit independently detects a dominant motion in each of an end portion and a screen center portion of the screen as a global vector, and the global vector of the screen end portion is detected. A video processing apparatus characterized in that the reliability of a video is determined from a global vector at the center of the screen.   10. The video processing apparatus according to claim 9, wherein when the pan detection unit determines that the video is pan video, and the amount of movement in the pan direction is equal to or greater than a predetermined threshold, the global vector is used as the area of the screen edge portion. The video processing apparatus is characterized in that the interpolation processing of the region at the edge of the screen is performed using the motion vector when the value is equal to or less than the threshold.   A video display device comprising the video processing device according to claim 9.

Images