JP2009147767A - Image converting apparatus, and image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image converting apparatus which achieves image conversion of high image quality at low costs. <P>SOLUTION: The image converting apparatus includes: a frame rate converting part which converts an image of a first frame rate to a different image of a second frame rate using a first group of motion vectors calculated by performing motion search on an input image; a motion vector converting part which converts the first group of motion vectors and generates a second group of motion vectors; and a resolution enhancement processing part which performs image quality enhancement processing using the second group of motion vectors generated in the motion vector converting part on the image for which frame rate conversion has been performed in the frame rate converting part. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image conversion process that performs frame rate conversion and resolution conversion processing on an input video.

  As a so-called frame rate conversion technique that converts the frame rate of the input video signal to a desired frame rate, a motion vector is searched from the video signal before the frame rate conversion, and the previous and next frames are searched according to the searched motion vector. A so-called motion compensation type frame rate conversion technique is known in which a new frame is generated by moving the position of the included image, and the generated new frame is inserted between the previous and next frames to increase the frame rate of the video signal. (For example, refer to Patent Document 1).

Japanese Patent Laid-Open No. 11-112939

  In the frame rate conversion technique described in Patent Document 1, the motion vector obtained in the frame rate conversion process corresponds to the input video, and does not correspond to the converted output video. Therefore, when performing both the frame rate conversion process described in Patent Document 1 and the high resolution conversion process with motion search, it is necessary to perform image motion search in the subsequent super-resolution conversion processing step.

  For this reason, since motion search must be performed twice, the processing becomes complicated, and the circuit scale and cost of the processing circuit increase.

  In order to solve the above-described problems, an embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, high-quality image conversion can be realized at low cost.

  The image conversion apparatus of the present invention includes a frame rate conversion unit that converts a frame rate of an input video to a desired frame rate, and a motion that performs rate conversion of a motion vector so that the input motion vector corresponds to the frame rate of the output video. It consists of a vector rate conversion unit and a high image quality improvement unit for improving the image resolution by inputting two frames before and after.

  FIG. 1 is a diagram showing a configuration of an image conversion apparatus according to the present invention.

  The image conversion apparatus 100 includes a frame rate conversion unit 101, a motion vector rate conversion unit 102, and an image quality improvement unit 103. The frame rate conversion unit 101 performs frame interpolation on the input video according to the interpolation mode signal 106 designated from an external control unit or the like, so-called frame rate conversion, and converts the input video to a frame rate different from that of the input video. Output video. The interpolation mode signal 106 is a control signal that designates an operation mode for frame rate conversion. The operation modes include, for example, 24 Hz to 60 Hz, 50 Hz to 60 Hz, 60 Hz to 120 Hz, and the like. The motion vector rate conversion unit 102 receives the motion vector output from the frame rate conversion unit 101, and performs rate conversion of the motion vector based on the input frame rate and the output frame rate specified by the interpolation mode signal. The output timing of the motion vector information for each frame output from the motion vector rate conversion unit 102 is synchronized with the timing of the frame rate output from the frame rate conversion unit 101. The image quality improving unit 103 performs high resolution processing such as super-resolution conversion processing for improving the feeling of image resolution by inputting two frames before and after.

  Here, it is also possible to set the input side and the output side to the same frame rate, such as 60 Hz to 60 Hz, as the interpolation mode signal designation condition. In this case, the frame rate conversion unit 101 outputs the input frame as it is without performing the frame interpolation operation, but executes the motion search between the frames and outputs the resulting motion vector.

  FIG. 2 is a diagram illustrating an example of video frame rate and motion vector rate conversion conditions.

  Frame rate is one of the video standards that indicates how many pictures are sent per second. It is known that the television broadcast frame rate is 30 fps for NTSC Japan and the United States, and 25 fps for PAL Europe and the SECOM Soviet Union. Also, movies are generally 24Hz. Among television receivers that are currently on the market, there are models with a frame rate conversion function. This function can be used to simply eliminate the difference in frame rate between the original video and the TV, but it also has the purpose of realizing smooth subject movement by interpolating interpolated frames from the viewpoint of improving video quality. .

  In the description of the embodiment of the image conversion apparatus 100 of the present invention, description will be made using operating conditions for converting a 24 fps video into a 30 fps video and outputting it. However, for convenience of describing the output side video using field video after PI conversion, in the following description, the video frequency (number of vertical sync signals per second) is expressed in Hz and 24 Hz (24 frames / second) Video is expressed as conversion to 60Hz (60 fields / second) video.

  The example in the figure shows a state in which the apparatus outputs a signal instructing frame rate conversion from 24 Hz to 60 Hz to the frame rate conversion unit 101 and the motion vector rate conversion unit 102 as an interpolation mode signal.

  The frame rate conversion unit 101 inputs the 24H input image 104, converts it into a 60Hz video, and outputs a 24Hz motion vector searched from the 24Hz input video 104 along with the output. The motion vector rate conversion unit 102 converts the 24 Hz motion vector searched by the frame rate conversion unit 101 into a 60 Hz motion vector for 60 Hz video. The high image quality unit 103 receives the rate-converted 60 Hz video and the rate-converted 60 Hz motion vector corresponding to the image, performs image conversion based on a predetermined image processing algorithm, and sends it as a 60 Hz output video 105 .

  FIG. 3 is a diagram illustrating a configuration of the frame rate conversion unit.

  In the frame rate conversion process, a motion vector is searched for in units of pixels between two temporally adjacent frames of the input video, and the internal motion is calculated from the relationship between the motion vector of the operation target pixel and the position of the interpolation frame on the time axis. The procedure for determining the pixel position on the insertion frame is repeated to generate an interpolation frame. The number of interpolation frames inserted between two frames of the input video is uniquely determined from the relationship between the frame rate of the input video and the output video. For example, when the frame rates of the input video and the output video are 24 Hz and 60 Hz, respectively, the number of interpolated frames between adjacent frames of the original input video is two.

  The frames of the input video 300 are sequentially stored in the frame memory 1 (301), and the frame delayed by one frame period by the frame delay unit 304 is stored in the frame memory 2 (302). The motion search unit 306 searches for a motion vector in units of pixels between two frames of the frame memory 1 (301) and the frame memory 2 (302). The motion vector conversion unit 307 determines the pixel position on the interpolation frame from the relationship between the motion vector of the target pixel and the position on the time axis of the interpolation frame. The image interpolation unit 305 performs pixel interpolation with reference to the two frames and the motion vector, creates one interpolation frame, and stores it in the frame memory 3 (303). The switching unit 308 selects either the frame memory 2 (302) in which the original video is stored or the frame memory 3 (303) in which the interpolation frame is stored, and outputs the selected storage frame as the output video. Send as 310. The control unit 309 transmits an interpolation mode signal 311 designated from the outside to the motion vector conversion unit 307 and the switching unit 308, and determines a timing corresponding to the interpolation phase with respect to the interpolation frame.

  FIG. 4 is a diagram illustrating a configuration of the image quality improving unit.

  The image quality improving unit 103 sequentially stores the frames of the input video 400 in the frame memory 11 (401), and the frame delayed by one frame period by the frame delay unit 404 is stored in the frame memory 12 (402). To do. The image conversion unit 405 performs a predetermined image conversion process with reference to the two frames and the motion vector 406, and stores it in the frame memory 13 (403). According to the frame rate specified by the interpolation mode signal 106, the video in the frame memory 13 (403) is transmitted as an output video 406 at a predetermined timing.

  An input video 407 is a video transmitted by the frame rate conversion unit 101 in the previous stage. The motion vector 406 is a motion vector output from the motion vector rate conversion unit 102 in the previous stage.

  An example of processing of the image conversion unit 405 is super-resolution processing, for example. Super-resolution processing is a process of aligning frames using multiple frames of video and motion vectors to obtain virtual sample points and their virtual pixel data with decimal point precision from the original video. This is a method of generating a resolution video higher than the original video by obtaining video data to be actually displayed from virtual pixel data.

  FIG. 5 is a diagram showing the relationship of motion search between the frame rate conversion unit and the image quality improvement unit.

  As described with reference to FIG. 4, since the image conversion of the image quality improving unit 103 uses a motion vector between frames, it is usually necessary to separately install a motion search unit. In general, it is known that motion search has a very large processing load and the circuit scale increases in proportion to the search range.

  On the other hand, the configuration of this apparatus includes a motion search unit having the same purpose of detecting a motion vector for each pixel in the preceding frame rate conversion unit 101, although the timing of an input frame is different. Therefore, a motion vector rate conversion unit 102 is provided, and the motion vector of the input video (24 Hz) obtained by the frame rate conversion unit 101 is converted into a motion vector of the output video (60 Hz) by a predetermined procedure. The motion search part of the subsequent high image quality improvement part is omitted. As a result, the processing load can be reduced and the circuit scale can be greatly reduced.

  The configuration of the motion vector rate conversion unit 102 will be described below.

  FIG. 6 is a diagram showing an outline of the motion vector rate conversion operation.

  The motion vector refers to two temporally adjacent frames, and a motion vector corresponding to the motion of each pixel is obtained by a procedure of comparing pixel values within a predetermined range. In order to refer to two adjacent frames at the same time, 2 frames of 24Hz video currently read out from the 24Hz input video and 24Hz video read out the previous time (ie, 24Hz video delayed by one frame) are retained. A 24 Hz motion vector group 603 (hereinafter, a plurality of motion vectors are referred to as “motion vector group”) is detected using a piece of 24 Hz video.

  In the figure, an operation period Tw of motion vector rate conversion will be described as an example.

  In the rate conversion from 24 Hz to 60 Hz, it is calculated to obtain 5 frames of output for 2 frames of input. The relationship of the number of motion vectors is the same as that of frames, and a motion vector group for five frames is obtained for the detected motion vector group for two frames. The period shown in the figure is equal to the length of the frame display period corresponding to each motion vector.

  A 24 Hz motion vector MV1 (613) is obtained at the timing of the frame F1 (606) of the 24 Hz video 601, and similarly, MV2 (614) is obtained at the timing of the frame F2 (607). MV1 (613) and MV2 (614) of the 24 Hz motion vector group 603 are converted into MVc1 (615) to MVc5 (619) of the 60 Hz motion vector group 604 by a predetermined conversion procedure of the motion vector rate conversion unit 102 described later. In the period Tw, the frame rate conversion unit 101 converts the frame F1 (606) and the frame F2 (607) of the 24 Hz video 301 from the frame Fc1 (608) of the 60 Hz video 602 to the frame Fc5 (612) according to a predetermined conversion procedure. Output.

  FIG. 7 is a diagram illustrating an example of a motion vector conversion coefficient table.

  In the motion vector rate conversion from 24 Hz to 60 Hz, the vector conversion coefficient table 700 is referred to. The conversion procedure consists of multiplying each detected motion vector of the detected 24-Hz motion vector group 701 for one frame or two consecutive frames by the first coefficient 702 or the second coefficient 703, and further calculating the product sum of the multiplications to obtain a 60 Hz motion. A vector group 704 is assumed. In order to obtain a 60 Hz motion vector group 704 for five frames, combinations of parameters at each step shown in NO.1 (705) to NO.5 (709) are used. For example, in step NO.1 (905), a result obtained by multiplying the 24 Hz motion vector MV1 by the first coefficient KA1 is obtained as a 60H motion vector MVc1. In step NO.2 (706), the result of multiplying the 24 Hz motion vector MV1 and the first coefficient KA2 is obtained as a 60H motion vector MVc2. The step of NO. 3 (707) multiplies the 24 Hz motion vector MV1 and the first coefficient KA3, and further multiplies MV2 and KB2 to obtain the product sum as a 60H motion vector MVc3. NO.4 (708) and NO.5 (709) are similar to the procedure of NO.1 (705).

  The first coefficient and the second coefficient used here represent the difference on the time axis represented by the motion vector constituting the 24 Hz motion vector group and the motion vector constituting the 60 Hz motion vector group as a linear weight component. It is a numerical value. In the vector conversion coefficient table 700, the number of 60 Hz motion vectors (that is, the number of steps), the number of referenced 50 Hz motion vectors, the first coefficient, and the second coefficient can be arbitrarily changed according to the change of the rate conversion condition. it can.

  An example of the first coefficient and the second coefficient is shown below.

Consider the case of converting from a first motion vector group to a second motion vector group. At this time, if the frame display period corresponding to one vector in the second motion vector group is included in the frame display period corresponding to one motion vector of the first motion vector group to be converted, The motion vector of the second motion vector group is calculated by using Equation 1 below. Here, MVn is one vector of the first motion vector group, MVcm is one vector of the second motion vector group, and KAm is the first coefficient.
(Equation 1) MVcm = KAm × MVn
That is, when Equation 1 is used, one motion vector of the second motion vector group is calculated by multiplying one motion vector of the first motion vector group by a coefficient.

  Next, a frame display period corresponding to one vector in the second motion vector group corresponds to a frame display period corresponding to one motion vector in the first motion vector group to be converted and another motion vector. When overlapping over the frame display period, the motion vector of the second motion vector group is calculated by using Equation 2 below.

That is, in the frame display period corresponding to one vector in the second motion vector group, at least a portion overlapping with the frame display period corresponding to one motion vector in the first motion vector group serving as the conversion source, When a frame display period corresponding to another motion vector different from the motion vector in the first motion vector group and an overlapping part are included, the motion of the second motion vector group is obtained by using the following Equation 2. Calculate the vector. Here, MVn1 and MVn2 are first motion vectors of two adjacent frames, MVcm is a second motion vector, KAm is a first coefficient, and KBm is a first coefficient.
(Equation 2) MVcm = KAm x MVn1 + KBm x MVn2
That is, when Expression 2 is used, one motion vector of the second motion vector group is calculated by multiplying the two motion vectors of the first motion vector group by a coefficient.

  As described above, by calculating the motion vector at the transition part of the motion vector in the first motion vector group among the motion vectors of the second motion vector group using Formula 2 instead of Formula 1, output is performed. There is an effect that the motion of the frame can be converted into a motion vector that can be expressed more smoothly.

  Here, an example of the second motion vector group calculation method in the case of the frame rate conversion from 24 Hz to 60 Hz shown in FIG. 6 will be described.

  First, in consideration of the overlapping relationship of the frame display periods corresponding to the respective motion vectors of the first motion vector group and the second motion vector group described above, which of Equation 1 and Equation 2 is used? decide.

  Of the second motion vector group in FIG. 6, the display period of MVc1 and MVc2 is included in the display period of MV1 of the first motion vector group, and the display period of MVc4 and MVc5 is the second motion vector group. Included in the MV2 display period. Therefore, Formula 1 is used to calculate MVc1, MVc2, MVc4, and MVc5.

  In the second motion vector group of FIG. 6, the display period of MVc3 overlaps the display period of MV1 and the display period of MV2 of the first motion vector group. That is, in the display period of MVc3, there are at least a part overlapping with the display period of MV1 and a part overlapping with the display period of MV2. Therefore, Formula 2 is used to calculate MVc3.

Therefore, equations used to calculate the motion vectors MVc1, MVc2, MVc3, MVc4, and MVc5 of the second motion vector group are as shown in Equations 3 to 7 below. Here, MV1 and MV2 are first motion vectors, MVc1 to MVc5 are second motion vectors, KA1 to KA5 are first coefficients, and KB3 is a second coefficient.
(Equation 3) MVc1 = KA1 × MV1
(Equation 4) MVc2 = KA2 × MV1
(Equation 5) MVc3 = KA3 x MV1 + KB3 x MV2
(Equation 6) MVc4 = KA4 × MV2
(Equation 7) MVc5 = KA5 × MV2
As an example of the first coefficients KA1, KA2, KA4, and KA5, for example, the following formula 9 may be used for the following formula 8, the first coefficient KA3, and the second coefficient KB3. Here, f1 is the first frame rate and f2 is the second frame rate. TA is the length of the frame display period corresponding to MVc3 and overlaps with the frame display period corresponding to MV1. TB is overlapped with the frame display period corresponding to MV2 among the frame display periods corresponding to MVc3. Indicates the length of the period.

なお、f1＝24Hz、f2＝60Hzとした例では、第1係数KA1、KA2、KA4、KA5、第1係数KA3および第2係数KB3は、それぞれ次の数式１１、数式１２に示したとおりとなる。

In the example in which f1 = 24 Hz and f2 = 60 Hz, the first coefficients KA1, KA2, KA4, KA5, the first coefficient KA3, and the second coefficient KB3 are as shown in the following expressions 11 and 12, respectively. .

ここで、数式１２においてKA3とKB3とが同じ値になっている理由は、f1＝24Hz、f2＝60Hzとした例では、図６に示されるようにMVc3に対応するフレームの表示期間とMV1に対応するフレーム表示期間が重複する期間の長さTAと、MVc3に対応するフレームの表示期間とMV2に対応するフレーム表示期間が重複する期間の長さTBとが等しくなるためである。一般的には、数式９、数式１０にしたがって、それぞれKA3と、KB3は、算出する動きベクトルに対応するフレームの表示期間と、元となる複数の動きベクトルのそれぞれに対応するフレームの表示期間の重複期間の比率に応じて変えることが望ましい。

Here, the reason why KA3 and KB3 have the same value in Equation 12 is that, in the example in which f1 = 24 Hz and f2 = 60 Hz, as shown in FIG. This is because the length TA of the period in which the corresponding frame display periods overlap is equal to the length TB of the period in which the frame display period corresponding to MV2 overlaps with the display period of the frame corresponding to MVc3. In general, according to Equations 9 and 10, KA3 and KB3 are respectively the frame display period corresponding to the calculated motion vector and the frame display period corresponding to each of the plurality of original motion vectors. It is desirable to change according to the ratio of the overlap period.

  Further, in the above Equation 5, the reason why MVc3 refers to the two first vectors is as follows. In order to maintain the natural motion of the video in the relationship between the input frame and the output frame after the frame rate conversion, it is necessary to maintain a high image correlation between frames with overlapping display periods. For this purpose, it is effective to perform frame conversion with reference to all input frames in which the output frame display periods overlap. Similarly, in the motion vector conversion, the correlation between the motion vector group of the input frame and the motion vector group of the output frame is obtained by referring to the motion vectors of all the input frames having the relationship in which the output frame display periods overlap. Can be kept high. For this reason, two first vectors are referred to for the conversion of MVc3.

  In addition, MVc3 can be converted by referring to the two first vectors to reduce the variation in the time axis of the motion vector, which can be converted into a motion vector that expresses the motion of the output frame more smoothly. is there.

  By using the motion vector calculation processing of the second motion vector group described above, it is possible to express smoother motion in the output frame after frame rate conversion.

  FIG. 8 is a diagram showing the input / output relationship of a motion vector group of 24 Hz and 60H.

  The motion vector rate conversion method of the present apparatus uses the fact that the correlation between motion frames obtained from natural images is very high. By applying this principle, the 60 Hz motion vector group 802 is converted by performing linear interpolation processing using the 24 Hz motion vector group 801 for one frame or two frames that are temporally close to each other.

  For example, the 60 Hz motion vector MVc1 (806) is determined to have a correlation only with the 24 Hz motion vector MV1 (804) on the time axis, and is converted with reference to only MV1 (804). MVc2 (807) is also converted by referring to only MV1 (804) for the same reason. The 60 Hz motion vector MVc3 (808) is determined to have a correlation with the 24 Hz motion vectors MV1 (804) and MV2 (805), and is converted with reference to these two motion vectors. The 60 Hz motion vectors MVc4 (809) and MVc5 (810) are determined to be correlated only with the 24 Hz motion vector MV2 (805), and are converted with reference to only MV2 (805).

  A period Tw (803) for two frames of 24 Hz video corresponds to a period of five frames of 60 Hz video. Therefore, it is necessary to output the motion vector conversion process corresponding to 2 frames from 24 Hz motion vectors to 5 frames of 60 Hz motion vectors.

  FIG. 9 is a diagram showing a calculation concept of the 60 Hz motion vector conversion process.

  FIG. 9 (a) is a calculation procedure using the parameters of the NO.1 (705), NO.2 (706), NO.4 (708) and NO.5 (709) steps of FIG. The 60 Hz motion vector MVca (803) is obtained by multiplying the 24 Hz motion vector MV1x (901) and the first coefficient KAx (802).

  FIG. 9B is a calculation procedure using the parameters of step NO. 3 (707) in FIG. The 60 Hz motion vector MVcb (908) is multiplied by the 24 Hz motion vector MV2x (904) and the first coefficient KAx (906), multiplied by the 24 Hz motion vector MV3x (905) and the second coefficient KBx (907), and the sum of the two. Ask for.

  FIG. 10 is a diagram illustrating a relationship between a frame and a motion vector on the time axis (on the reproduction time axis).

  In the figure, in the period Tw of the motion vector rate conversion operation, a 60 Hz image of 5 frames is obtained from a 24 Hz image of 2 frames, and a 60 Hz motion vector group of 5 frames is obtained from a 24 Hz motion vector group of 2 frames. This will be described using an example to be obtained.

  As described with reference to FIG. 6, each motion vector of the 24 Hz motion vector group is generated with reference to two adjacent 24 Hz video frames. For example, MV1 (1002) is generated with reference to frame F1 (1000) and frame F2 (1001). In MV2 (1003), the generation procedure is the same with reference to the adjacent frame F2 (1001) and frame F3.

  Since the reference relationship for obtaining the 60 Hz motion vector group for 5 frames from the 24 Hz motion vector group for 2 frames has been described with reference to FIG. 8, it is omitted here.

  The procedure for obtaining 5 frames of 60 Hz images from 2 frames of 24 Hz images is as follows. First, the frame F1 (1000) that is the starting point of the period Tw is output as it is as a frame Fc1 (1004) of a 60 Hz video. Frames Fc2 (1005) to Fc5 (1008) of 60 Hz video do not match any display time axis of 24 Hz video, so an interpolation frame by a predetermined algorithm is used from two 24 Hz videos that are close to each other on the time axis. Generate and output. For example, a frame Fc2 (1005) of a 60 Hz video is output as an interpolated frame with reference to two frames F1 (1000) and F2 (1001) that are close to each other on the time axis. In the frames Fc3 (1006) to Fc5 (1008), the procedure for referring to the two frames of the 24-Hz video that are close to each other on the time axis is the same. In the motion vector rate conversion, a predetermined calculation is performed using the vector conversion coefficient 1009. The method has been described with reference to FIGS.

  The image quality improvement unit 103 of the image conversion apparatus 100 of the present invention performs image conversion using the 60 Hz video after the frame rate conversion and the 60 Hz motion vector group, and the correspondence between the 60 Hz video and the 60 Hz motion vector group referred to at that time is performed. The relationship is as shown in the figure. For example, the frame Fc1 (1004) performs image conversion using MVc1 (1009). Similarly, the frame Fc2 (1005) is MVc2 (1010), the frame Fc3 (1006) is MVc3 (1011), the frame Fc4 (1007) is MVc4 (1012), the frame Fc5 (1008) is MVc5 (1013) ) To convert the image.

  FIG. 11 is a diagram illustrating an example of processing timing of the motion vector rate conversion unit.

  The reference relationship between the 24 Hz motion vector group and the 60 Hz motion vector group is as described with reference to FIG. The 60 Hz motion vector group 1102 is generated at the timing when the motion search unit 306 of the frame rate conversion unit 101 detects and outputs the 24 Hz motion vector group 1100.

  In the procedure of motion vector rate conversion, it is necessary to refer to a 24 Hz motion vector input in the past for timing. For this reason, the motion vector is stored in the temporary storage memory simultaneously with the new reading of the 24 Hz motion vector.

  An operation period Tw of motion vector rate conversion will be described as an example. In motion vector rate conversion from 24 Hz to 60 Hz, the maximum time length of vector conversion processing per frame is Tw / 5.

  In the first cycle of the period Tw, the 24 Hz motion vector MV1 (1103) is input and then converted into the 60 Hz motion vector MVc1 (1107). At the same time, MV1 (1103) is stored in the memory 1105. In the second cycle, the motion vector is read from the memory 1105 and converted into MVc2 (1108). In the third cycle, the motion vector is read from the input of the MV2 (1104) and the memory 1105 and converted to MVc3 (1109). At the same time, MV2 (1104) is stored in the memory 1106. In the fourth cycle, the motion vector is read from the memory 1106 and converted into MVc4 (1110), and in the fifth cycle, the motion vector is read from the memory 1106 and converted into MVc5 (1111).

  FIG. 12 is a diagram illustrating an example of another configuration of the motion vector rate conversion unit.

  In motion vector rate conversion, a motion vector stored in the temporary storage memory 1204 in order to omit the conversion processing in a conversion cycle in which a calculation under the same conditions as the previous cycle is performed with reference to a previously input 24 Hz motion vector. May be stored after conversion into a 60 Hz motion vector.

  For example, when a 60 Hz motion vector MVc1 (1205) is converted from a 24 Hz motion vector MV1 (1203), this MVc1 (1205) is stored in the temporary storage memory 1204 and read from the memory 1204 in the next conversion cycle. The motion vector is used as it is as MVc2 (1206).

  FIG. 13 is a diagram showing an operation flow of the image conversion apparatus of the present invention.

  The conversion period FN is set from the relationship between the input frame rate of video input to the apparatus and the output frame rate of video output from the apparatus. For example, the conversion cycle of images and motion vectors from 24 Hz to 60 Hz is set to 5 (1300). A period counter FCNT for identifying a position on the time axis within the conversion period is set to 1 (1301).

  Thereafter, the process branches according to the value of FCNT.

  Here, when FCNT is 1 or 4 (1302), a 24 Hz motion vector MVn is detected from 24 Hz video images of two adjacent frames (1303). Next, the first coefficient KAn is read from the motion vector conversion coefficient table (1304), the 60H motion vector MVcx is output (1305) by motion vector conversion, and then the 24 Hz motion vector MVn is stored in the temporary storage memory as MVn-1. Then, the process proceeds to process 1315.

  When FCNT is 2 or 5 (1307), the 24-Hz motion vector MVn-1 used in the past conversion cycle is read from the temporary storage memory (1308). Next, the first coefficient KAn is read from the motion vector conversion coefficient table (1321), the 60H motion vector MVcx is output (1322) by motion vector conversion, and the process proceeds to processing 1315.

  When FCNT is 3 (1309), a 24 Hz motion vector MVn is detected from 24 Hz video images of two adjacent frames (1310). Next, the 24-Hz motion vector MVn-1 used in the past conversion cycle is read from the temporary storage memory (1311). Further, the first coefficient KAn is read from the motion vector conversion coefficient table (1312) and the second coefficient KBn is read (1313), and the 60H motion vector MVcx is output (1314) by motion vector conversion, and then the 24 Hz motion vector MVn is output. The MVn-1 is stored in the temporary storage memory (1323), and the process proceeds to processing 1315.

  In the process 1315, in all states of the cycle counter FCNT, the 60H motion vector MVcx is used to perform the image quality improvement processing of the 60 Hz video and output as a 60 Hz video after image conversion (1316). Subsequently, a process end determination is performed (1317), and if it is continued (not terminated), 1 is added to the period counter FCNT (1318), and it is determined whether FCNT is larger than the conversion period FN (1319). . If FCNT is larger than the conversion period FN, the period counter FCNT is reset to 1 (1320), and the process returns to step 1302. If the process end determination in process 1317 is the process end, the operation of this apparatus is ended.

  FIG. 14 is a diagram showing an example of the image conversion apparatus of the present invention.

  The image processing method of the present embodiment can be applied to, for example, a digital television that has a function of displaying on a display after performing frame rate conversion and super-resolution conversion on an input video.

  In the figure, description will be made as a configuration of a plasma television. The plasma TV 1400 includes a video input / output unit 1401 that controls operations such as receiving digital TV broadcasts, downloading digital video content existing on a network, or outputting video content recorded by a plasma TV, and recorded video content. A video content storage unit 1405 for storing video, a recording / playback unit 1404 for controlling recording and playback operations, a user interface unit 1402 for receiving user operations, and a recording / playback unit 1404 according to the received operating conditions. On the other hand, the controller 1403 controls the recording operation and the reproduction operation, and simultaneously transmits an interpolation mode signal to the frame rate conversion unit 1406 at the time of reproduction, and the frame rate conversion unit 1406 performs the frame frequency rate conversion according to the interpolation mode signal. And rate conversion of motion vectors according to the interpolation mode signal. Motion vector rate conversion unit 1407, super-resolution conversion unit 1408 that improves the image quality by increasing the resolution of the frame, audio signal processing unit 1410 that converts the reproduced audio in a predetermined procedure, A plasma panel 1409 for display and a speaker 1411 for outputting sound are included.

  The image conversion processing of the present invention may be implemented in the form of being implemented as hardware, or may be implemented in the form of software that uses a separate video signal processing controller and is linked to other hardware functions. good. Further, in addition to the plasma type digital television, the present invention may be applied to a liquid crystal type digital television having a function of improving the output image quality.

  Although the embodiment of the image conversion apparatus 100 of the present invention has been described using the operating conditions for converting a 24 Hz video to a 60 Hz video and outputting it, the operation of the apparatus of the present invention is limited to this condition only. It is not a thing. There are a plurality of standards regarding the frame rate of video, and any combination of the frame rates can be implemented by the operation means of the present invention.

  Examples of other operating conditions regarding the combination of video frame rates include 24 Hz to 60 Hz, 30 Hz to 60 Hz, 50 Hz to 100 Hz, 60 Hz to 120 Hz, and the like. In addition, 23.98 Hz, 29.97 Hz, 59.94 Hz, 119.88 Hz, and the like defined as the video frequency standard can be applied as the input or output frequency of the video frame rate.

The figure which shows the structure of the image conversion apparatus of this invention. The figure which shows an example of the rate conversion conditions of an image | video and a motion vector. The figure which shows the structure of a frame rate conversion part. The figure which shows the structure of the image quality improvement part. The figure which shows the relationship of the motion search with a frame rate conversion part and an image quality improvement part. The figure which shows the outline | summary of motion vector rate conversion operation | movement. The figure which shows an example of a motion vector conversion coefficient table. The figure which shows the input / output relationship of a motion vector. The figure which shows the concept of the calculation of a motion vector conversion process. The figure which shows the relationship on the time-axis of a flame | frame and a motion vector. The figure which shows an example of the process timing of a motion vector rate conversion part. The figure which shows an example of another structure of a motion vector rate conversion part. The figure which shows the operation | movement flow of the image conversion apparatus of this invention. The figure which shows an example of the image conversion apparatus of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Image converter, 101 ... Image input part, 102 ... Image storage part, 103 ... Image rate conversion part, 104 ... Image correction part, 105 ... Image output part, 106 ... Motion vector detection unit, 107 ... Motion vector storage unit, 108 ... Motion vector rate conversion unit, 109 ... Input / output frame rate control unit, 110 ... Video 1, 111 ... Video 2, 112 ... Output frame rate value, 113 ... Input frame rate detector

Claims (10)

Using a first motion vector group calculated by performing a motion search on an input video, a frame rate conversion unit that converts a video with a first frame rate into a video with a different second frame rate;
A motion vector converter that converts the first motion vector group to generate a second motion vector group;
A high-resolution processing unit that performs high-quality image processing on the video that has undergone frame rate conversion in the frame rate conversion unit, using the second motion vector group generated in the motion vector conversion unit. A featured image conversion apparatus.   The second motion vector group generated by the motion vector conversion unit includes a vector generated by multiplying one motion vector included in the first motion vector group by a coefficient, and the first motion vector group. The image conversion apparatus according to claim 1, further comprising: a vector generated by multiplying two included motion vectors by two coefficients.   The coefficient is such that the display period of the frame corresponding to the motion vector included in the second motion vector group overlaps the display period of the frame corresponding to the motion vector included in the first motion vector group on the playback time axis. The image conversion apparatus according to claim 2, wherein a different value is set according to a period of time.   The frame rate conversion unit includes two frames whose display periods corresponding to one motion vector of the second motion vector group correspond to each of two motion vectors included in the first motion vector group. When the two motion vectors overlap each other during the display period, one motion vector of the second motion vector group is calculated by multiplying and adding each of the two motion vectors by a coefficient. Item 2. The image conversion apparatus according to Item 1.   The frame rate conversion unit includes a plurality of motion vectors corresponding to each of a plurality of motion vectors included in at least the first motion vector group in a display period of a frame corresponding to one motion vector of the second motion vector group. When an overlapping portion with the display period of the frame is included, one motion vector of the second motion vector group is calculated by multiplying and adding each of the plurality of motion vectors by a coefficient. The image conversion apparatus according to claim 1. An input unit for inputting a video signal;
Using the first motion vector group calculated by performing a motion search on the video input to the input unit, the video of the first frame rate is converted into a video of a different second frame rate, and the first An image processing unit that converts the motion vector group to generate a second motion vector group, and performs an image quality improvement process on the video at the second frame rate using the second motion vector group; ,
An image display device comprising: a display unit that displays a video image that has been subjected to image quality enhancement processing by the image processing unit.   The second motion vector group generated by the image processing unit includes a vector generated by multiplying one motion vector included in the first motion vector group by a coefficient, and the first motion vector group. The image display apparatus according to claim 6, further comprising: a vector generated by multiplying the two motion vectors by two coefficients.   The coefficient is such that the display period of the frame corresponding to the motion vector included in the second motion vector group overlaps the display period of the frame corresponding to the motion vector included in the first motion vector group on the playback time axis. The image display device according to claim 7, wherein the image display device has different values depending on a period of time.   In the generation of the second motion vector group, the image processing unit includes two frames in which a display period of a frame corresponding to one motion vector of the second motion vector group is included in the first motion vector group. When overlapping over the display period of two frames corresponding to each of the motion vectors, one of the two motion vectors is added by multiplying each of the two motion vectors by a coefficient. The image display apparatus according to claim 6, wherein a vector is calculated.   In the generation of the second motion vector group, the image processing unit includes a plurality of frames included in at least the first motion vector group in a frame display period corresponding to one motion vector of the second motion vector group. In the case where an overlapping portion with a display period of a plurality of frames corresponding to each of the motion vectors is included, each of the plurality of motion vectors is multiplied by a coefficient to be added, thereby adding the second motion vector group. The image display apparatus according to claim 6, wherein one motion vector is calculated.

Images