JP2010088066A - Frame rate conversion method, frame rate conversion apparatus, frame rate conversion program, and computer readable recording medium with the program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a novel frame rate conversion technology for suppressing a code amount low when encoding the video signal of a desired frame rate obtained by selecting a frame of a high frame rate video signal through down sampling. <P>SOLUTION: When a leading frame after down sampling is selected and processing is started, each time a frame after down sampling is selected, a next frame position specified by down sampling at equal intervals is specified and a frame positioned in the vicinity is defined as a processing target. For each processing target frame, a value is calculated which indicates encoding efficiency of a low frame rate video signal to be generated when performing down sampling on a processing target frame. On the basis of the calculated value, processing for selecting a frame indicative of optimal encoding efficiency from among processing target frames is repeated, thereby executing frame rate conversion from the high frame rate video signal to the low frame rate video signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a frame rate conversion method and apparatus for converting a frame of a high frame rate video signal into a low frame rate video signal by selecting by downsampling, and a frame rate conversion program used for realizing the frame rate conversion method. And a computer-readable recording medium on which the program is recorded.

  In recent years, there are growing expectations for super-high-quality images such as large-screen sports images and digital cinema that are full of realism. In response to this, research on high-quality video has been vigorously conducted.

  The following four elements are necessary to realize super high-quality video. That is, spatial resolution, pixel value depth, color reproducibility, and temporal resolution. In response, the former three elements are being studied in applications such as digital cinema and natural vision projects. In addition, improvement of time resolution, that is, an increase in video frame rate, which is indispensable for expressing a natural movement of a subject, has been studied.

  High-speed cameras capable of capturing high frame rate images exceeding 1000 [frame / sec] are already on the market. However, video captured by such a high-speed camera is used for slow playback applications. The upper limit of the frame rate of the video input / output system is asymmetric. Since the upper limit of the current display is about 120 [fps], it is necessary to thin out the frame rate of a video source shot with a high-speed camera in a display format intended for real-time playback.

  Normally, as shown in FIG. 15, downsampling is performed so that the frame time intervals after downsampling are equal (see, for example, Patent Document 1). This is based on the assumption that jerkiness (image quality degradation) occurs in non-uniform time sampling.

  This assumption is correct when the downsampling target is a low frame rate video. However, this is not the case when the object of downsampling is a high frame rate video. When the target of downsampling is a high frame rate video, even if the frames are not arranged at regular intervals, if the deviation from the regular intervals is within a certain threshold, visually large jerkiness (image quality degradation) will occur. Does not occur.

Therefore, when the target of downsampling is a high frame rate video, it is possible to relax the constraint condition of equal length with respect to the frame interval after downsampling. By relaxing this constraint, the degree of freedom regarding frame selection after downsampling is increased. That is, there is room for studying an optimal frame rate downsampling method from the viewpoint of coding efficiency.
JP 2004-201165 A

  As described above, when the target of downsampling is a high frame rate video, it is possible to relax the constraint condition of equal length with respect to the frame interval after downsampling. Can be high.

  However, the prior art has not studied this point at all, and from now on, when encoding a low frame rate video obtained by downsampling a high frame rate video, the inter-frame prediction error is reduced. There was still room for improvement.

  The present invention has been made in view of such circumstances, and in video encoding processing using a low frame rate video signal obtained by downsampling a high frame rate video signal as an input, a low level after downsampling is achieved. An object of the present invention is to establish an efficient frame rate downsampling technique for determining a low frame rate video signal in consideration of the encoding efficiency of the frame rate video signal.

For high frame rate video signal to be converted frame rate, the frame interval as [delta] _t, the time _{t = jδ t (j = 0} , 1, .....) the pixel value of the position x in the frame of f (X, t) (x = 0, 1,..., X−1).

Consider a case where the number of frames is converted to 1 / M by down-sampling the pixel signal f (x, t). A frame rate ratio M before and after downsampling is called a downsampling ratio. That is, this conversion assumes that the frame rate is converted from “1 / δ _t ” to “1 / Mδ _t ”.

  In the following, a one-dimensional signal will be described as an example for the sake of simplicity, but the same discussion can be easily extended to a two-dimensional signal.

When performing the above conversion by frame thinning, conventionally,
f (x, iMδ _t ) i = 0, 1,...
Thus, a method of multiplying the frame interval by M has been used.

This is based on an empirical rule that the smoothness of motion is lost when the frame interval after downsampling is unequal. This rule of thumb is valid in the case of a conventional frame rate video signal such as δ _t = 1/30 [seconds].

However, this is not the case in the case of a video signal with a high frame rate such as δ _t = 1/1000 [second]. For example, when converting a video signal with a high frame rate with a frame interval of δ _t = 1/1000 [seconds] into a video signal with a low frame rate with a frame interval of 1 / 62.5 [seconds] as M = 16, Even if the frame interval after conversion expands or contracts by about 1/1000 [second], it cannot be visually detected.

  For this reason, image quality deterioration due to non-equal frame intervals is not a problem. Therefore, it is possible to relax the constraint condition of equal length with respect to the frame interval after downsampling. By relaxing this constraint, the degree of freedom regarding frame selection after downsampling is increased.

Therefore, in the present invention, frame selection candidates are
f (x, (iM + L _i ) δ _t ) i = 0, 1,...
L _i = −Δ,..., Δ−1
In this way, the frame is expanded to 2Δ frame.

Here, L _i is a parameter that takes an integer value in the range of −Δ,..., Δ−1 (hereinafter, L _i is referred to as an expansion / contraction parameter), and is used to specify a frame after downsampling. It is assumed that 2Δ ≦ M. Further, Δ represents an existing section of a frame candidate to be down-sampled, and is a parameter given from the outside, for example.

The reason why the expansion / contraction parameter L _i is limited to Δ is that if the value of L _i is too large, the frame interval greatly deviates from the equal length, which causes deterioration of image quality.

  As described above, in the present invention, the frame selection candidates are expanded to 2Δ frames, and as to which frame is selected, for example, the following criteria are used.

[I] Selection criteria 1
In selection criterion 1, frame selection after downsampling is performed so as to minimize the motion compensation prediction error.

The signal f (x, (iM + L _i ) δ _t ) is divided into sections B [k] (k = 0, 1,..., K−1) of size S, and each section B [k] When motion compensation (estimated displacement d [k]) is performed in units of (k = 0, 1,..., K−1), the prediction error after motion compensation in the section is expressed by the following equation ( It can be expressed as 1). Here, in Expression (1), a prediction error after motion compensation is obtained with respect to the previous frame.

According to this equation (1), first, for each L _i , the estimated displacement d [k] (k = 0, 1,..., K− so as to minimize σ _i ² [L _i ]. 1) is set. At this time, the accuracy of the estimated displacement d [k] is given from the outside (eg, integer pixel accuracy, 1/2 pixel accuracy, 1/4 pixel accuracy, etc.). Further, the smallest σ _i ² [L _i ] is selected from σ _i ² [−Δ],..., Σ _i ² [Δ−1], and the corresponding L _i is set as the expansion / contraction parameter. To select the frame indicated by the set expansion / contraction parameter.

[B] Selection criteria 2
In selection criterion 2, frame selection after down-sampling is performed so that the motion compensation prediction error is minimized when the accuracy of the estimated shift amount is integer pixel accuracy.

The signal f (x, (iM + L _i ) δ _t ) is divided into sections B [k] (k = 0, 1,..., K−1) of size S, and each section B [k] When motion compensation (estimated displacement d [k]) is performed in units of (k = 0, 1,..., K−1), the prediction error after motion compensation in that section is the above-described equation ( It can be expressed as follows in 1).

According to this equation (1), first, for each L _i , the estimated displacement d [k] (k = 0, 1,..., K− so as to minimize σ _i ² [L _i ]. 1) is set. Further, the smallest σ _i ² [L _i ] is selected from σ _i ² [−Δ],..., Σ _i ² [Δ−1], and the corresponding L _i is set as the expansion / contraction parameter. To select the frame indicated by the set expansion / contraction parameter.

[C] Selection criteria 3
In selection criterion 3, encoding is performed by a specific encoder (for example, an encoder compliant with H.264), and a Lagrangian cost (encoding cost: J) of the weighted sum of the encoding distortion D and the code amount R is encoded. = D + λR) to minimize the frame selection after down-sampling.

  Here, λ is a parameter for adjusting the weight of the coding distortion D and the code amount R, and is given from the outside.

[D] Selection criteria 4
In selection criterion 4, encoding is performed by a specific encoder (for example, an encoder conforming to H.264) with the accuracy of the estimated shift amount as integer pixel accuracy, and the weighting of the encoding distortion D and the code amount R is performed. Frame selection after down-sampling is performed so as to minimize the total Lagrangian cost (encoding cost: J = D + λR).

  Here, λ is a parameter for adjusting the weight of the coding distortion D and the code amount R, and is given from the outside.

  Next, the configuration of the present invention will be described.

[1] First Configuration The frame rate conversion apparatus according to the first configuration of the present invention realizes conversion to a low frame rate video signal by selecting a frame of a high frame rate video signal by downsampling. (1) input means for inputting a downsampling ratio; (2) first frame selection means for selecting a frame to be the first frame after downsampling from among frames of a high frame rate video signal; When a frame after downsampling is selected by the frame selection means or the frame selection means described later, it is defined by equal-length interval downsampling based on the downsampling ratio input by the input means and the frame interval of the high frame rate video signal. Specifying means for specifying the next frame position, and (4) special features of the specifying means. A value that indicates the coding efficiency of the low frame rate video signal that is generated when down-sampling is performed for each frame to be processed as a processing target frame within a predetermined range including the specified frame position. A calculation means for calculating a certain coding efficiency display value; and (5) selecting a frame indicating the optimum coding efficiency display value from the processing target frames based on the coding efficiency display value calculated by the calculation means. And frame selection means for selecting it as a frame after downsampling.

  When this configuration is adopted, the size of the predetermined range is further increased from the initial value to perform frame selection by the frame selection means, and each of the optimum encoding efficiency display values used for the frame selection is selected. When the first increase value whose value or the sum of these values indicates the desired encoding efficiency is obtained, and frame selection is performed by the frame selection means using a predetermined range having the magnitude of the obtained increase value. There may be a determination means for finally determining the selected frame as a frame after downsampling.

  The frame rate conversion method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program. This computer program can be provided by being recorded on an appropriate computer-readable recording medium. The present invention is realized by being provided via a network, installed when the present invention is implemented, and operating on a control means such as a CPU.

  In the frame rate conversion device according to the first configuration of the present invention configured as described above, when the size of the predetermined range is preset, for example, the first frame of the high frame rate video signal is downgraded. As shown in Fig. 1, each time a downsampled frame is selected, the next frame position specified by equal length downsampling is specified. Then, a predetermined number of frames located in the vicinity of the specified frame position are processed, and processing is performed on each processing target frame based on the processing target frame and a selected frame (for example, the previously selected frame). Coding showing the coding efficiency of the low frame rate video signal generated when down-sampling in the target frame A rate display value (for example, motion compensated prediction error power) is calculated, and based on the calculated encoding efficiency display value, a frame indicating the optimal encoding efficiency is selected from the processing target frames, and the frame is reduced. By repeating the selection as a frame after sampling, a frame rate conversion process from a high frame rate video signal to a low frame rate video signal is executed.

  In this way, the frame rate conversion apparatus of the present invention takes into account the coding efficiency of the low frame rate video signal after downsampling when performing the frame rate conversion from the high frame rate video signal to the low frame rate video signal. In this way, processing is performed to execute the frame rate conversion processing.

[2] Second Configuration The frame rate conversion apparatus according to the second configuration of the present invention realizes conversion to a low frame rate video signal by selecting a frame of a high frame rate video signal by downsampling. , (1) input means for inputting a downsampling ratio, and (2) start of selecting a plurality of consecutive frames as candidates for the first frame after downsampling from the frames of the high frame rate video signal as start frames. (3) Select a frame after downsampling as a starting point from the start frame selected by the start frame selection unit by a frame final selection unit described later, or select a frame after downsampling by a frame selection unit described later. When a frame is selected, the downsampler input by the input means Specifying means for specifying a next frame position defined by equal-length interval downsampling based on the frame ratio and the frame interval of the high frame rate video signal, and (4) a predetermined range including the frame position specified by the specifying means For each processing target frame, a coding efficiency display value that is a value indicating the coding efficiency of the low frame rate video signal generated when down-sampling with the processing target frame is calculated for each processing target frame. (5) Based on the encoding efficiency display value calculated by the calculating means, a frame indicating the optimal encoding efficiency display value is selected from the processing target frames, and the frame is used as a frame after downsampling. The frame selection means to be selected and (6) the frame starting from each of the start frames selected by the start frame selection means. Frame selection means, based on the sum of the optimal encoding efficiency display values used for the frame selection, a start frame showing the optimal encoding efficiency is selected from the start frames, and the selection is performed. Frame final selection means for finally selecting the frame selected when the frame selection means performs frame selection starting from the start frame as a frame after downsampling.

  When this configuration is adopted, the optimum encoding efficiency used for the final selection of the frame by further increasing the size of the predetermined range from the initial value and performing the final selection of the frame by the frame final selection means. The frame final selection means using the predetermined range having the magnitude of the obtained increase value by obtaining the first increase value in which the sum of the display values or the individual values constituting the sum indicates the desired encoding efficiency When the final selection of the frame is performed, there may be provided a determination means for finally determining the finally selected frame as a frame after downsampling.

  The frame rate conversion method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program. This computer program can be provided by being recorded on an appropriate computer-readable recording medium. The present invention is realized by being provided via a network, installed when the present invention is implemented, and operating on a control means such as a CPU.

  In the frame rate conversion apparatus of the present invention having the second configuration configured as described above, when the size of the predetermined range is preset, for example, the first frame of the high frame rate video signal and Processing is started by selecting a plurality of subsequent frames as start frames, and as shown in FIG. 2, frame selection similar to that of the frame rate conversion device of the present invention having the first configuration is started from each start frame. The frame is selected by executing the process, the sum of the optimum encoding efficiency display values used for the frame selection is calculated, and based on the calculated sum of the encoding efficiency display values, the start frame is selected. Select the start frame that shows the optimal coding efficiency, and down-samp the selected frame when selecting the frame starting from the selected start frame. By final selection as frame after ring, it executes the frame rate conversion processing to the low frame rate video signal from the high frame rate video signal.

  In this way, the frame rate conversion apparatus of the present invention takes into account the coding efficiency of the low frame rate video signal after downsampling when performing the frame rate conversion from the high frame rate video signal to the low frame rate video signal. In this way, processing is performed to execute the frame rate conversion processing.

  In the present invention, when performing frame rate conversion from a high frame rate video signal to a low frame rate video signal, the frame rate conversion process is performed in consideration of the encoding efficiency of the low frame rate video signal after downsampling. Execute.

  Thus, according to the present invention, when encoding a video signal having a desired frame rate obtained by downsampling a high frame rate video signal, the video signal obtained by thinning out the uniform frame interval is used. The amount of codes can be kept low.

  Hereinafter, the present invention will be described in detail according to embodiments.

  FIG. 3 shows an example of the device configuration of the frame rate conversion device 1 of the present invention.

  As shown in this figure, the frame rate conversion apparatus 1 of the present invention includes a high frame rate video file 10 storing a high frame rate video signal to be subjected to frame rate conversion processing, and a low frame rate subjected to frame rate conversion processing. A low frame rate video file 11 that stores a video signal, and a frame rate downsampling unit 12 that executes a frame rate conversion process from a high frame rate video signal to a low frame rate video signal are provided.

  The frame rate downsampling unit 12 includes a motion estimation unit 1200, and uses a motion compensation prediction error calculation unit 120 that calculates a motion compensation prediction error, and a motion compensation prediction error calculated by the motion compensation prediction error calculation unit 120. The downsampling execution unit 121 that executes the frame rate downsampling process by selecting a frame from the frames of the high frame rate video signal, and the work data (data such as the array) of the downsampling execution unit 121 are stored. And a work memory 122.

[1] Process of Motion Estimation Unit 1200 Before entering the description of the frame selection process executed by the downsampling execution unit 121, the process of the motion estimation unit 1200 included in the motion compensation prediction error calculation unit 120 will be described.

  The motion estimation unit 1200 receives the prediction target frame (the selection target frame becomes the prediction target frame) and the reference frame (for example, the frame selected immediately before becomes the reference frame), and performs the following processing To estimate the motion vector d [k] for each block in the prediction target frame, that is, d [k] (k = 0, 1,..., K−1) in Expression (1). Perform the process. Here, k is an index for identifying a block.

Processing: • Finds a motion vector d [k] that minimizes the term of Σ {x∈B [k]} (prediction error sum of k-th block) in Equation (1). Selection is made from values in the range −D ≦ d [k] ≦ D−1. • The selection method calculates the sum of prediction errors when using the candidate vectors for all candidate vectors in the search range. Then, the motion compensation prediction error calculation unit 120 that performs the prediction error sum by using d [k] as a vector that minimizes the prediction error sum uses the vector d [k] estimated by the motion estimation unit 1200, and Based on this, a motion compensation prediction error between the prediction target frame and the reference frame is calculated.

[2] Processing of Downsampling Execution Unit 121 Next, the frame rate downsampling processing executed by the processing of the downsampling execution unit 121 will be described.

  In the processing of the downsampling execution unit 121 described below, the case where the motion compensation prediction error corresponding to the selection criterion 1 described above is used as the selection criterion for the downsampling frame is shown. The process is the same.

  First, the “flow rate of the frame rate downsampling process (part 1)” for executing the frame rate downsampling process in the form shown in FIG. 1 will be described.

Here, when this “frame rate downsampling process flow (part 1)” is executed, the downsampling execution unit 121, as shown in FIG.
(1) an input unit 1210 for reading the number of frames J, the frame interval δ _t , the downsampling ratio M, and the expansion / contraction parameter maximum value Δ;
(2) a leading frame selection unit 1211 that selects a frame that is a leading frame after downsampling from frames of a high frame rate video signal;
(3) the top when the frame selection unit 1211 and later frame selector 1215 selects the frame after downsampling, the frame interval [delta] _t of the down-sampling ratio M input by the input unit 1210 high frame rate video signal A next frame position specifying unit 1212 for specifying a next frame position defined by downsampling at equal intervals based thereon;
(4) a processing target frame extracting unit 1213 that extracts 2Δ frames located near the frame position specified by the next frame position specifying unit 1212 as processing target frames;
(5) For each processing target frame extracted by the processing target frame extraction unit 1213, a motion compensation prediction error that is a value indicating the encoding efficiency of the low frame rate video signal generated when downsampling is performed on the processing target frame. A processing target frame prediction error calculation unit 1214 to calculate,
(6) Based on the motion compensation prediction error calculated by the processing target frame prediction error calculation unit 1214, a frame indicating the minimum motion compensation prediction error is selected from the processing target frames, and is used as a frame after downsampling. A frame selection unit 1215 to select;
(7) When the frame selection unit 1215 selects a frame, it is determined whether or not the processing has been completed for all the frame positions to be downsampled, and it is determined that the processing has not been completed. When this is done, the all frame position end determination unit 1216 that activates the next frame position specifying unit 1212 to execute processing for the next frame position.
The structure provided with.

[2-1] Flow of frame rate downsampling processing (part 1)
0. Captured image signal (high frame rate video signal subject frame rate conversion process), reads the number of frames J, the frame rate (used in order to calculate the frame interval [delta] _t) 1. 1. Read downsampling ratio M As the value of L ₀ that defines the first frame after downsampling, reads the previously given value. For example, 0 or Δ / 2 is read. for i = 1,..., J / M-1
4). for L _i = −Δ,..., Δ−1
5). A motion vector that minimizes the motion-compensated prediction error (equation (1)) when the frame after downsampling is f (x, (iM + L _i ) δ _t ) is obtained, and the motion vector is represented by ^ d [L _i ] [K] (k = 0, 1,..., K−1), and the minimized motion compensation prediction error is stored in σ _i ² [L _i ]. Here, the symbol ^ in “^ X” (where X is a letter) indicates a symbol attached to “X”. 6. Search for the minimum value in σ _i ² [L _i ] (L _i = −Δ,..., Δ−1) and identify the corresponding L _i ^* F (x, (iM + L _i ^* ) δ _t ) (x = 0,..., X−1) is stored as the i-th frame after downsampling.

  In this way, when executing the “frame rate downsampling process flow (1)”, the downsampling execution unit 121 uses, for example, the first frame of the high frame rate video signal as the first frame after downsampling. As shown in FIG. 1, every time a frame after downsampling is selected, the next frame position specified by downsampling at equal intervals is specified and the specified frame is entered. A motion compensation prediction error is calculated when downsampling is performed on the processing target frame based on the processing target frame and the previously selected frame, with a predetermined number of frames (2Δ frames) positioned in the vicinity of the position as the processing target. Based on the calculated motion compensation prediction error, A frame rate conversion process from a high frame rate video signal to a low frame rate video signal is performed by repeatedly selecting a frame indicating a motion compensated prediction error and repeatedly selecting it as a frame after downsampling. It is.

  Next, a “frame rate downsampling process flow (part 2)” for executing the frame rate downsampling process in the form shown in FIG. 2 will be described.

Here, when executing this “flow rate of the frame rate downsampling process (part 2)”, the downsampling execution unit 121, as shown in FIG.
(1) an input unit 1210 for reading the number of frames J, the frame interval δ _t , the downsampling ratio M, and the expansion / contraction parameter maximum value Δ;
(2) a start frame selection unit 1211α that selects, as start frames, a plurality of consecutive frames that are candidates for the first frame after downsampling from the frames of the high frame rate video signal;
(3) A frame final selection unit 1217 described later selects a frame after downsampling as a starting point from the start frames selected by the start frame selection unit 1211α, or a frame selection unit 1215 described later selects a frame after downsampling. when you select the particular next frame position identifying the next frame position defined by the down-sampling of the input equal length interval based on the frame interval [delta] _t of the down-sampling ratio M and the high frame rate video signal input section 1210 Part 1212;
(4) a processing target frame extracting unit 1213 that extracts 2Δ frames located near the frame position specified by the next frame position specifying unit 1212 as processing target frames;
(5) For each processing target frame extracted by the processing target frame extraction unit 1213, a motion compensation prediction error that is a value indicating the encoding efficiency of the low frame rate video signal generated when downsampling is performed on the processing target frame. A processing target frame prediction error calculation unit 1214 to calculate,
(6) Based on the motion compensation prediction error calculated by the processing target frame prediction error calculation unit 1214, a frame indicating the minimum motion compensation prediction error is selected from the processing target frames, and is used as a frame after downsampling. A frame selection unit 1215 to select;
(7) When the frame selection unit 1215 selects a frame, it is determined whether or not the processing has been completed for all the frame positions to be downsampled, and it is determined that the processing has not been completed. When performing, the all frame position end determination unit 1216 that activates the next frame position specifying unit 1212 to execute processing for the next frame position;
(8) The frame selection unit 1215 performs frame selection using each of the start frames selected by the start frame selection unit 1211α as a starting point, and based on the sum of the minimum motion compensation prediction errors used for the frame selection, A frame that finally selects a frame that has been selected as a frame after downsampling when a frame is selected from the selected start frame and frame selection unit 1215 performs frame selection from the selected start frame. Final selection unit 1217
The structure provided with.

[2-2] Flow of frame rate downsampling process (part 2)
0. Captured image signal (high frame rate video signal subject frame rate conversion process), reads the number of frames J, the frame rate (used in order to calculate the frame interval [delta] _t) 1. 1. Read downsampling ratio M for L ₀ = 0,..., Δ−1
3. for i = 1,..., J / M-1
4). E [L ₀ ] = 0
5). for L _i = −Δ,..., Δ−1
6). The frame after downsampling f (x, (iM + L i) δ t) that if the motion compensated prediction error (Equation (1)) in search of the motion vector that minimizes, the motion vector ^ d [L _i ] [k] (k = 0, 1,..., K−1), and the minimized motion compensation prediction error is stored in σ _i ² [L _i ]. The minimum value in σ _i ² [L _i ] (L _i = −Δ,..., Δ−1) is searched, and the corresponding L _i ^* is identified. At this time, σ _i ² [L _i ^* ] is stored in ^ σ _i ² [L _i ^* ] [L ₀ ], and L _i ^* is stored in ^ L _i ^* [L ₀ ]. E [L ₀ ] = E [L ₀ ] + ^ σ _i ² [L _i ^* ] [L ₀ ]
9. Search for the minimum value in E [L ₀ ] (L ₀ = 0,..., Δ−1) and identify the corresponding L ₀ ^*
Ten. F (x, (iM + ^ L _i ^* [L ₀ ] ^* ) δ _t ) (x = 0,..., X−1) is stored as the i-th frame after downsampling.

  In this way, when executing the “frame rate downsampling process flow (2)”, the downsampling execution unit 121, for example, calculates the first frame of the high frame rate video signal and the following frames. As shown in FIG. 2, the processing is started by selecting as a start frame, and the same frame selection processing as that of “flow of frame rate downsampling processing (part 1)” is executed starting from each start frame as shown in FIG. To select the sum of the optimal motion compensation prediction errors used for the frame selection, and based on the calculated sum of the motion compensation prediction errors, the minimum motion compensation prediction error sum from the start frames is calculated. When selecting a start frame that indicates and selecting a frame starting from the selected start frame By final select-option frame as a frame after downsampling is to perform a frame rate conversion processing to the low frame rate video signal from the high frame rate video signal.

[2-3] Δ value automatic setting process The unit of Δ used in the flow of the frame rate downsampling process (part 1) and the flow of the frame rate downsampling process (part 2) is a frame, and is subject to downsampling Represents the existing section of the frame candidate. In other words, Δ represents the maximum value of the expansion / contraction parameter L _i .

As described above, the value of the expansion / contraction parameter L _i is limited to Δ. If this value is too large, the degree of non-uniformity between frames after down-sampling becomes too large, and a moving image with motion is attached. It is to become.

  This Δ is a value that depends on the frame rate before and after downsampling and the image. In the above-described frame rate downsampling process flow (part 1) and frame rate downsampling process flow (part 2), the value is externally set in advance. Explained by being given by.

  Next, a method for automatically setting the value of Δ will be described.

  When automatically setting Δ, the downsampling execution unit 121 receives the error ε, the initial value Δ_ {0} of Δ, the maximum number of repetitions N, and the step width β of Δ, and performs the following processing. By executing, a process of calculating the value of Δ and the expansion / contraction parameter L_i (i = 1,..., J / M−1) corresponding to the Δ is executed.

Processing: (S1)
For Δ_ {n}, the flow of the frame rate downsampling process (part 1) or the flow of the frame rate downsampling process (part 2) is executed, and the expansion / contraction parameter L_i (i = 1,...). , J / M-1). Then, the motion compensation prediction error at this time is stored as E_ {n} (S2).
If the number of repetitions n is smaller than the given threshold value N, the process proceeds to the next process, and if not, the process ends (S3).
If E_ {n} is smaller than the given error ε, the process is terminated. Otherwise, Δ_ {n + 1} = Δ_ {n} + β and return to (S1) (S4)
Δ_ {n} is output as Δ. Then, L_i (i = 1,..., J / M−1) is output as the expansion / contraction parameter corresponding to Δ.

  In this way, when the value of Δ is automatically set using the flow (part 1) of the frame rate downsampling process, the downsampling execution unit 121 increases the frame rate while increasing the value of Δ from the initial value. By executing the down-sampling process flow (Part 1) to obtain the motion compensation prediction error at each down-sampling point, calculate the sum, and obtain the value of Δ when the sum is smaller than the error ε. , Δ is automatically set.

  At this time, if the value of Δ is increased without limitation, the degree of non-uniformity between frames after downsampling becomes too large, so by setting an upper limit value for the number of repetitions n, the value of Δ is increased. An upper limit is set.

  When the value of Δ is automatically set using the flow (part 2) of the frame rate downsampling process, the downsampling execution unit 121 increases the value of Δ from the initial value while increasing the frame rate downsampling process. When the start frame is determined based on the sum of motion compensation prediction errors when the start frame is used as a starting point by executing the flow (No. 2), the sum of the determined start frames is an error ε. By obtaining the value of Δ when it becomes smaller than that, it is possible to automatically set the value of Δ.

  At this time, if the value of Δ is increased without limitation, the degree of non-uniformity between frames after downsampling becomes too large, so by setting an upper limit value for the number of repetitions n, the value of Δ is increased. An upper limit is set.

  Next, the present invention will be described in detail according to examples.

[1] First Embodiment [1-1] Configuration when Δ is Preset FIG. 6 shows a case where the frame rate downsampling process is executed according to the flow (part 1) of the frame rate downsampling process. The flowchart which the downsampling execution part 121 performs is illustrated. Here, in this flowchart, it is assumed that Δ is set in advance.

  Next, according to this flowchart, the frame rate downsampling process executed by the downsampling execution unit 121 will be described in detail.

When the downsampling execution unit 121 executes the frame rate downsampling process in the form shown in FIG. 1 in accordance with the flow (No. 1) of the frame rate downsampling process, as shown in the flowchart of FIG. first, in step S101, the high frame rate video signal subject frame rate conversion process, and the frame number J, and the frame interval [delta] _t, and down-sampling ratio M, the maximum value of the stretch parameter L _i delta And read.

  Subsequently, in step S102, the first frame of the read high frame rate video signal is selected as a frame after down-sampling (first frame).

Subsequently, in step S103, it sets a variable i for specifying the frame position defined by the down-sampling, such as interval length based on the downsampling ratios M and frame interval [delta] _t. That is, 1 is set as a value for designating the next downsampling position of the first frame.

Subsequently, in step S104, −Δ which is the minimum value is set in the expansion / contraction parameter L _i , and in the subsequent step S105, the array σ _i ² [L _i ] (L _i = −Δ) which stores the value of the motion compensation prediction error. ,..., Δ−1) are set to 0.

Subsequently, in step S106, the minimum value of the motion compensation prediction error between the frame defined by the values of the variable i and the expansion / contraction parameter L _i and the previous frame is calculated according to the equation (1). calculating a motion compensated prediction error by continues at step S107, stored in the corresponding storage location of the calculated motion compensated prediction error sequence sigma _i ² [L _i] (L _i location pointed to).

Subsequently, in step S108, the value of the stretch parameter L _i is incremented by one, at the following step S109, the value of the stretch parameter L _i determines whether or not exceeded delta-1 which is the maximum value.

When it is determined by the determination process in step S109 that the value of the expansion / contraction parameter L _i does not exceed Δ−1, the processes in steps S106 to S108 are performed for the next frame indicated by the expansion / contraction parameter L _i . The process returns to step S106.

On the other hand, when it is determined by the determination process in step S109 that the value of the expansion / contraction parameter L _i exceeds Δ−1, the process for the downsampling position indicated by the variable i (the process for calculating the motion compensation prediction error for the 2Δ frame). ) Is completed, the process proceeds to step S110, the minimum value in the array σ _i ² [L _i ] is searched, and the expansion / contraction parameter L _i ^* associated therewith is specified.

Subsequently, in step S111, a frame defined by the values of the variable i and the expansion / contraction parameter L _i ^* is selected as a frame after downsampling.

  Subsequently, in step S112, the value of the variable i is incremented by one. In the subsequent step S113, it is determined whether or not the value of the variable i exceeds the maximum value J / M−1. When determining that the value does not exceed −1, the process returns to the process of step S104 to execute the processes of steps S104 to S112 for the next down-sampling position indicated by the variable i.

  On the other hand, when it is determined by the determination process in step S113 that the value of the variable i has exceeded J / M−1, it is determined that the processes for all the downsampling positions have been completed, and the process ends.

  In this way, the downsampling execution unit 121 executes the flowchart of FIG. 6, and in accordance with the flow (part 1) of the frame rate downsampling process, from the high frame rate video signal in the form as shown in FIG. A frame rate conversion process into a low frame rate video signal is executed.

[1-2] Configuration for automatically setting Δ (part 1)
In FIG. 7, when the frame rate downsampling process is executed according to the flow (part 1) of the frame rate downsampling process while automatically setting the maximum value Δ of the expansion / contraction parameter L _i , the downsampling execution unit 121 executes it. A flowchart is illustrated.

  Next, according to this flowchart, the frame rate downsampling process executed by the downsampling execution unit 121 will be described in detail.

When the downsampling execution unit 121 executes the frame rate downsampling process according to the flowchart of FIG. 7, first, in step S201, the high frame rate video signal to be subjected to the frame rate conversion process and its frame are displayed. read the number J, and the frame interval [delta] _t, a downsampling ratio M.

Subsequently, in step S202, sets 0 to the variable n for counting the number of repetitions, in the following step S203, sets the initial value is a small value in a variable Δ which specifies the maximum value of the stretch parameter L _i.

  Subsequently, in step S204, the process of steps S102 to S113 in the flowchart of FIG. 6 is executed to determine a selection frame (selection frame at each down-sampling position) when using the set Δ, The sum of motion compensation prediction errors at that time (the sum of motion compensation prediction errors obtained for each selected frame) is calculated.

  Subsequently, in step S205, it is determined whether or not the sum of motion compensation prediction errors calculated in step S204 is smaller than a preset error ε.

  When it is determined by the determination processing in step S205 that the sum of the motion compensation prediction errors calculated in step S204 is not smaller than the error ε, the process proceeds to step S207, and the value of the variable n is incremented by one, and the following step In S208, it is determined whether or not the value of the variable n exceeds a preset maximum value N, and when it is determined that the value does not exceed the maximum value N, the process proceeds to step S209, and the value of Δ is changed to β Then, the process returns to step S204.

  Then, according to the determination process of step S205, when it is determined that the sum of the motion compensation prediction errors calculated in step S204 is smaller than the error ε, the value of the variable n exceeds the maximum value N according to the determination process of step S208. When determining that this is the case, the process proceeds to step S206, where the selected frame when using the set Δ is selected as the frame after downsampling, and the process is terminated.

In this way, the downsampling execution unit 121 executes the flowchart of FIG. 7 to automatically set the maximum value Δ of the expansion / contraction parameter L _i , and the frame rate according to the flow (part 1) of the frame rate downsampling process. The downsampling process is executed.

[1-3] Configuration for automatically setting Δ (part 2)
In FIG. 8, when the frame rate downsampling process is executed according to the flow (No. 1) of the frame rate downsampling process while automatically setting the maximum value Δ of the expansion / contraction parameter L _i , the downsampling execution unit 121 executes it. Another flowchart is illustrated.

When the downsampling execution unit 121 executes the flowchart of FIG. 7, the sum of motion compensation prediction errors obtained at each downsampling position of the variable i (i = 1,..., J / M−1). Is calculated, and the maximum value Δ of the expansion / contraction parameter L _i is automatically set by providing a constraint that the total sum falls within a preset error ε. In the case of execution, as shown in FIG. 9, Δ is independently increased at each down-sampling position of the variable i (i = 1,..., J / M−1), By providing a constraint that each motion compensation prediction error obtained at the downsampling position falls within a preset error ε ′, it is possible to automatically set the maximum value Δ of the expansion / contraction parameter L _i .

That is, when the downsampling execution unit 121 executes the frame rate downsampling process according to the flowchart of FIG. 8, first, in step S301, the high frame rate video signal to be subjected to the frame rate conversion process, read and the frame number J, and the frame interval [delta] _t, and a downsampling ratio M.

  Subsequently, in step S302, the first frame of the read high frame rate video signal is selected as a frame after down-sampling (first frame).

Subsequently, in step S303, sets a variable i for specifying the frame position defined by the down-sampling, such as interval length based on the downsampling ratios M and frame interval [delta] _t, in the following step S304, the number of repetitions set to 0 to the variable n that counts, in a succeeding step S305, sets the initial value to a variable Δ which specifies the maximum value of the stretch parameter L _i.

Subsequently, in step S306, the process of steps S104 to S110 in the flowchart of FIG. 6 is executed using the set Δ, and the optimal expansion / contraction parameter L _i ^{* at the} down-sampling position indicated by the variable i, The associated motion compensation prediction error is calculated.

  Subsequently, in step S307, it is determined whether or not the motion compensation prediction error calculated in step S306 is smaller than a preset error ε '.

When it is determined by the determination process in step S307 that the motion compensation prediction error calculated in step S306 is smaller than the error ε ′, the process proceeds to step S308 and is defined by the value of the variable i and the expansion / contraction parameter L _i ^*. Select the frame as the frame after downsampling.

  Subsequently, in step S309, the value of the variable i is incremented by one, and in the subsequent step S310, it is determined whether or not the value of the variable i exceeds the maximum value J / M-1, and J / M When determining that the value does not exceed −1, the process returns to the process of step S305 to execute the processes of steps S305 to S309 for the next downsampling position indicated by the variable i. When the determination process in step S310 determines that the value of the variable i exceeds J / M-1, the process ends.

  On the other hand, when it is determined by the determination process in step S307 that the motion compensation prediction error calculated in step S306 is not smaller than the error ε ′, the process proceeds to step S311 to increment the value of the variable n by one, and the following step In S312, when it is determined whether or not the value of the variable n exceeds the preset maximum value N and it is determined that the value does not exceed the maximum value N, the process proceeds to step S313, and the value of Δ is changed to β Then, the process returns to step S306.

  When it is determined by the determination process in step S312 that the value of the variable n has exceeded the maximum value N, the process proceeds to step S308, and a process similar to the process described above is performed, whereby the frame after downsampling is performed. Select.

In this way, the down-sampling execution unit 121, by executing the flowchart of FIG. 8, while automatically setting the maximum value Δ of the expansion parameter L _i in accordance configuration shown in FIG. 9, the frame rate down-sampling processing flow ( The frame rate down-sampling process is executed according to 1).

[2] Second Embodiment [2-1] Configuration when Δ is Preset FIG. 10 shows a case where frame rate downsampling processing is executed according to the flow (part 2) of frame rate downsampling processing. The flowchart which the downsampling execution part 121 performs is illustrated. Here, in this flowchart, it is assumed that Δ is set in advance.

  Next, according to this flowchart, the frame rate downsampling process executed by the downsampling execution unit 121 will be described in detail.

When the downsampling execution unit 121 executes the frame rate downsampling process in the form shown in FIG. 2 according to the flow (part 2) of the frame rate downsampling process, as shown in the flowchart of FIG. first, at step S401, the high frame rate video signal subject frame rate conversion process, and the frame number J, and the frame interval [delta] _t, and down-sampling ratio M, the maximum value of the stretch parameter L _i delta And read.

Subsequently, in step S402, 0 is set to a variable L ₀ that designates the number of frames from the first frame of the read high frame rate video signal.

Subsequently, in step S403, 0 is set in all storage locations of the array ^ L _i ^* [L ₀ ] for storing the expansion / contraction parameters, and the array ^ σ _i ² [L _i ^* ] [ 0 is set in all storage locations of L ₀ ], and 0 is set in all storage locations of the array E [L ₀ ] that stores the sum of motion compensation prediction errors.

Subsequently, in step S404, a frame that is after the first frame of the read high frame rate video signal by the value L ₀ is set as a frame after down-sampling (first frame).

Subsequently, at step S405, 1 is set to the variable i specifying the frame position defined by the down-sampling, such as interval length based on the downsampling ratios M and frame interval [delta] _t. That is, 1 is set as a value for designating the next downsampling position of the first frame.

Subsequently, in step S406, the processes of steps S104 to S110 in the flowchart of FIG. 6 are executed, and the optimal expansion / contraction parameter L _i ^{* at the} down-sampling position indicated by the variable i and the motion compensation prediction error associated therewith are obtained. calculate.

Subsequently, in step S407, the expansion / contraction parameters calculated in step S406 are stored in the array ^ L _i ^* [L ₀ ], and the motion compensation prediction error calculated in step S406 is stored in the array ^ σ _i ² [L _i ^* ] [L ₀ ].

Subsequently, in step S408, for the set L ₀ , the corresponding value of the array E [L ₀ ] and the corresponding value of the array ^ σ _i ² [L _i ^* ] [L ₀ ] are added. Based on this, the value of the array E [L ₀ ] is updated. That is, the motion compensation prediction error is obtained by performing the frame selection process of the first embodiment with the first frame selected based on the set L ₀ as the starting point, and the motion compensation prediction error thus obtained is obtained. Cumulative addition is performed.

  Subsequently, in step S409, the value of the variable i is incremented by one. In the subsequent step S410, it is determined whether or not the value of the variable i exceeds J / M−1 which is the maximum value. When it is determined that the value does not exceed −1, the process returns to step S406 to execute the process of steps S406 to S409 for the next down-sampling position indicated by the variable i.

On the other hand, when it is determined that the value of the variable i exceeds J / M−1 in accordance with the determination process in step S410, the process proceeds to step S411, and the value of the variable L ₀ is incremented by 1, and in the subsequent step S412 When it is determined whether or not the value of the variable L ₀ exceeds the maximum value Δ−1 and it is determined that the value does not exceed Δ−1, the next head frame indicated by the variable L ₀ is set as the starting point. In order to execute the processing in steps S404 to S411, the processing returns to step S404.

Then, when it is determined that the value of the variable L ₀ exceeds Δ−1 according to the determination process in step S412, the process proceeds to step S413, and the minimum value in the array E [L ₀ ] is searched. The expansion / contraction parameter [^ L _i ^* ] [L ₀ ] ^* to be associated is specified. That is, the value of L ₀ that minimizes the sum of motion compensation prediction errors is specified, and the expansion / contraction parameter at each down-sampling position obtained by frame selection processing starting from the first frame selected based on L ₀ is determined. Identify the set.

Subsequently, in step S414, a frame defined by the value of the variable i and the expansion / contraction parameter [^ L _i ^* ] [L ₀ ] ^* is selected as a frame after downsampling, and the process is terminated.

  In this way, the downsampling execution unit 121 executes the flowchart of FIG. 10, and in accordance with the flow (No. 2) of the frame rate downsampling process, from the high frame rate video signal in the form as shown in FIG. A frame rate conversion process into a low frame rate video signal is executed.

[2-2] Configuration for automatically setting Δ (part 1)
In FIG. 11, when the frame rate downsampling process is executed according to the flow (2) of the frame rate downsampling process while automatically setting the maximum value Δ of the expansion / contraction parameter L _i , the downsampling execution unit 121 executes it. A flowchart is illustrated.

  Next, according to this flowchart, the frame rate downsampling process executed by the downsampling execution unit 121 will be described in detail.

When the downsampling execution unit 121 executes the frame rate downsampling process according to the flowchart of FIG. 11, first, in step S501, the high frame rate video signal to be subjected to the frame rate conversion process and its frame read the number J, and the frame interval [delta] _t, and a downsampling ratio M.

Subsequently, in step S502, sets 0 to the variable n for counting the number of repetitions, in the subsequent step S503, sets the initial value is a small value in a variable Δ which specifies the maximum value of the stretch parameter L _i.

  Subsequently, in step S504, the processing of steps S402 to S414 in the flowchart of FIG. 10 is executed, and the selected frame when using the set Δ (each downsampling position obtained in the form shown in FIG. 2). (Selected frame) is determined, and the total of motion compensation prediction errors at that time is calculated.

  Subsequently, in step S505, it is determined whether or not the sum of motion compensation prediction errors calculated in step S504 is smaller than a preset error ε.

  If it is determined by the determination processing in step S505 that the sum of motion compensation prediction errors calculated in step S504 is not smaller than the error ε, the process proceeds to step S507, and the value of the variable n is incremented by one, and the following step In S508, it is determined whether or not the value of the variable n exceeds a preset maximum value N, and when it is determined that the value does not exceed the maximum value N, the process proceeds to step S509, and the value of Δ is changed to β Then, the process returns to step S504.

  Then, according to the determination process in step S505, when it is determined that the sum of motion compensation prediction errors calculated in step S504 is smaller than the error ε, the value of the variable n exceeds the maximum value N according to the determination process in step S508. When determining that this is the case, the process proceeds to step S506, where the selected frame when using the set Δ is selected as the frame after downsampling, and the process is terminated.

In this way, the downsampling execution unit 121 executes the flowchart of FIG. 11 to automatically set the maximum value Δ of the expansion / contraction parameter L _i , and the frame rate according to the frame rate downsampling process flow (part 2). The downsampling process is executed.

[2-3] Configuration for automatically setting Δ (part 2)
12 and 13, when the frame rate down-sampling process is executed according to the flow (No. 2) of the frame rate down-sampling process while automatically setting the maximum value Δ of the expansion / contraction parameter L _i , the down-sampling execution unit 121 Fig. 4 illustrates another flowchart executed by the program.

When the flowchart of FIG. 11 is executed, the downsampling execution unit 121 calculates the sum of motion compensation prediction errors used for final frame selection (the array E [L ₀ ] according to the process of step S413 in the flowchart of FIG. 8). The maximum value Δ of the expansion / contraction parameter L _i has been automatically set by providing a constraint that the sum of motion compensation prediction errors searched from within the error ε set in advance is set. When the flowcharts of FIGS. 12 and 13 are executed, as shown in FIG. 14, when the frame rate down-sampling process is executed according to the flow (No. 2) of the frame rate down-sampling process, the variable i (i = 1) , ...., J / M-1) at each down-sampling position, Δ is independently increased and each down-sampling is performed. By providing a constraint that each motion compensation prediction error obtained at the ring position falls within a preset error ε ′, it is possible to automatically set the maximum value Δ of the expansion / contraction parameter L _i .

That is, when the downsampling execution unit 121 executes the frame rate downsampling process according to the flowcharts of FIGS. 12 and 13, first, in step S601, the high frame rate video to be subjected to the frame rate conversion process. reading a signal, and the frame number J, and the frame interval [delta] _t, and a downsampling ratio M.

Subsequently, in step S602, 0 is set to a variable L ₀ that specifies the number of frames from the first frame of the read high frame rate video signal.

Subsequently, in step S603, 0 is set in all storage locations of the array ^ L _i ^* [L ₀ ] for storing the expansion / contraction parameters, and the array ^ σ _i ² [L _i ^* ] [ 0 is set in all storage locations of L ₀ ], and 0 is set in all storage locations of the array E [L ₀ ] that stores the sum of motion compensation prediction errors. In step S604, a variable n for counting the number of repetitions is set to 0.

Subsequently, in step S605, it sets the frame from the first frame of the high frame rate video signal read after only the value of L ₀ as a down-sampled frame (first frame).

Subsequently, in step S606, the sets a variable i for specifying the frame position defined by the down-sampling, such as interval length based on the downsampling ratios M and frame interval [delta] _t, in the subsequent step S607, the expansion parameter L _An initial value is set to a variable Δ that specifies the maximum value of _i .

Subsequently, in step S608, the process of steps S104 to S110 in the flowchart of FIG. 6 is executed using the set Δ, and the optimal expansion / contraction parameter L _i ^{* at the} down-sampling position indicated by the variable i, The associated motion compensation prediction error is calculated.

  Subsequently, in step S609, it is determined whether or not the motion compensation prediction error calculated in step S608 is smaller than a preset error ε '.

When it is determined by the determination processing in step S609 that the motion compensation prediction error calculated in step S608 is smaller than the error ε ′, the process proceeds to step S610, and the expansion / contraction parameters calculated in step S608 are arranged in the array ^ L _i ^* [ L ₀ ] and the motion compensation prediction error calculated in step S 608 is stored in the array σσ _i ² [L _i ^* ] [L ₀ ].

Subsequently, in step S611, for the set L ₀ , the corresponding value of the array E [L ₀ ] and the corresponding value of the array ^ σ _i ² [L _i ^* ] [L ₀ ] are added, Based on this, the value of the array E [L ₀ ] is updated. That is, the motion compensation prediction error is obtained by performing the frame selection process of the first embodiment with the first frame selected based on the set L ₀ as the starting point, and the motion compensation prediction error thus obtained is obtained. Cumulative addition is performed.

  Subsequently, in step S612, the value of the variable i is incremented by one, and in the subsequent step S613, it is determined whether or not the value of the variable i exceeds the maximum value J / M−1. When it is determined that the value does not exceed −1, the process returns to step S607 to execute the processes of steps S607 to S612 for the next downsampling position indicated by the variable i.

On the other hand, when it is determined that the value of the variable i exceeds J / M−1 in accordance with the determination process in step S613, the process proceeds to step S614, and the value of the variable L ₀ is incremented by one, and in the subsequent step S615. When it is determined whether or not the value of the variable L ₀ exceeds the maximum value Δ−1 and it is determined that the value does not exceed Δ−1, the next head frame indicated by the variable L ₀ is set as the starting point. In order to execute the processing of step S605 to step S614, the processing returns to the processing of step S605.

Then, when it is determined that the value of the variable L ₀ exceeds Δ−1 according to the determination process in step S615, the process proceeds to step S616 (the flowchart in FIG. 13), and the minimum _value in the array E [L ₀ ]. The value is searched, and the expansion / contraction parameter [^ L _i ^* ] [L ₀ ] ^* associated therewith is specified. That is, the value of L ₀ that minimizes the sum of motion compensation prediction errors is specified, and the expansion / contraction parameter at each down-sampling position obtained by frame selection processing starting from the first frame selected based on L ₀ is determined. Identify the set.

Subsequently, in step S617, a frame defined by the variable i and the expansion / contraction parameter [^ L _i ^* ] [L ₀ ] ^* is selected as a frame after downsampling, and the process is terminated.

  On the other hand, when it is determined that the motion compensation prediction error calculated in step S608 is not smaller than the error ε ′ according to the determination process in step S609, the process proceeds to step S618 (flowchart in FIG. 13), and the value of the variable n is set to 1. In the subsequent step S619, it is determined whether or not the value of the variable n has exceeded a preset maximum value N, and when it is determined that the value has not exceeded the maximum value N, the process proceeds to step S620. , Δ is increased by β, and the process returns to step S608.

  Then, when it is determined by the determination process in step S619 that the value of the variable n has exceeded the maximum value N, the process proceeds to step S610, and the same process as the process described above is performed, whereby the frame after downsampling is performed. Select.

In this way, the down-sampling execution unit 121, by executing the flowcharts of FIGS. 12 and 13, while automatically setting the maximum value Δ of the expansion parameter L _i in accordance configuration shown in FIG. 14, the frame rate down-sampling processing The frame rate down-sampling process is executed according to the flow (No. 2).

  Although the present invention has been described with reference to the illustrated embodiments, the present invention is not limited thereto. For example, in the embodiment, the motion compensation prediction error is calculated with respect to the previous frame selected according to the equation (1), but the motion compensation prediction error with a plurality of selected frames is calculated. May be calculated.

  The present invention can be applied when performing frame rate conversion from a high frame rate video signal to a low frame rate video signal. By applying the present invention, the present invention can be obtained by downsampling a high frame rate video signal. When a video signal having a desired frame rate is encoded, the code amount can be suppressed to be lower than that of a video signal obtained by thinning out uniform frame intervals.

It is explanatory drawing of the frame rate downsampling process by this invention. It is explanatory drawing of the frame rate downsampling process by this invention. It is an apparatus block diagram of the frame rate conversion apparatus of this invention. It is an apparatus block diagram of the frame rate conversion apparatus of this invention. It is an apparatus block diagram of the frame rate conversion apparatus of this invention. It is a flowchart which a downsampling execution part performs. It is a flowchart which a downsampling execution part performs. It is a flowchart which a downsampling execution part performs. It is explanatory drawing of the frame rate downsampling process by this invention. It is a flowchart which a downsampling execution part performs. It is a flowchart which a downsampling execution part performs. It is a flowchart which a downsampling execution part performs. It is a flowchart which a downsampling execution part performs. It is explanatory drawing of the frame rate downsampling process by this invention. It is explanatory drawing of the frame rate downsampling process by a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Frame rate conversion apparatus 10 High frame rate video file 11 Low frame rate video file 12 Frame rate downsampling unit 120 Motion compensation prediction error calculation unit 121 Downsampling execution unit 122 Work memory 1210 Input unit 1211 First frame selection unit 1211α Start frame Selection unit 1212 Next frame position specification unit 1213 Processing target frame extraction unit 1214 Processing target frame prediction error calculation unit 1215 Frame selection unit 1216 All frame position end determination unit 1217 Frame final selection unit 1200 Motion estimation unit

Claims (10)

A frame rate conversion method for converting a frame of a high frame rate video signal into a low frame rate video signal by selecting by downsampling,
Input the downsampling ratio;
When a frame after downsampling is selected, a process of specifying a next frame position defined by equal-length interval downsampling based on the downsampling ratio and the frame interval of the high frame rate video signal;
A value indicating the encoding efficiency of a low frame rate video signal generated when a plurality of frames within a predetermined range including the frame position are to be processed and each processing target frame is down-sampled with the processing target frame. A process of calculating an encoding efficiency display value,
Selecting a frame indicating an optimal encoding efficiency display value from the processing target frame, and selecting it as a frame after downsampling.
A featured frame rate conversion method. The frame rate conversion method according to claim 1,
Comprising a step of selecting a frame to be a first frame after downsampling from frames of a high frame rate video signal;
A featured frame rate conversion method. In the frame rate conversion method according to claim 2,
The frame is selected by increasing the size of the predetermined range from an initial value, and the individual values of the optimum encoding efficiency display value used for the frame selection, or the sum of these values is a desired code. The first increase value indicating the conversion efficiency is obtained, and the frame selected when the frame selection is performed using the predetermined range having the magnitude of the obtained increase value is finally set as a frame after downsampling. Having a process to decide,
A featured frame rate conversion method. The frame rate conversion method according to claim 1,
A process of selecting a plurality of consecutive frames as candidates for the first frame after downsampling from the frames of the high frame rate video signal as start frames,
The start frame indicating the optimum coding efficiency among the start frames based on the sum of the optimum coding efficiency display values used for the frame selection by performing the frame selection from each of the start frames. And a final selection of the frame selected when the frame is selected from the selected start frame as a frame after downsampling.
A featured frame rate conversion method. In the frame rate conversion method according to claim 4,
The size of the predetermined range is increased from the initial value to perform the final selection of the frame, and the sum of the optimum coding efficiency display values used for the final selection of the frame, or individual constituting the sum The first increase value whose value indicates the desired encoding efficiency is obtained, and when the final selection of the frame is performed using the predetermined range having the magnitude of the obtained increase value, the last selected frame is downed. Having a process of final determination as a frame after sampling,
A featured frame rate conversion method. The frame rate conversion method according to any one of claims 1 to 5,
In the calculating step, calculating the motion compensation prediction error power calculated based on the processing target frame and the selected frame as the coding efficiency display value;
A featured frame rate conversion method. The frame rate conversion method according to any one of claims 1 to 5,
In the calculating step, calculating the encoding cost calculated based on the processing target frame and the selected frame as the encoding efficiency display value,
A featured frame rate conversion method. A frame rate conversion device that converts a frame of a high frame rate video signal into a low frame rate video signal by selecting it by downsampling,
Means for inputting a downsampling ratio;
Means for specifying a next frame position defined by equal-length interval downsampling based on the downsampling ratio and the frame interval of the high frame rate video signal when a frame after downsampling is selected;
A value indicating the encoding efficiency of a low frame rate video signal generated when a plurality of frames within a predetermined range including the frame position are to be processed and each processing target frame is down-sampled with the processing target frame. Means for calculating an encoding efficiency display value,
Selecting a frame indicating an optimal encoding efficiency display value from the processing target frame, and selecting the frame as a frame after downsampling.
A frame rate conversion device.   A frame rate conversion program for causing a computer to execute the frame rate conversion method according to any one of claims 1 to 7.   A computer-readable recording medium on which a frame rate conversion program for causing a computer to execute the frame rate conversion method according to any one of claims 1 to 7 is recorded.

Images