JP5462592B2 - VIDEO PROCESSING DEVICE, ITS CONTROL METHOD, AND PROGRAM - Google Patents

Description

   The present invention relates to a technique for converting an interlace video signal into a progressive video signal.

  Conventionally, most of video signals are interlaced images, that is, images on the premise that they are displayed by interlaced scanning. In this interlaced image, line flicker is known, and when the line of sight moves at a speed such that the Y-direction component becomes an integral multiple of the line, interlace interference occurs, such as the lack of a line, and image quality deteriorates.

  In order to solve this problem, many technologies and products have been announced in which an interlaced image is converted into a progressive image (hereinafter referred to as IP conversion) and sequentially scanned.

  Conventional IP conversion mainly includes a motion adaptive type and a motion compensation type. The motion adaptive IP conversion determines whether or not there is motion in the video, and if there is motion, interpolates the missing line by in-field processing. If there is no movement, the missing line is complemented by inter-field processing.

  As a method of complementing the missing line by the in-field processing, for example, there are a method of applying the same data as the upper or lower line as it is, a method of applying the average value of the upper and lower lines, and the like. There is also a method of applying the same address in the front or rear field as it is, or applying it by averaging it. Note that the presence / absence of motion may be determined for the entire image or for each region.

  On the other hand, in motion compensated IP conversion, a missing line is always complemented by inter-field processing regardless of the presence or absence of motion (Patent Document 1). In this case, a motion vector of the video is detected, an image of the field sandwiched between the preceding and succeeding fields is predicted, and a missing line of the field is interpolated based on the prediction.

JP-A-10-112845

  In motion-adaptive IP conversion, when there is motion, the vertical resolution of an image often decreases by interpolating missing lines from the upper and lower lines. In addition, there is a problem that an oblique edge portion is not smooth. In addition, the motion compensated IP conversion has a problem that, first, the calculation scale becomes enormous, or errors often occur when calculating a motion vector.

  The present invention has been made in view of such a problem, and intends to provide a technique for converting an interlaced video signal into a progressive video signal by performing in-field processing and without causing edge portions to deteriorate with respect to the interlaced image. Is.

In order to solve this problem, for example, a video processing apparatus of the present invention has the following configuration. That is,
A video processing device that generates one progressive video signal from one field video signal in interlaced video,
A spatial low frequency component data is generated by filtering the video signal of each line of the input field video signal, and the spatial low frequency component data is subtracted from the video signal of each line of the field video signal. a filter means for generating a high-frequency component data,
Calculating means for calculating an average value of spatial low-frequency component data for two adjacent lines obtained by the filter means and outputting averaged low-frequency component data for one line;
And generating means for generating a progressive video signal by alternately arranging lines of spatial high-frequency component data obtained by the filter means and lines of averaged low-frequency component data calculated by the calculating means.

  According to the present invention, it is possible to convert an interlaced video signal into a progressive video signal by in-field processing and without causing edge portions to deteriorate with respect to the interlaced image.

The block block diagram of the video processing apparatus of embodiment. The figure showing the arrangement | sequence of data when carrying out IP conversion by the conventional method. The figure showing the data arrangement when carrying out IP conversion of an embodiment. The figure showing the example of the interlaced odd image and the even image. The figure showing the example of an odd image and an even image when carrying out IP conversion by the conventional method. The figure which shows the example of the odd image and the even image at the time of carrying out IP conversion in embodiment. The flowchart which shows the process sequence in 1st Embodiment. , The flowchart which shows the process sequence in 2nd Embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  In the present embodiment, a method for IP conversion of an input interlaced image by intra-frame processing is proposed. In this embodiment, by dividing the spatial frequency low-frequency component of the non-missing line into upper and lower missing lines for each field video signal, the data of the non-missing line is converted into high-frequency emphasized data (more precisely, low-frequency suppression data). To do. The missing line is used as high-frequency suppression data based on the spatial low-frequency component moved from the upper and lower non-missing lines. As a result, the part where high-frequency components dominate, such as the edge part, is a characteristic close to an interlaced image, and in the part that is not the edge, the missing line is the average value of the upper and lower non-missing lines, and the overall level is halved. Have characteristics close to the image.

  In order to simplify the description, the input source of the interlaced image to be processed is assumed to be a digital interlaced video signal from a video camera, a DVD player or the like, but the type of the input source is not limited. An interlaced image (field image) input to the apparatus of this embodiment is A. An image composed only of non-missing lines of the interlaced image A is expressed as A_int, particularly A_int in the case of odd is expressed as A_odd, and A_int in the case of even is expressed as A_even. Let H_odd and H_even be the spatial high-frequency components generated by filtering. Similarly, the low frequency components generated by filtering are L_odd and L_even, respectively, and the data corresponding to the non-missing lines as the result of the filtering process and the subtraction process are SH_odd and SH_even and the data corresponding to the missing lines are SL_odd and SL_even. Images obtained by alternately arranging the data corresponding to the non-missing lines and the data corresponding to the missing lines for each line are A_prg_odd and A_prg_even. In addition, an image in which A_prg_oddA and prg_even alternately alternate in time (that is, a progressive image) is expressed as A_prg. For example, when dealing with 1920 * 1080 pixel video as a target, A_int, A_odd, A_even, H_odd, H_even, L_odd, L_even, SH_odd, SH_even, SL_odd, SL_even are all two-dimensional array data of 1920 * 540, and A_prg_odd A_prg_even and A_prg are 1920 * 1080 two-dimensional array data.

  In FIG. 1, the field buffer 1 has a capacity for storing one field video signal (interlaced image) A_int. The switches 2 and 3 switch between odd and even for each field time based on a control signal (not shown). While the switch 2 is connected to the terminal odd, the switch 6 is connected to the terminal even. Conversely, while the switch 2 is connected to the terminal even, the switch 6 is connected to the terminal odd. The odd field processing unit 101 generates a progressive image from the odd field A_odd. The even field processing unit 102 generates a progressive image from the even field A_even. Since both the odd field processing unit 101 and the even field processing unit 102 have the same configuration, the configuration of the even field processing unit 102 is not shown in the drawing.

  The filter unit 3 in the odd field processing unit 101 divides the input interlaced image A_int into spatial low frequency component data and spatial high frequency component data, and outputs them. The line averaging unit 4 is a block that averages adjacent lines (adjacent lines as progressive data) of field data. The synthesizing unit 5 has a frame memory 5a therein, and creates progressive one-frame data by alternately storing non-missing line data and missing line data for each line. When the creation of one progressive frame data is completed, the switch 6 is switched to the terminal odd, so the synthesizer 5 outputs the frame data stored in the frame 5a. At this time, the switch 2 is switched to the terminal even, and the creation process of 1 progressive frame data by the even field processing unit 102 is started.

Hereinafter, the processing of the odd field processing unit 101 will be described in more detail. The input interlaced image A_int is signaled to the switch 2 at the odd side path as shown in FIG. 1 at the odd timing. The filter unit 3 in the odd field processing unit 101 calculates and outputs a low frequency component L_odd by performing filtering processing by the LPF 7 on the interlaced image A_odd.
L_odd = LPF (A_odd)… Formula 1-1

The two-dimensional spatial filter used in the filtering process of the LPF 7 does not particularly define the function, but a Gaussian function, for example, is preferable. However, since A_odd is data composed only of non-missing lines, the image data is an image shrunk in half in the Y direction (vertical direction). Therefore, the distance constant of the LPF 7 needs to be ½ times the original distance constant only in the Y direction. Therefore, when using a Gaussian function,
σx = σ, σy = σ / 2 Formula 1-2
It is necessary to.

Further, the subtractor 8 in the filter unit 3 calculates data corresponding to the non-missing line as in the following equation.
SH_odd = A_odd−L_odd / 2 Equation 1-3

The line averaging unit 4 is represented by pixel data of a line that is one line delayed by one line by the line delay unit 9 (two lines as progressive data) and L_odd 1 from the filter unit 3. The average of the pixel data of the line is calculated. This calculation is performed by the adder 10 and the ½ multiplier 11. That is, the adder 10 adds pixel data at the same position in the horizontal direction for two lines, and the ½ multiplier 11 halves the addition result to obtain the averaged low-frequency component data. It can be calculated. If this point is clearly shown, the line averaging unit 4 is nothing but calculating the missing line (data of the line one above the progressive data L_odd) according to the following equation.
SL_odd = (L_odd + L_odd (1 line shift)) / 2 ... Formula 1-4

  The synthesizer 5 stores the non-missing line data SH_odd and the missing line data SL_odd alternately in the internal frame memory 5a. As a result, progressive data A_prg_odd is generated in the frame memory 5a. The even field processing unit 102 performs the same processing as the odd field processing unit 101. However, the order of SH_odd and SL_odd stored in the frame memory of the synthesis unit (not shown) in the even field processing unit 102 is opposite to that of the synthesis unit 5 of the odd field processing unit 101. As described above, the switch 6 selects the one for which the creation of progressive data has been completed, and outputs the progressive data. As a result, the switch 6 outputs a progressive image A_prg in which A_prg_odd and A_prg_even are alternately arranged in time.

  Here, FIG. 2 shows an IP conversion example of the conventional method, and FIG. 3 shows an IP conversion example of the present embodiment. 4 shows the state of the interlaced odd image and the even image, FIG. 5 shows the state of the odd image and the even image when IP conversion is performed by the conventional method, and FIG. 6 shows the state of the IP conversion by the method of this embodiment. It shows the state of the odd image and the even image. As can be seen from the comparison between FIG. 5 and FIG. 6, the edge portion has been blurred in the past, whereas in the processing of this embodiment, the high-frequency component of the edge portion is maintained and the deterioration of the interlaced image is small. You can see that

  Next, a processing procedure from 2 field input to 2 frame generation in the present embodiment will be described with reference to the flowchart of FIG. Note that the processing according to the flowchart shown in FIG. 11 is basically repeated until a stop instruction is given from the user or until there is no field image to be converted.

  Steps S101 to S105 correspond to the processing of the odd field processing unit 101, and steps S106 to S110 correspond to the processing of the even field processing unit 102.

  First, the odd field A_odd of the input image is stored in the field buffer 1 (S101). Next, the LPF 7 performs a two-dimensional filter process on the odd field to calculate low frequency component data L_odd (image data from which spatial high frequency component data has been removed) (S102). Next, SH_odd is calculated from A_odd and L_odd, and written to the address position corresponding to the odd row of the frame memory 5a as the data of the odd row of A_prg (S103). Next, SL_odd is calculated from A_odd and L_odd, and written to the address position corresponding to the even row of the frame memory 5a as the data of the even row of A_prg. Thereafter, the image data stored in the frame memory 5a is set as one frame of the progressive image, and output to the outside (display device or the like) is started.

  During the output to the outside by the odd field processing unit 101, the even field processing unit 102 performs the generation processing of one frame data of the progressive image based on the even field, similarly to the odd field processing unit 101. It will be.

  Specifically, the even field A_even of the input image is stored in the field memory 1 (S106). Next, L_even is calculated by performing two-dimensional filter processing (LPF) on A_even (S107). Thereafter, SH_even is calculated from A_even and L_even, and written as data of even rows of A_prg at the address positions corresponding to the even rows of the frame memory 5a included in the synthesis unit of the Even field processing unit 102 (S108). Next, SL_odd is calculated from A_odd and L_odd, and is written in the address position corresponding to the even row of the frame memory as data of the even row of A_prg (S109). Thereafter, the image data stored in the frame buffer is set as one frame of the progressive image, and output to the outside is started.

  As described above, according to the present embodiment, it is possible to perform IP conversion with in-field processing and without causing edge portions to deteriorate as compared with an interlaced image and without reducing the resolution in the Y direction.

  In the embodiment, the configuration in FIG. 1 has been described as having the odd field processing unit 101 and the even field processing unit 102. However, since these differ only in the storage order in the frame memory 5a, if the field input by the combining unit 5 is determined to be odd or even and the storage order is changed accordingly, the field processing unit May be one.

  1 is provided with a motion detector that determines whether or not there is motion in the input interlaced image, and when the motion detector determines that there is motion, the processing of the above embodiment is performed. When it is determined that there is no movement, processing may be performed in the same manner as in the conventional technique.

[Second Embodiment]
In the first embodiment, the spatial filter used by the LPF 7 has been described as a two-dimensional filter. In this way, the resolution in the Y direction of the interlaced image can be maintained even after IP conversion. However, in this case, the line flicker at the straight edge (the straight line having no edge perpendicular to the Y direction) remains. Therefore, in the second embodiment, the resolution in the Y direction remains the same as that in the conventional method. A method for suppressing flicker and maintaining the smoothness of the interlace with the oblique edges will be described.

  In the present embodiment, a one-dimensional filter is used instead of the two-dimensional filter in the second embodiment. For example, when a Gaussian function is used, the LPF is defined only by σx = σ and only in the X direction. Other calculation processing methods are the same as those in the first embodiment. By doing so, the intention of the second embodiment can be realized while suppressing line flicker. Further, according to the second embodiment, the calculation process can be performed in units of lines, not in the field, and the starting scale can be further reduced.

  Since the apparatus of the second embodiment is substantially the same as the first embodiment except that the LPF 7 uses a one-dimensional filter, description of the configuration of the apparatus is omitted. Hereinafter, the processing procedure from the two-field input to the generation of two frames according to the second embodiment will be described with reference to FIGS.

  First, the odd field (A_odd) of the input image is stored in the field memory 1 (S201). At this time, a variable yo described below is initialized with “1”. Thereafter, the A_odd data of the yo row is read from the field memory 1 (S202), and the L_odd of the yo row is calculated as d (S203). The calculated L_odd is stored in the 1-line delay unit 9 at this time. Next, SH_odd corresponding to the yo-th row is calculated by the subtracter 8 and written to the frame memory 5a as the 2 * y0 + 1-th row data of the progressive image (A_prg) (S204). Further, the adder 10 and the 1/2 multiplier 11 calculate SL_odd corresponding to the yo-th row and write it into the frame memory 5a as the 2 * yo-th row data of the progressive image (A_prg) (S205). Then, it is determined whether or not the variable yo has become a value of 540 or more, that is, whether or not the processing for all the lines in the input field has been completed (S207). If not, the variable yo is increased by “1”, and the processes in and after step S202 are repeated. If the determination in step S207 is “Yes”, since one frame of the progressive image is stored in the frame memory 5a, output to the outside is started (S209).

  The above is the processing of the odd field processing unit 101. Next, the even field processing unit 102 is performed. The processes S210 to S218 are substantially the same as S201 to S209. However, the correspondence between the line number in the field of the input image and the line number of the progressive image to be output is different from the processing of the odd field processing unit 101.

  As described above, according to the second embodiment, the vertical resolution is the same as that of the conventional method, line flicker is suppressed, and the slanted edge maintains the smoothness of the interlace. It becomes possible.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  In the embodiment, the output destination after conversion from an interlaced video signal to a progressive video signal is described as a display device. However, the converted video signal may be stored and saved as an electronic file. .

Claims (6)

A video processing device that generates one progressive video signal from one field video signal in interlaced video,
A spatial low frequency component data is generated by filtering the video signal of each line of the input field video signal, and the spatial low frequency component data is subtracted from the video signal of each line of the field video signal. a filter means for generating a high-frequency component data,
Calculating means for calculating an average value of spatial low-frequency component data for two adjacent lines obtained by the filter means and outputting averaged low-frequency component data for one line;
Generation means for generating a progressive video signal by alternately arranging lines of spatial high-frequency component data obtained by the filter means and lines of averaged low-frequency component data calculated by the calculation means;
A video processing apparatus comprising:   2. The video processing apparatus according to claim 1, wherein the filter unit is a two-dimensional spatial filter, and a distance constant in the vertical direction of the two-dimensional spatial filter is ½ times the horizontal direction.   The video processing apparatus according to claim 1, wherein the filter means is a horizontal one-dimensional filter.   4. The high frequency component data is generated by subtracting 1/2 of the spatial low frequency component data from the video signal of each line of the input field video signal. The video processing apparatus according to any one of the above. A control method of a video processing apparatus for generating one progressive video signal from one field video signal in interlaced video,
A spatial low frequency component data is generated by filtering the video signal of each line of the input field video signal, and the spatial low frequency component data is subtracted from the video signal of each line of the field video signal. a filter step of generating a high-frequency component data,
A calculation step of calculating an average value of spatial low-frequency component data for two adjacent lines obtained in the filtering step and outputting averaged low-frequency component data for one line;
A step of generating a progressive video signal by alternately arranging lines of spatial high-frequency component data obtained in the filtering step and lines of averaged low-frequency component data calculated in the calculation step;
A control method for a video processing apparatus comprising:   A program for causing a computer to function as the video processing apparatus according to claim 1 by being read and executed by a computer.