JPH0232835B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION (Technical Field) The present invention relates to a television format conversion method for converting the number of frames and number of scanning lines of a television image signal. This is designed to reduce image quality deterioration such as blurring of parts and jaggedness.

(Prior Art) As shown in FIG. 8, a conventional television format conversion device converts an M-line/N-field television signal into a K-line/L-field television signal of a different format. ,
An interpolation filter 41 and a thinning circuit 42 perform spatiotemporal linear signal processing to convert the number of fields and the number of scanning lines per second. For example, when converting an input television signal of N=60 fields/second to an output television signal of L=50 fields/second, the first to
The 6th field is the 5th field a~e of the output signal
The conversion period is the conversion to . In such field number conversion, linear signal processing is performed to obtain a weighted average of the input image signals of the preceding and following fields to form an output image signal of a new field located between the input image signals of two consecutive fields. When there is movement in an image, the positions of the input image signals of the two fields for which the weighted average is calculated are shifted, causing blurring and jaggedness at the edges of the image, which is a serious problem in that moving images cause a significant deterioration in image quality. There were flaws.

This conventional drawback naturally occurs due to simple linear interpolation. We proposed a method that detects motion and motion vectors and interpolates the image position in a controlled manner, thereby significantly improving the above-mentioned drawbacks of ordinary images. It is true that many of the images contain simple movements such as panning and tilting to follow a specific subject, and although this method can achieve a considerable effect, it is relatively easy to notice different movements in the same image. Some images include a plurality of image parts, and in such cases, there is a drawback that a sufficient effect of improving image quality cannot be obtained.

(Effects of the Invention) It is an object of the present invention to eliminate the above-mentioned remaining drawbacks, and to eliminate motion blur in the converted output image signal even if the image of the television signal to be subjected to format conversion has a plurality of different parts. An object of the present invention is to provide a format conversion device that can significantly reduce image quality deterioration due to image quality and jitteriness.

(Structure of the Invention) That is, when converting the number of frames of a television system, the system conversion device of the present invention converts the original image signals of consecutive N and N+1 frames into a linearly interpolated image of a new frame by linear interpolation. At the same time as forming a signal, the original image is divided into a plurality of regions, a motion vector is detected according to the direction and distance of movement of the image in each region, and the N and N+1 frames are determined according to these vectors. forming a plurality of N and N+1 frames of motion compensated image signals by respectively shifting the entire original image signal to form a plurality of new frames of motion compensated interpolated image signals; and the motion compensated image signals of the plurality of N and N+1 frames, the linearly interpolated image signal corresponding to the minimum value of these absolute values, or the plurality of motion compensated interpolants. The present invention is characterized in that a converted output image signal is formed by selecting one of the image signals.

(Example) Before explaining the embodiments of the present invention, in order to facilitate understanding of the apparatus of the present invention, the above-mentioned patent application related to the present invention will be described.
The embodiment of No. 59-244638 will be described with reference to the drawings.

FIG. 10 shows an example of the configuration of a motion compensated frame number converter in the television system converter according to the patent application. In the frame number converter having the illustrated configuration, sequentially scanned image signals are supplied to linear interpolation filters 14, 15, 16, 19, motion adjustment buffer memories 22, 23, and motion and motion vector detection circuit 24. The linear interpolation filter performs linear interpolation on the input progressive scanning image signal by a weighted average of two consecutive frame signals to form a new interpolated frame image signal, as in the conventional one. On the other hand, the motion adjustment buffer memory 22,
23 is a unidirectional vector detected by the motion vector detection circuit 24 for the sequentially scanned image signal once stored, that is, the direction of movement due to panning, tilting, etc. of the imaging camera and the frame interpolation timing. By reading the image signals at the memory addresses controlled by the control signals θ ₁ and θ ₂ , a motion-compensated interpolated image signal that compensates for the rounding of the signal waveform of the image is formed. It is assumed that the control signals θ ₁ and θ ₂ are generated for each first frame, that is, for each converted frame.

_In this case, the moving object moves in _the panning direction of the camera at the same speed as the panning speed. Since the moving object is moving in parallel with the speed, the relative speed to the camera is zero, so the moving object image becomes a still image area on the screen, and conversely, the moving object image becomes a static image area on the screen. becomes a moving image area on the screen. In the motion/motion vector detection circuit 24, control signals θ ₁ and θ ₂ are detected based on the inter-frame difference signal of the input sequential scanning image signal, and the control signals θ ₁ and θ ₂ are used to detect the motion adjustment buffer memory as described above. 22 and 23 are controlled to form a motion compensated interpolated image signal that compensates for blur and jerkiness due to image movement in the moving image area of the input sequentially scanned image signal. Next, when the motion compensated interpolation image signal and the linear interpolation image signal are weighted and combined, the variable coefficient unit 17 and A control signal θ ₃ is formed to control each of the signals 18 and 18 respectively.

Therefore, while supplying the linear interpolation image signal from the linear interpolation filter to the variable coefficient unit 17,
The motion compensation interpolation image signals from the motion adjustment buffer memories 22 and 23 are supplied to the variable coefficient unit 18, and the respective coefficients l and (1-l) of the variable coefficient units 17 and 18 are supplied from the motion/motion vector detection circuit 24. After complementary variable control using the control signal θ ₃ described above, the output image signals of these variable coefficient units are added and synthesized by the adder 20, and the frame number conversion output sequential scanning image signal of the M line/L frame method is obtained. For the pan imaging output image signal, a large-area background image that is a moving image area on the screen becomes a converted output image with a large motion compensation interpolation image signal component with less blur and jaggedness due to image movement. In addition, a moving object image that is a still image area on the screen becomes a converted output image signal with a large linear interpolation image signal component by linear interpolation, and as a result, the entire screen is As a result, a frame number-converted output image signal with extremely little image quality deterioration due to image movement can be obtained.

However, as mentioned above, in the above patent application, although it is sufficiently effective for images created by panning and tilting the camera to follow a specific subject, for example, when shooting images on the same screen as shown in Figure 2, In the case of a special image in which a car traveling left and right is conspicuously large, there is a drawback that the effect cannot be obtained sufficiently. This is because only one screen has a motion vector detection motion adjustment function, and the present invention aims to reduce image quality deterioration during frame conversion, including in the case of such special images.

FIG. 1 shows the configuration of an embodiment of the device of the present invention. Block diagram No. 10 of the embodiment of the invention of the above-mentioned Japanese Patent Application No. 59-244638
As is clear from the comparison between the figure and FIG. 1, which is a block diagram of an embodiment of the present invention, the gist of the present invention is
This invention is one step further than No. 244638. In FIG. 10, the movement adjustment buffer memory set 22, 23 is simply 1
However, in FIG. 1, there are a plurality of sets 6, 7 (an example of 4 sets is shown in FIG. 1). With this, it is possible to follow objects of two or more sizes on the same screen as shown in FIG.

Furthermore, in FIG. 10, the linear interpolation image signal (output of 19) and the motion compensation interpolation image signal (output of 21) are controlled by the control signal θ ₃ from the motion/motion vector detection circuit 24 before being mixed 20. While multiplied by controlled complementary variable coefficients l, 1-l,
In FIG. 1, first, a new frame of linearly interpolated image signals (formed by 1, 2, 3, 4, 5, 13, 13 in FIG. 10, 14, 15, 16, 19
, and this part is the same), and a plurality of N and N+1 frames of motion compensated image signals (outputs of 6 and 7, respectively) controlled by a plurality of motion vectors detected by the motion vector detection circuit 9. New frames of motion-compensated interpolated image signals (8 outputs each) are prepared, and N and N+1 frames of original image signals and multiple N and N+1 frames of motion-compensated image signals (6 and 7 outputs, respectively) are prepared. ) and (10 outputs each) are led to the minimum value label detection circuit 11, the absolute values of these interframe differences are compared to find the minimum value, and the minimum value of these absolute values is The signal selection circuit 12 selects either the linearly interpolated image signal for the linear interpolation image signal or the plurality of motion compensated interpolated image signals to obtain the converted output image signal (M lines/N frames are converted into M lines/L frames). Conversion to).

The above is an explanation of the main parts of the configuration of the device of the present invention.Next, an example of the contents of the motion vector detection circuit shown in the first diagram will be described, and it will correspond to a plurality of moving image parts that make noticeable movements on the screen. We will clarify the correct method for detecting motion vectors.

Motion Vector Detection Method As already explained, in the present invention, in order to form an interpolated image signal in which appropriate motion compensation has been performed for a plurality of different motion image parts included in the same image, the method shown in FIG. A motion vector detection circuit 24 is provided to detect motion vectors corresponding to the plurality of different motion image parts.

In order to detect the correct motion vector suitable for this case, the present invention particularly provides a plurality of motion vectors corresponding to the number of different motion image parts and the statistical positions on the screen of the image signal to be converted. In addition to dividing the image into regions, in detecting motion vectors from each divided region, a load detection method corresponding to the amount of motion vectors in the required motion image was applied.

Examples of this detection method will be explained below with reference to FIG. FIG. 3a shows an example of dividing an image into a plurality of regions, in which the image is divided horizontally and vertically (four divisions). In order to detect each motion vector in each divided area, a large number of motion vector detection small blocks are provided in each divided area as shown in FIG. 3b. There are about 4,000 small blocks in one entire screen, so there are about 1,000 in one divided area.

The size of each small block is determined by taking into account the maximum value of the motion vector to be detected, and the number of pixels included in that range is, for example, 4 lines vertically (up and down) and 20 samples horizontally (left and right), as shown in Figure 3c. selected according to degree. Therefore, the center pixel of the small block
For Pij, about 320 motion vectors Vn from this pixel to each sample point can be considered.

Required motion vector detection methods that utilize motion vector detection small blocks include the well-known matching method, and the details are omitted, but the gist is as follows. In N and N+1 frame signals of image signals, for example, after giving a movement of vector Vn to each small block of the N frame image, N+1
The difference from the frame signal is determined, and the vector that minimizes the sum of the differences in each small block is determined as a motion vector. This relationship can be expressed as follows.

Dvn= _n 〓 ^ij=1 dvnpij where DVn is the sum of each block of the difference between the N frame signal giving the movement of the vector Vn and the N+1 frame signal, and dvnpij is the sum of the difference between the N frame signal giving the vector Vn and N+1 in the small block Pij The difference from the frame signal, m, is the number of small blocks.

In the motion vector detection method expressed by this formula,
This method has the drawback that a motion vector is detected regardless of the size of a uniformly moving portion of an image and the amount of motion, and is therefore insufficient as a motion vector detection method applied to the present invention. The reason for this is that although the image area is divided, even the same divided image area may contain image parts with different movements, and most of the images within this area have no movement, but a small area that attracts attention This is because there may be a moving image portion.

Therefore, in motion vector detection applied to the present invention, from DVn to Vn corresponding to its minimum value.
Rather than directly finding the required load factor Wn
For the multiplied Wn×DVn, Vn corresponding to the minimum value is found.

Figures 3d to 3f show examples of this load factor. The case shown in Figure 3 d is an example of a load in which relatively large motion vectors in the horizontal direction are likely to be detected, while small motion vectors in the vertical direction are difficult to detect. This is applied when you want to detect a motion vector from a moving image part.

FIG. 3e shows an example of a load when a relatively large motion vector in the vertical direction is detected, and FIG. 3f shows an example of a load when a relatively large motion vector in the vertical and horizontal directions is detected.

Note that in any of cases d, e, and f in FIG. 3, the weighting coefficient for vectors in directions that are easy to detect is small, and the weighting coefficient for vectors in directions that are difficult to be detected is large.

Next, a preferred embodiment in which the present invention is applied to the portion from the minimum value label detection circuit 11 to the signal selection circuit 12 will be described.

Improving the image quality of motion-compensated interpolated image signals Up to now, from a linear interpolated image signal and multiple motion-compensated interpolated image signals, a weighting coefficient has been applied to the minimum absolute value of the inter-frame difference or the sum of the inter-frame difference signals. As described above, the required interpolated image signal is selected according to the minimum value obtained by multiplying by If this is selected, so-called bug-like image quality deterioration may occur at connection points between a linear interpolation image portion and a plurality of motion-compensated interpolation image portions, or between motion-compensation interpolation image portions.

FIG. 4 shows an example of a configuration for preventing this image quality deterioration. Minimum value label detection circuit 11 in Fig. 4
The signal selection circuit 12 and the signal selection circuit 12 have the same block with the same name in FIG. 1 which is an embodiment of the present invention, and in FIG.
1 is provided to prevent the above-mentioned bug-like image quality deterioration.

For example, as shown in FIG. 5, the area filter 31 is composed of a plurality of 1H memories 33, 1 sample memories 34, etc., and is a filter that forms a two-dimensional sample area. FIG. 6 shows an example of a two-dimensional sample area centered on the sample point and data stored in the area.
In this example, there is a sample area of 75 samples with 5 vertical samples and 15 horizontal samples centered around the sample point. Also, the stored data showed two cases with labels "0" and "1". These two types of labels may be considered to correspond to, for example, two types of motion vectors. In the case of the stored data in FIG. 6, with the configuration in FIG. 1, the interpolated image signal with the label "1", which is the data in the central hatched part (the sample point) in FIG. 6, is selected. In this configuration example, the number of "0" and "1" labels included in the sample area is checked and compared with a separately set threshold level in the comparator circuit 36 of FIG. A number of rals are output. In the sample area of Figure 6, the total number
75, so more than 38 labels will be output.

The same applies if there are three or more types of labels,
In that case, the maximum number of labels will be output and the corresponding interpolated image signal will be selected.

FIG. 7 shows another configuration example for preventing the bug-like image quality deterioration. For the input of the minimum value judgment circuit, the above-mentioned area filter is provided for each judgment output, and the maximum value among the output values of these filters is judged and used as a label for the sample point and an interpolated signal is selected. This is what we do.

(Effects of the Invention) As described above, if the device of the present invention is used, the number of frames of a television image signal can be converted even if there are a plurality of image parts that are relatively easily visible and different. It is possible to reduce image quality deterioration such as blurring and jaggedness that occurs.

Further, by using an unprecedented weighting coefficient method in the motion vector detection method of the apparatus of the present invention, movement in a direction that is easily seen can be emphasized and detected in advance, so that better frame number conversion is possible.

Furthermore, by performing signal processing by installing an area filter between the minimum value label detection circuit and the signal selection circuit, the so-called bug-like image quality deterioration at the boundary between different parts of the interpolated image signal can be removed and improved. Frame number conversion can be made possible.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the configuration of an embodiment of the device of the present invention;
Fig. 2 is an illustration for explaining one feature of the present invention. Fig. 2a is an N frame image (original image),
Figure b is the N+1 frame image (original image), Figure 2 c
is a linear interpolation image, FIG. 2d is a motion-compensated interpolation image for car A, FIG. 2e is a motion-compensated interpolation image for car B, and FIG. 3 explains the motion vector detection method of the present invention. Figure 3a shows an example of image division, Figure 3b shows the division of one image division area into small blocks, and Figure 3c shows the division of one image division area into small blocks.
3d, e, and f are diagrams each showing an example of a load coefficient; FIGS. 4, 5, 6, and 7 are diagrams for explaining the function of the area filter of the device of the present invention; and FIG. 8 is a diagram showing a small block. , 9 and 10 are diagrams for explaining a conventional conversion device. 1, 13...Frame order circuit, 2, 14...
Frame memory, 3, 4, 15, 16, 17, 1
8... Coefficient accumulator, 5, 8, 19, 20, 21...
...Adder, 6, 7, 22, 23...Motion adjustment buffer memory, 9...Motion vector detection circuit, 10
... Absolute value circuit (ABS), 11 ... Minimum value label detection circuit, 12 ... Signal selection circuit, 24 ... Motion/motion vector detection circuit, 31 ... Area filter, 32 ... Maximum value judgment circuit, 33 ... 1H memory, 34 ... 1 sample memory, 35 ... total sum addition, 34 ... comparison circuit, 41 ... interpolation filter, 42 ... thinning circuit.

Claims (1)

[Scope of Claims] 1. In converting the number of frames of a television system, a linearly interpolated image signal of a new frame is formed by linear interpolation from the original image signals of consecutive N and N+1 frames, and The original image is divided into a plurality of regions, a motion vector is detected according to the direction and distance of movement of the image in each region, and the above-mentioned N and N are detected according to these vectors.
forming a plurality of N and N+1 frames of motion compensated image signals by respectively shifting the entire +1 frame of the original image signal to form a plurality of new frames of motion compensated interpolated image signals; The original image signal and the plurality of N and N
Select either the linearly interpolated image signal or the plurality of motion-compensated interpolated image signals corresponding to the minimum value of the absolute values from the absolute values of the inter-frame differences between the +1 frame motion-compensated image signal and the motion-compensated image signal of +1 frame. A system conversion device characterized in that a converted output image signal is formed by performing the following steps. 2. When determining the minimum value of those absolute values,
N frame signal and N that supported the movement of vector Vn
2. The system conversion device according to claim 1, wherein the minimum value is determined by multiplying the sum of each small block of the difference from the +1 frame signal by a desired coefficient. 3. When selecting one of the above, an area filter is provided between the minimum value label detection circuit and the signal selection circuit to perform signal processing, and the boundary between different regions of a new frame in which different interpolated image signals are output is detected. 2. The system conversion device according to claim 1, wherein the system conversion device prevents a bug-like image quality deterioration.