JP3480011B2 - Image information conversion device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information conversion device suitable for use in, for example, a television receiver, a video tape recorder device, or the like, and particularly to a standard resolution image information supplied from the outside in a high resolution image. The present invention relates to an image information conversion device that converts and outputs information.

[0002]

2. Description of the Related Art Nowadays, due to the increase in audio / visual orientation, it is desired to develop a television receiver capable of obtaining a higher resolution image than a standard resolution. In response to this demand, a high definition system is developed. Was done. This high-definition system has more than double the number of scanning lines defined by the NTSC system, which is 525.
It has five investigation lines. In addition, the aspect ratio of the display screen is 3: 4 in the NTSC system, whereas the HDTV system is
It has a wide-angle screen of 9:16. Therefore, with the high-definition method, it is possible to obtain a screen with high resolution and a realistic feeling.

Although the HDTV system has the above-described excellent characteristics as compared with the NTSC system, a generally supplied NTSC system video signal is displayed on a high-definition television receiver. I can't. This is because the standards of the NTSC system and the high-definition system are different as described above. Therefore, NT
Conventionally, when an image corresponding to an SC video signal is displayed on a high-definition television receiver,
For example, a video signal rate conversion is performed using an image information conversion device as shown in FIG.

In FIG. 6, a conventional image information conversion apparatus includes a horizontal interpolation filter 101 for performing horizontal interpolation processing on an NTSC video signal supplied through an input terminal 100, and a horizontal interpolation processing. Of NTS
A vertical interpolation filter 102 that performs vertical direction interpolation processing on a C-system video signal.

Specifically, the horizontal interpolation filter 101 is
It has a structure as shown in FIG. The NTSC video signal supplied via the input terminal 100 is supplied to the first to mth multipliers 111 to 111m via the input terminal 110, respectively. Each multiplier 111 multiplies the video signal by a coefficient and outputs the product. The video signals multiplied by the coefficient are supplied to the first to mth adders 112 to 112m−1, respectively. Delay registers 113 to 113m for time T are provided between the adders 112 to 112m-1, respectively.

Here, the video signal output from the m-th multiplier 111m is delayed by the time T by the m-th delay register 113m and supplied to the (m-1) th adder 112m-1. In the m-1th adder 112m-1, the mth adder 112m-1
The video signal delayed by the time T from the delay register 113m and the video signal multiplied by the coefficient from the (m-1) th multiplier 111m-1 are added and output. The added video signal is the m-1th delay register 113m-
The video signal delayed by the time T again by 1 is added to the video signal multiplied by the coefficient from the (m-2) th multiplier 111m-2 (not shown). Horizontal interpolation filter 101
In this way, after the horizontal interpolation processing for the NTSC video signal is completed, the video signal is supplied to the vertical interpolation filter 102 via the output terminal 120.

The vertical interpolation filter 102 has the same structure as the horizontal interpolation filter 101, and the video signal subjected to the horizontal interpolation processing is subjected to vertical pixel interpolation. That is, the vertical interpolation filter interpolates the pixels in the vertical direction with respect to the NTSC video signal subjected to the horizontal interpolation. The video signal subjected to the horizontal interpolation and the vertical interpolation in this way is supplied to a high-definition television receiver. As a result, an image corresponding to the NTSC video signal can be displayed on the high-definition television receiver.

[0008]

However, the conventional image information conversion apparatus merely interpolates pixels in the horizontal and vertical directions based on the video signal of the NTSC system, and therefore the resolution is It was no different from the underlying NTSC video signal. Especially when converting normal moving images from the NTSC system to the high definition system,
In general, interpolation in the vertical direction is performed by intra-field processing, and since the correlation between the fields of the image is not used, there is a problem that the resolution is deteriorated in the still image portion as compared with the video signal of the NTSC system. It was

On the other hand, the method described in the image signal conversion apparatus of Japanese Patent Application No. 5-172617 is divided into classes according to the three-dimensional (spatiotemporal) distribution of the input standard resolution image signal level. Then, each class has a memory that stores a prediction coefficient value acquired by learning in advance, and outputs an optimum estimated value by a calculation based on a prediction formula.

This method utilizes the correlation in the time direction such as between fields and between frames. However, there is a problem in that the number of classes becomes large because it is necessary to divide the image data having a three-dimensional structure into classes and to express two patterns of motion and spatial level distribution by the class division. .

Therefore, an object of the present invention is to solve the above-mentioned problems, and to improve the resolution of a standard resolution image signal, for example, an NTSC system video signal, the NTSC system image signal is improved. An object is to provide an image information conversion device capable of converting a high-resolution image signal, for example, a high-definition video signal.

[0012]

According to the present invention, there is provided a first image information converting apparatus for converting a digital image signal into a digital image signal having a larger number of pixels, the interlace structure being supplied from the outside. By applying motion compensation to image information ,
There line non interlaced combined pixel position, a plurality off
Create a non-interlaced image containing the image for the field
The non-interlacing means, the image information dividing means for dividing the non-interlaced image created by the non-interlacing means into a plurality of blocks, and the characteristics of the blocks divided by the image information dividing means. Class detecting means for determining a class according to which the class information is output, and coefficient data storing means for outputting coefficient data according to the class information output from the class detecting means;
Image information conversion means for converting the input first image information into second image information having a resolution higher than that of the first image information and outputting the second image information according to the coefficient data supplied from the coefficient data storage means. An image information conversion apparatus having:

[0013]

The image information conversion apparatus according to the present invention first converts a normal image having an interlaced structure into an image having a non-interlaced structure by using means such as motion vector detection and motion compensation. Then, the non-interlaced image is divided into a plurality of areas by the image information dividing means. The divided image information detects a pattern of the level distribution of the image information for each area, determines the class to which the image information of the area belongs based on the detected pattern, and outputs the class information. This class information includes
Coefficient data of a linear estimation formula for converting image information supplied from the outside into image information having a resolution higher than this image information is stored for each class. Then, the image information conversion means uses the coefficient data supplied from the coefficient data storage means to convert the image information supplied from the outside into image information having a higher resolution than that.

[0014]

Embodiments of the image information conversion apparatus according to the present invention will be described below in detail with reference to the drawings. Figure 1
Shows a schematic configuration of the signal processing of this embodiment, that is, the image information conversion apparatus. An SD (StanderdDifi), in which a so-called NTSC video signal is digitized as image information supplied from the outside, is input to the input terminal 1
nition) data is supplied.

The SD data supplied from the input terminal 1 is
It is supplied to the motion vector detection circuit 2. The motion vector detection circuit 2 has a field memory and holds image data of one field before. If the current field to be processed is k field, the motion vector detection circuit 2 determines the effective area of the image data of the (k-1) field as (3 × 3) pixels as shown in FIG. Divide into small areas of size. In FIG. 3, a line of the (k-1) field shown by a solid line and a line of the k field shown by a dotted line are shown. In FIG. 3, (k-1)
Regarding the field, a block of (3 × 3) pixels is surrounded by a square.

The motion vector detection circuit 2 calculates a motion vector between (k-1) field and k field for each block by the block matching method for the divided image data. In addition, the motion vector detection circuit 2
Applies vertical interpolation filtering to each of k field data and (k-1) field data to double the number of pixels in the vertical direction, and then calculates a motion vector between fields by the block matching method. May be.
The motion vector detection circuit 2 outputs k field image signal data, (k−1) field image signal data, and motion vector data for each block to the motion compensation circuit 3.

The motion compensation circuit 3 shifts the image signal data of the (k-1) field in accordance with the motion vector from the motion vector detection circuit 2, and a non-interlaced image composed of the shifted image signal and the k field image signal. Race S
Create a D image. An example of this is shown in FIG. FIG. 4 shows (k
-1) It is a figure showing a part of the image of the field and the k field in the time direction and the vertical direction. Black circles represent pixel data of (k-1) field, and white circles represent pixel data of k field. Moreover, a large circle represents SD data, and a small circle represents HD (High Definition) data. Arrows in FIG. 4 represent vertical components of the detected motion vector. The two arrows on the upper side indicate that the pixel in this block is (k-
1) The field and the k field are vertically shifted to the same position. The two arrows on the lower side show that the vertical component of the motion vector is `1`, so that the pixels in this block are shifted up one line from the (K-1) field.

Conventionally, the up-conversion of moving images is
Up-conversion was performed by performing interpolation processing on the interlaced image. Therefore, for example, in class classification of spatiotemporal structure, two patterns of motion and spatial level distribution are expressed, so that there is a problem that the number of necessary classes becomes enormous. In the present invention, a non-interlaced SD image is created by introducing the motion vector detection circuit 2 and the motion compensation circuit 3 before class classification. That is, in the k field of interest, a still image having a non-interlaced structure is generated. This allows
The process of performing up-conversion on a non-interlaced SD image is equivalent to the process of up-converting a still image having a non-interlaced structure. The motion compensation circuit 3 supplies the created non-interlaced SD image in the k field to the area division circuit 4 as an output signal.

The area division circuit 4 divides the non-interlaced SD image supplied from the motion compensation circuit 3 into a plurality of areas. In this embodiment, the non-interlaced SD image is divided into blocks each having a total of 10 pixels of 1 pixel × 10 lines as shown in FIG. The SD data for each block is sent from the area dividing circuit 4 to the ADRC (Ad
It is supplied to the aptive dynamic range cording circuit 5 and the delay circuit 8. S supplied from the area dividing circuit 4
The delay circuit 8 delays the SD data by the time for transmitting the D data to the estimation calculation circuit 9 via the ADRC circuit 5, the class code generation circuit 6 and the ROM table 7.

The ADRC circuit 5 is provided for class detection, and the ADRC circuit detects a pattern of level distribution of SD data detected for each block. The pattern of the level distribution of all the data of each block or a part of the data is detected. The ADRC circuit 5 forms pattern compressed data by performing an operation such as compressing 8-bit SD data to 2-bit SD data, and supplies the pattern compressed data to the class code generation circuit 6. Originally, ADRC is a digital V
This is an adaptive requantization method developed for high-efficiency coding for TR. However, since the local pattern of the signal level can be efficiently expressed with a short word length, the embodiment of the present invention uses Used for code generation. ADRC circuit 5
Is a dynamic range within the region, the number of allocated bits is n, the data level of the pixels within the region is L, and the requantization code is Q.
The area between X and the minimum value MIN is equally divided by the designated bit length and requantization is performed.

DR = MAX-MIN + 1 Q = [(L-MIN + 0.5) × 2 ⁿ / DR] (1) However, [] means a truncation process.

In this embodiment, an SD of 4 pixels, for example, is located at the center of each block of 10 pixels separated from the non-interlaced SD image by the area dividing circuit 4.
Data (x ₄ ~x ₇ in FIG. 5) is to be compressed into 2 bits each. The SD data compressed to 2 bits is quantized data q _{1 to} q ₄ .

The class code generation circuit 6 is based on the pattern compression data supplied from the ADRC circuit 5 and outputs (2)
The class to which the block belongs is detected by performing the operation of the expression, and the class code class indicating the class is R
Supply to the OM table 7. This class code class
Indicates a read address for reading coefficient data from the ROM table 7. Also, in this example,
In the equation (2), n and p are both '2'.

[0024]

[Equation 1]

The ROM table 7 stores the coefficient data obtained by learning the relationship between the SD data pattern and the corresponding HD data for each class. For this coefficient data and SD data, HD data corresponding to the pattern of SD data is calculated using a linear estimation formula. That is, the coefficient data is information for converting the SD data into HD data that is image information having a resolution higher than the normal resolution, that is, HD data that conforms to the so-called high-definition standard by using the linear estimation formula. The method of creating the coefficient data stored in the ROM table 7 will be described later.

In the ROM table 7, the coefficient data w _i corresponding to the class is read from the address indicated by the class code class supplied from the class code generation circuit 6. The read coefficient data w _i is supplied to the estimation calculation circuit 9.

By inputting the calculation shown in the equation (3) on the basis of the SD data supplied from the area dividing circuit 4 via the delay circuit 8 and the coefficient data w _i supplied from the ROM table 7. The estimation calculation circuit 9 calculates HD data (hd ′) corresponding to the SD data thus obtained.
The calculated HD data is output to the horizontal interpolation filter 10.

[0028]   hd '= w₁x₁+ W₂x₂+ W₃x₃+ W_Fourx_Four+ W_Fivex_Five+ W₆x₆+ W ₇ x₇+ W₈x₈+ W₉x₉+ W_Tenx_Ten              (3)

The horizontal interpolation filter 10 has the same configuration as the horizontal interpolation filter 101 of FIG. 6, and doubles the number of pixels in the horizontal direction by performing an interpolation process.
The output of the horizontal interpolation filter 10 is output via the output terminal 11. HD output via this output terminal 11
The data is supplied to, for example, a high-definition television receiver or a video tape recorder device.

In the above-mentioned estimation operation circuit 9, the coefficient data for estimating the HD data corresponding to the SD data is obtained in advance by learning for each class, and then the ROM table 7
The calculation is performed based on the input SD data and the coefficient data read from the ROM table 7. This calculation is the H corresponding to the input SD data.
By inputting S by forming and outputting D data
Unlike the process of simply interpolating D data, data closer to the actual HD data can be output.

Moreover, the above-mentioned Japanese Patent Application No. 5-172617.
Comparing with the method described in No. 1 to express two patterns of motion and spatial level distribution by classifying spatio-temporal structure, in the present invention, motion compensation is applied to all input images. The difference is that the pattern of the level distribution only in the space is expressed by converting the input image of 1 to a still image and then classifying it. Therefore, in the present invention, since it is not necessary to include motion in the expression, the number of classes can be significantly reduced. continue,
A method of creating coefficient data stored in the ROM table 7 will be described with reference to FIG.

To obtain coefficient data by learning,
First, an SD image corresponding to an already known HD image and having a pixel number ¼ of that of the HD image is generated. Specifically, the pixels in the vertical direction of the HD data supplied via the input terminal 21 are thinned by the vertical thinning filter 22 so that the frequency in the vertical direction in the field becomes 1/2.
Further, the horizontal thinning filter 23 thins out pixels in the horizontal direction of the HD data to generate SD data.

S obtained by the horizontal thinning filter 23
The D data is supplied to the motion vector detection circuit 24.
The motion vector detection circuit 24 is for calculating a motion vector by the block matching method as described above.

The output signal of the motion vector detection circuit 24 is
It is supplied to the motion compensation circuit 25. The motion compensation circuit 25
The configuration is exactly the same as that of the motion compensation circuit 3 described above. The motion compensation circuit 25 performs motion compensation based on the output signal of the motion vector detection circuit 24, so that the non-interlaced SD image in the processing target field is supplied from the motion compensation circuit 25 to the area division circuit 26. .

In this embodiment, the learning target image is
Although the moving image is used, the learning target image may be limited to a still image. In that case, the motion vector detection circuit 2
4 and the motion compensation circuit 25 are unnecessary, and a deinterlacing circuit is required instead. The non-interlacing circuit required here can be realized by simply combining SD data of two consecutive fields, so that the circuit for learning is greatly simplified by using this non-interlacing circuit. Be converted.

The area division circuit 26 is a motion compensation circuit 25.
The non-interlaced SD image supplied from the is divided into a plurality of blocks. The area dividing circuit 26 operates similarly to the area dividing circuit 4 described above. In this embodiment, as in the area dividing circuit 4, for example, as shown in FIG. 5, a non-interlaced SD image is divided into blocks each including 1 pixel × 10 lines and a total of 10 pixels, and the SD data for each block is Area division circuit 26 to ADR
It is supplied to the C circuit 27 and the normal equation circuit 29.

The ADRC circuit 27 detects the pattern of the one-dimensional level distribution of the SD data supplied for each block, and as described above, all or a part of the data in each area is, for example, 8 Pattern compressed data is formed by performing an operation such as compressing the bit SD data into the 2-bit SD data, and the pattern compressed data is supplied to the class code generation circuit 28. The ADRC circuit 27 has the same configuration as the ADRC circuit 5 described above. In this embodiment, the area dividing circuit 26
10 separated from non-interlaced SD image by
SD of 4 pixels located at the center of each block of pixels
Data (x ₄ ~x ₇ in FIG. 5) are intended to be compressed into 2 bits each by the ADRC.

The class code generating circuit 28 has the same configuration as the class code generating circuit 6 described above, and by performing the operation of the equation (2) on the basis of the pattern compressed data supplied from the ADRC circuit 27, its block Finds the class to which the class belongs and indicates the class
It outputs s. Class code generation circuit 28
Outputs the class code class to the normal equation adding circuit 29.

Here, in order to explain the normal equation adding circuit 29, learning of a conversion formula from a plurality of SD pixels to HD pixels and signal conversion using the prediction formula will be described. Hereinafter, for the sake of explanation, a case will be described in which pixels are generalized and prediction is performed using n pixels. Level x ₁ of the SD pixel, ‥‥, x _n is the re-quantized data of results in each was p bit ADRC q _1, ‥‥, and q _n. At this time, the class code class of this area is defined by the equation (2).

As described above, the SD pixel level is set to x
_1, ‥‥, as x _n, when the HD pixel level was y, the coefficient for each class, w _1, ‥‥, linear estimation equation of n tap according to w _n is set. This is shown in equation (4). Before learning, the coefficient data w _i is an undetermined coefficient.

Y = w ₁ x ₁ + w ₂ x ₂ + ... + w _n x _n (4)

Learning is performed on a plurality of signal data for each class. When the number of data is m, the following expression (5) is set according to the expression (4).

Y _j = w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (j = 1, 2, ..., _M ) (5)

In the case of m> n, w ₁ , ..., W _n are not uniquely determined. Therefore, the elements of the error vector E are defined by the equation (6), and the coefficient by which the equation (7) can be minimized. Ask. This is a so-called least squares method.

E _j = y _j − (w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn ) (j = 1, 2, ..., m) (6)

[0046]

[Equation 2]

Here, the partial differential coefficient based on the coefficient data w _i of the equation (7) is obtained. The following expression (8) is set to `0 '
Each coefficient data w _i may be calculated as follows.

[0048]

[Equation 3]

When X _ij Y _i is defined as in equations (9) and (10) below, equation (8) is rewritten into equation (11) using a matrix.

[0050]

[Equation 4]

[0051]

[Equation 5]

This equation is generally called a normal equation. The normal equation adding circuit 29 has a class code class supplied from the class code generating circuit 28, SD data supplied from the area dividing circuit 26, x ₁ , ..., X.
_n , using the HD pixel level y corresponding to the SD data supplied from the input terminal 21, an addition operation is performed in this normal equation.

After inputting all the learning data,
The normal equation addition circuit 29 outputs the normal equation data to the prediction coefficient determination circuit 30. The prediction coefficient determination circuit 30
A normal matrix solving method such as a sweeping method is used to solve the coefficient data w _i and calculate the prediction coefficient. The prediction coefficient determination circuit 30 writes the calculated prediction coefficient in the memory 31.

As a result of learning as described above, the memory 3
1 stores a prediction coefficient for estimating the HD data of interest y for each pattern defined by the quantized data q _{1 to} q ₄ , which is statistically closest to the true value. The table stored in this memory 31 is, as described above,
ROM used in the image information conversion apparatus of the present invention
Table 7. Through the above processing, learning of coefficient data for creating HD data from SD data is completed by the linear estimation formula.

In the description of the embodiment, for the sake of simplicity, the horizontal interpolation filter 9 is used for the horizontal interpolation, but instead, the above vertical interpolation processing is used in the horizontal direction, that is, the horizontal direction. It is also possible to perform block formation on the block, and use coefficient data and peripheral SD pixels to perform horizontal interpolation by a linear estimation formula.

Here, in the description of the embodiment, the pattern of the signal waveform is expressed in one dimension by the area dividing circuit 4, but the pattern of the signal waveform is expressed in the two dimension. May be.

Further, in the description of the embodiment, the inter-field vector is calculated by the motion vector detection circuit to perform the motion compensation. However, this is not limited to the inter-field, but the vector at a constant interval such as inter-frame. Any calculation can be selected as appropriate.

Further, in the description of the embodiment, the values stored in the ROM table are the coefficient data, but by storing the representative value by the center of gravity method, the HD data may be interpolated without performing the estimation calculation. It is possible.

[0059]

According to the present invention, an input interlaced structure moving image is converted from an interlaced structure moving image into a non-interlaced structure still image by performing motion compensation processing, and then classified in a space by class classification. Only the level distribution pattern is represented. Therefore, since it is not necessary to include motion in the expression, the number of classes can be significantly reduced compared to the conventional method, and the algorithm related to the operation can be easily
It is Ru can be. In addition, the present invention is based on time-based learning.
Perform processing to increase resolution by using coefficients that are not used.
Even using the coefficient learned in the time direction, the resolution
The effect is almost the same as when the processing to increase is performed.
You can

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of an image information conversion apparatus according to the present invention.

FIG. 2 is a block diagram for explaining when creating a ROM table of the image information conversion apparatus according to the present invention.

FIG. 3 is a schematic diagram for explaining a motion vector detection circuit.

FIG. 4 is a schematic diagram for explaining a motion compensation circuit.

FIG. 5 is a schematic diagram for explaining an area dividing circuit.

FIG. 6 is a block diagram of an embodiment of a conventional image information conversion device.

FIG. 7 is a block diagram of an example of a filter according to a conventional image information conversion device.

[Explanation of symbols]

2 Motion vector detection circuit 3 Motion compensation circuit 4 area division circuit 5 ADRC circuit 6 class code generation circuit 7 ROM table 8 delay circuits 9 Estimating arithmetic circuit 10 Horizontal interpolation filter

[Equation 1]

[Equation 1]

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 00-7/088

Claims (4)

(57) [Claims] 1. An image information conversion device for converting a digital image signal into a digital image signal having a larger number of pixels, wherein motion compensation is performed on first image information having an interlaced structure supplied from the outside. of a plurality of fields by the applied
There line non interlaced combined pixel position of the image,
Non-interlaced images including images for multiple fields
Non-interlacing means to be created , and the non-interlacing means created by the non-interlacing means
Image information dividing means for dividing the interlaced image into a plurality of blocks, and class detecting means for determining a class according to the characteristics of the blocks divided by the image information dividing means and outputting class information. A coefficient data storage unit for outputting coefficient data in accordance with the class information output from the class detection unit; and the input first data corresponding to the coefficient data supplied from the coefficient data storage unit. An image information conversion device comprising image information conversion means for converting one image information into second image information having a resolution higher than that of the first image information and outputting the second image information. 2. The coefficient data is a feature in the spatial direction of the image.
Characteristic that the data is pre-learned based on sex
The image information conversion device according to claim 1. 3. An image information conversion device for converting a digital image signal into a digital image signal having a larger number of pixels, wherein high-resolution second image information for learning supplied from the outside is converted into a standard resolution image. Information conversion means for converting the first image information, and a plurality of frames by performing motion compensation of the first image information.
There line non interlaced combined pixel position of the image field, Non'intare including images of a plurality of field of
To create over scan images, and non-interlaced means, the non-created by the non-interlace means
An image information dividing unit for dividing an interlaced image into a plurality of blocks, and a class to which the image information according to the characteristics of the blocks divided by the image information dividing unit belongs and to output the class information. The class detection means and the predetermined value from a plurality of pixel values included in the first image information and spatially adjacent to a predetermined pixel in the second image information, and a plurality of coefficient data. A coefficient determining means for determining the plurality of coefficient data for each class so as to minimize the error between the created value and the true value of the predetermined pixel when the pixel value is created, An image information conversion apparatus comprising: storage means for storing the plurality of coefficient data together with the class information. 4. The image information conversion apparatus according to claim 3 , wherein the coefficient determining means uses a least squares method.