JP3321915B2 - Image signal converter - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an input standard resolution (hereinafter, referred to as SD) when performing image signal conversion.
The present invention relates to a signal converter for up-converting a signal into a high-resolution (hereinafter, referred to as HD) signal.

[0002]

2. Description of the Related Art FIG. 12 is a block diagram showing an example of a conventional signal converter for performing up-conversion. The SD signal input from the input terminal 60 is converted into a horizontal interpolation filter 61.
, The number of pixels in the horizontal direction is doubled, the number of lines in the vertical direction is doubled by the vertical interpolation filter 62, and output from the output terminal 63 as an HD signal. That is, image up-conversion is performed using a filter.

FIG. 13 shows a configuration example of the interpolation filter. The signal supplied from the input terminal 64 is multiplied by a multiplier.
Multiplied by the filter coefficients α _n , α _n−1 , ‥‥ α ₀ , sequentially delayed and added by the register of the unit delay amount T, and the interpolation output is output from the output terminal 65. In the horizontal interpolation filter 61, this unit delay amount T is selected as the sample period, and in the vertical interpolation filter 62, this is selected as the line period.

[0004]

In the above-described conventional image signal converter, when up-converting from an SD signal to an HD signal using a filter, the output signal is merely an interpolated signal, and the resolution is high. It is no different from the input SD signal. In particular, for an interlaced image, high-precision conversion is difficult because the line interval in the same field is large.

Conventionally, a signal conversion device that performs up-conversion performs intra-field processing when an image to be converted is a moving image, and performs inter-field processing when a signal to be converted is a still image. Was. That is, the processing method is switched depending on whether the image to be converted is a moving image or a still image. Therefore, a separate motion detection circuit is required, and if the motion detection is not accurate, the image quality is likely to deteriorate.

Accordingly, an object of the present invention is to perform a conversion by spatial processing using time direction information instead of simply interpolating. Therefore, there is motion by learning from a known plurality of fields of HD signals. It is an object of the present invention to provide an image signal conversion device that enables highly accurate conversion even at times.

[0007]

In an image signal conversion apparatus for converting a standard resolution image signal into a high resolution image signal, class division is performed according to the shape of the three-dimensional distribution of the level of the standard resolution input image signal. Means, storage means for storing prediction coefficient values obtained by learning in advance for each class obtained by class division, and an optimum estimation value output from an operation based on a prediction formula including the prediction coefficient value, which is combined with the storage means An image signal conversion device comprising means for outputting an image signal having a higher resolution than an input image signal by including an estimated value.

[0008]

Since the HD signal corresponding to the SD signal is determined by learning using the time direction information, up-conversion based on the characteristics of the actual image can be performed. Also, since the class is adaptively selected according to the three-dimensional level distribution of the SD signal, up-conversion that follows the local properties of the image becomes possible. Therefore, compared to interpolation by a filter,
An HD signal with higher resolution and image quality can be obtained. Also, good signal conversion can be performed for a motion image.

[0009]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration at the time of learning according to an embodiment of the present invention. An input terminal 1 receives a number of standard HD signal still images and a vertical thinning filter 2.
Is supplied to the learning unit 4. The vertical thinning filter 2 to which the HD image is supplied from the input terminal 1 thins the HD image in the vertical direction by １ ／, and halves the horizontal direction by the horizontal thinning filter 3 connected to the vertical thinning filter 2. The thinning-out is performed to obtain a still image having the same pixels as the SD signal.
To supply. The prediction coefficient memory 5 stores the class code created by the learning unit 4 and the coefficients w1 to wn.

When performing the above-mentioned thinning-out, a 1/2 thinning-out circuit including a normal frequency limiting filter can be used for horizontal thinning-out, but as shown in FIG.
Asymmetric coefficients such as 8, 4/8, 3/8 must be used in reverse order for each field to preserve the interlace structure of the SD signal. That is, if two lines of the HD image are simply synthesized to form one line of the SD image, the distance between the line of the n field and the line of the (n + 1) field becomes non-uniform, and the interlaced structure is lost.

FIG. 3 is a flowchart showing the operation of the learning section 4 when the learning section 4 has a software processing configuration. Control of the learning unit is started from step 11, and in the corresponding data block conversion of step 12, the HD signal and the SD signal are supplied, and a process of extracting HD pixels and SD pixels having an arrangement relationship as described later is performed. Step 13
At the end of the data, if the processing of all the input data, for example, the data of one frame has been completed, the control is shifted to the determination of the prediction coefficient in step 16, and if not, the control is shifted to the determination of the class in step 14.

In the class determination in step 14, a class is determined from the signal pattern of the SD signal. In this control, adaptive dynamic range coding (hereinafter, referred to as ADRC) as described below can be used to reduce the number of bits. In the normal equation addition in step 15, equations 7, 8 and 9 described later are created.

After the processing of all the data is completed from the end of the data in step 13, the control moves to step 16 and
In the prediction coefficient determination of No. 6, a prediction coefficient is determined by solving Expression 9 described later using a matrix solution method. The prediction coefficients are stored in the memory in the prediction coefficient storage in step 17,
Then, the control of the learning unit ends.

Classification is performed, for example, in a 12-dimensional distribution.
Number of SD pixels are used. One H for interpolation
The pattern of the 12 SD pixels according to the position of the D pixel is
There can be four types. These four types will be referred to as mode 1 to mode 4. The arrangement of pixels in each mode is shown in FIGS. 4, 5, 6, and 7, respectively. A circle represents an HD pixel to be interpolated, and a square represents an SD pixel generated by thinning out an HD image as described above.

Hereinafter, based on FIGS. 4, 5, 6, and 7, learning of a conversion formula from 12 SD pixels to HD pixels and signal conversion using the prediction formula will be described. In addition,
It is assumed that HD pixels in the figure have been obtained by calculation from 12 SD pixels based on the prediction formula of the present invention. In order to simplify the hardware, in FIGS. 5 and 7, a half of the SD pixels of the (n-1) field and the (n + 1) field are averaged by 1/2, and the SD field is converted to the n field.
Pixels have been acquired (dotted squares). 4, 5, and 6
7 and FIG. 7 show the spatial correspondence between SD pixels and HD pixels at the time of learning and prediction, and show only a plurality of necessary SD pixels and one HD pixel to be interpolated.

In FIG. 4, in mode 1, six SD pixels of n fields and three SD pixels of (n-1) field and (n + 1) field, respectively, a total of 1
An HD pixel is generated from two SD pixels. This mode 1
Is used to interpolate the HD pixels on the line multiplied by a factor of 1/8, using the vertical thinning diagram in FIG.

In mode 2 (FIG. 5), three SD pixels required in n fields are averaged by averaging SD pixels included in lines at the same position in (n-1) field and (n + 1) field. Interpolate. Then, six SD pixels of n fields including the interpolated SD pixels,
HD pixels are obtained from a total of 12 SD pixels, each of which is 3 SD pixels excluding the SD pixels used for averaging in the (n-1) field and the (n + 1) field. this is,
Using the vertical decimation diagram of FIG. 2, HD pixels of a line multiplied by a 4/8 coefficient are interpolated.

In mode 3 (FIG. 6), six SD pixels of n fields and four S pixels of (n-1) fields are used.
HD pixels are generated from D pixels and two SD pixels in the (n + 1) field, that is, a total of 12 SD pixels. this is,
Using the vertical thinning diagram in FIG. 2, HD pixels are interpolated in the horizontal direction of the HD pixels on the line multiplied by a factor of 1/8.

In the mode 4 (FIG. 7), the SD pixels of the (n-1) field and the (n + 1) field are averaged to obtain four pixels required in the n fields (2 pixels each for 2 lines).
) Of the SD pixels. And the interpolated SD
8 SD pixels of n fields including pixels, and (n−
1) Two SD pixels each except the SD pixel used for averaging in the field and the (n + 1) field, for a total of 1
An HD pixel is obtained from two SD pixels. This is shown in FIG.
In the vertical thinning diagram, the HD pixels are interpolated in the horizontal direction of the HD pixels on the line multiplied by a 4/8 coefficient.

As described above, HD pixels are obtained vertically from 12 three-dimensional SD pixels. In the modes 2 and 4, the HD pixels are generated on the HD pixel line at half the sampling interval of the SD pixels in the figure with respect to the HD pixels in the modes 1 and 3. That is, this is a process for generating twice as many HD pixels as SD pixels on the HD pixel line.

The learning unit 4 controls the mode 1 as described above.
The prediction coefficient of each of the modes 4 to 4 is determined.
Next, the class division will be described. The simplest method of the class division is a method in which a bit sequence of learning data in a block is used as a class number as it is. However, this method requires a huge amount of memory.

In this example, ADRC is used for class division by the signal pattern of the SD signal. Originally ADRC
Is an adaptive requantization method developed for high-efficiency coding for VTRs, and can efficiently represent a local pattern of a signal level with a short word length. When the SD pixel block is used, the levels of the SD pixels are x1 to xn, respectively. Also, re-quantized data obtained by performing p-bit ADRC on the data of x1 to xn are q1 to qn, the dynamic range is DR, the maximum value is MAX, and the minimum value is MIN. At this time, the class of this block is
Defined by Equation 1.

[0023]

(Equation 1)

Here, taking 1-bit ADRC as an example, A
The DRC will be described. FIG. 8 shows an example of the ADRC encoding circuit 43. In FIG. 8, the detection circuit 22 detects the maximum value MAX and the minimum value MIN for each block of the data converted into the order of the blocks from the input terminal 21. MAX and MIN are supplied to the subtraction circuit 23, and a dynamic range DR is generated at the output. The input data and MIN are supplied to the subtraction circuit 24, and the subtraction circuit 2
By removing the minimum value from 4, the normalized pixel data is generated.

The dynamic range DR is supplied to a division circuit 25, the normalized pixel data is divided by the dynamic range DR, and the output data of the division circuit 25 is supplied to a comparison circuit 26. In the comparison circuit 26, it is determined whether the dividing power of the eight pixels other than the center pixel is larger or smaller based on 0.5. Depending on this result,
Data DT of '0' or '1' is generated. The comparison output DT is taken out to the output terminal 27.

The processing for obtaining a coefficient for identifying the relationship between the HD pixel and the SD pixel in FIG. 2 will be described in more detail. Generally, when the SD pixel level is x1 to xn and the HD pixel level is y, n by the coefficients w1 to wn for each class
The linear estimation equation of the tap is set as y '= w1x1 + w2x2 + ｗ + wnxn (2). Before learning, wi is an undetermined coefficient.

As described above, learning is performed for a plurality of Hs for each class.
Performed for D data and SD data. If the number of data is m, according to Equation _{2, y j '= w1x j} 1 + w2x j 2 + ‥‥ + wnx j n (3) ( where, j = 1,2, ‥‥ m)

[0028] m> For n, since w1~wn is not uniquely determined, elements of an error vector _{E e j = y j - (} w1x j 1 + w2x j 2 + ‥‥ + wnx j n) (4) ( where, j = 1,2, ‥‥ m), and a coefficient that minimizes the following Expression 5 is obtained.

[0029]

(Equation 2)

This is a so-called least squares solution. Here, the partial differential coefficient by wi in Expression 4 is obtained.

[0031]

(Equation 3)

Since each wi may be determined so that Equation 6 is set to 0,

[0033]

(Equation 4)

Using a matrix,

[0035]

(Equation 5)

## EQU1 ## This equation is generally called a normal equation. When this equation is solved for wi using a general matrix solution such as a sweeping-out method, a prediction coefficient wi is obtained, and the prediction coefficient w
i is stored in the memory.

As described above, the learning unit 4 uses the actual data H
The prediction coefficient wi can be obtained using the D signal, and this is stored in the memory. Next, when an SD signal is converted into an HD signal, that is, when the SD signal is up-converted, an output HD image can be generated for an arbitrary input SD image. The configuration for this is shown in the block diagram of FIG.

An SD image of (n + 1) fields is supplied from an input terminal 31 as a signal d0, and d0 is supplied to a field memory 32 and up-conversion circuits 34a to 34d. Field memory 32 supplied with d0
To output an SD image of n fields as a signal d1, and this signal d1 is supplied to the field memory 33 and the up-conversion circuits 34a to 34d, respectively. An SD image of (n-1) fields is output as a signal d2 from the field memory 33 supplied with d1, and supplied to the up-convert circuits 34a to 34d, respectively.

The SD image signals d0, d1, and d2 supplied to the up-conversion circuits 34a to 34d are up-converted into HD images and output. These up-conversion circuits 34a to 34d are responsible for signal conversion in each of the above-described modes 1 to 4. The selector 35 is controlled by a select signal from the input terminal 37.
Which H of mode 1 to mode 4 corresponding to the position of the HD pixel
This is an identification signal that has been accurately upconverted by checking whether it is a D image, and the output of the upconversion circuit selected corresponding to this mode identification is output from the output terminal 36.

Up-conversion circuits 34a-3 in FIG.
4d have the same configuration as each other except for the stored prediction coefficients, an example of which is shown in FIG. SD image signals d0, d1, and d2 input from the input terminal 41, that is, (n-1) fields, n fields, (n + 1)
The fields are supplied to the blocking circuits 42, respectively. The blocking circuit 42 extracts the data of the block of the unit to be converted from the image, and converts the raster scanning order into the data of the block order as shown in FIG. The output data of the blocking circuit 42 is supplied to an ADRC circuit 43 and a prediction operation circuit 45. In the ADRC circuit 43,
The supplied block unit data is converted to, for example, 1-bit AD.
RC encoding is performed, and the class is determined according to Equation 1.

The class code generation circuit 44 generates a class code corresponding to the determined class, and this class code is supplied to the prediction coefficient memory 5 as an address. The prediction coefficient of the class is read from the memory 5, and the prediction operation circuit 45 determines the data in the block unit supplied from the blocking circuit 42 and the determined prediction coefficient w
1 to wn, the estimated HD data y ′ is output to the output terminal 46 by the calculation according to the prediction formula y ′ = w1x1 + w2x2 + ‥‥ + wnxn (10)
Output from

FIG. 11 is a flowchart of the above-described up-conversion process. Up-conversion control is started from step 51, and in the data block formation in step 52, an SD signal is supplied, and processing for extracting SD pixels in processing block units is performed. At the end of the data in step 53, if the processing of all the input data has been completed, the process proceeds to the end of step 57.
Control transfers to step 54 for class determination.

In step 54, the class is determined from the signal pattern of the SD signal. In this control,
It is preferable to use 1-bit ADRC to reduce the number of bits as in the case of learning. In the prediction coefficient restoration in step 55, the prediction coefficient corresponding to the class code is restored from the memory. In the prediction operation of step 56, the prediction expression operation of Expression 10 is performed, and the prediction data of the HD pixel is output. This series of control is repeated for all data, and when all data is completed, control is shifted from the end of data in step 53 to the end of step 57, and the up-conversion process is completed.

The ADRC circuit 43 is provided as information compression means. Instead of the ADRC circuit 43, for example, DCT (Discrete Cosine Transform), V
As long as it is a means capable of performing data compression, such as providing a Q (vector quantization) or a DPCM (prediction coding) circuit, what is provided can be appropriately selected.

[0045]

According to the present invention, a class is divided according to a three-dimensional (spatio-temporal) distribution of the level of an input image signal, and a signal conversion is performed for each class based on a prediction formula obtained by learning in advance. Also, information in the time direction can be effectively used, and a more accurate converted image signal can be output for a motion image.

Since information in the time direction can also be used effectively, it is particularly effective when converting interlaced signals.

[Brief description of the drawings]

FIG. 1 is a block diagram of an example of a configuration for obtaining a prediction coefficient in an image signal conversion device according to the present invention.

FIG. 2 is an example of a schematic diagram used for describing HD pixels and SD pixels in one embodiment of the present invention.

FIG. 3 is a flowchart of an example of a configuration for obtaining a prediction coefficient in the image signal conversion device according to the present invention.

FIG. 4 is an example of a schematic diagram used for describing HD pixels and SD pixels in one embodiment of the present invention.

FIG. 5 is an example of a schematic diagram used for describing HD pixels and SD pixels in one embodiment of the present invention.

FIG. 6 is an example of a schematic diagram used for describing HD pixels and SD pixels in one embodiment of the present invention.

FIG. 7 is an example of a schematic diagram used for describing HD pixels and SD pixels in one embodiment of the present invention.

FIG. 8 is a block diagram used for explaining an ADRC circuit;

FIG. 9 is an example of a block diagram used for describing a mode in one embodiment of the present invention.

FIG. 10 is a block diagram of an embodiment of an image signal conversion device according to the present invention.

FIG. 11 is a flowchart of one embodiment of the present invention.

FIG. 12 is a block diagram used to explain conventional up-conversion from an SD signal to an HD signal.

FIG. 13 is a block diagram of an example of an interpolation filter used in a conventional signal conversion device.

[Explanation of symbols]

 5 Prediction coefficient memory 41 Input terminal 42 Blocking circuit 43 ADRC circuit 44 Class code generation circuit 45 Prediction operation circuit 46 Output terminal

Claims (3)

(57) [Claims] 1. An image signal conversion apparatus for converting a standard resolution image signal into a high resolution image signal, comprising: means for performing class division according to a shape of a three-dimensional distribution of levels of the standard resolution input image signal; Storage means for storing a prediction coefficient value obtained by learning in advance for each of the classes into which the class has been divided; and an optimum estimation value output from a calculation based on a prediction formula including the prediction coefficient value, which is combined with the storage means An image signal conversion device comprising means for outputting an image signal having a higher resolution than the input image signal by including the estimated value. 2. The image signal conversion device according to claim 1, wherein the class division uses adaptive dynamic range coding for saving memory capacity. 3. The image signal conversion device according to claim 1, wherein the standard resolution image signal and the high resolution image signal are interlaced scanning signals in a plurality of pixels adjacent to the high resolution signal. Image signal conversion device.