JP2825482B2 - Digital image signal interpolation device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an interpolation device provided on a receiving side or a reproducing side of a digital image signal transmission system or a recording / reproducing system. 2. Description of the Related Art When transmitting a digital video signal obtained by encoding a video signal, subsampling is used to narrow a transmission band. By the sub-sampling, for example, the image data of 1/2 pixel is thinned out. On the receiving side,
An interpolation circuit for interpolating the thinned pixels is provided.
Conventionally, a digital low-pass filter designed in the frequency domain has been used as the interpolation circuit. [Problems to be Solved by the Invention] When designing an interpolation filter in the frequency domain, iterative operations based on experience are required from the frequency characteristics of input and output signals, and when the sampling frequency of the digital video signal is different, It was necessary to design an interpolation filter according to the sampling frequency. Therefore, the design of the interpolation filter is troublesome, and the versatility is poor. Further, in the case of a composite color video signal in which a carrier chrominance signal is superimposed on a luminance signal, it is necessary to consider the phase of the carrier chrominance signal, and thus it has been difficult to apply a conventional interpolation filter. An object of the present invention is to provide an interpolating apparatus which can easily design an optimum configuration, is versatile, and can be applied to a composite color video signal. [Means for Solving the Problems] The present invention relates to a digital image signal interpolating apparatus for interpolating and generating image data of a predetermined pixel between existing input image data and corresponding pixels by using existing input image data. Means for receiving existing input image data and extracting a predetermined number of existing pixel data existing at a position around a predetermined pixel; and using the pixel data for calculating a coefficient, an error between an interpolation value and a true value. The pixel data of a predetermined pixel is interpolated by a linear linear combination of a predetermined number of coefficients predetermined by the least square method so that the sum of the squares of the pixels becomes minimum and the extracted predetermined number of existing pixel data. And a means for generating a digital image signal. [Operation] Image data of a thinned pixel is predicted by a linear linear combination of a predetermined number of existing pixel data surrounding the thinned pixel and a predetermined number of weighting coefficients in the same field (or frame). A weighting coefficient is determined in advance so as to minimize the sum of squares of the error between the predicted interpolation value and the true value. The identification of the weight coefficient is performed by photographing a plurality of images with a video camera, digitizing the image signals, and obtaining image data.
Processing is performed by processing this image data using an electronic computer. The number (order) of image data actually existing in the vicinity used for identifying the weight coefficient is set according to the scale of hardware. By setting this order, a weight coefficient that minimizes the sum of squares of the error is identified. The interpolation circuit according to the present invention can have an optimum configuration irrespective of the data sampling frequency, and can have a simpler design method than a design in the frequency domain. Hereinafter, an embodiment of the present invention will be described with reference to the drawings. This description will be made in the order of the following items. a. Configuration on the transmission side b. Configuration on the reception side c. Blocking circuit d. Dynamic range detection circuit e. Quantization circuit f. Interpolation circuit g. Modification a. Configuration on the transmission side FIG. 1 shows the configuration of the recording side as a whole. For example, a color video signal of the NTSC system is supplied to an input terminal denoted by reference numeral 1. This color video signal is A /
The sample is supplied to the D converter 2 and, for example, one sample is 8
The digital color video signal quantized to bits is A /
Obtained from the D converter 2. The digital color video signal is supplied to the sub-sampling circuit 3, and the output signal of the sub-sampling circuit 3 is supplied to the blocking circuit 4. A pre-filter for band limitation is not provided at a stage preceding the sub-sampling circuit 3, so that a high frequency component of the input color video signal is not lost. In the sub-sampling circuit 3, the digital color video signal is sampled at a sampling frequency of 2 fsc. Also, the input digital television signal is converted by the blocking circuit 4 into a continuous signal for each two-dimensional block which is a unit of encoding. In this embodiment, 1
One block obtained by dividing the screen of the field has a size of (4 lines × 8 pixels = 32 pixels). FIG. 3 shows this one block. In FIG. 3, a solid line shows a line of an odd field, and a broken line shows a line of an odd field.
Indicates a line in an even field. Unlike this example, the present invention can be applied to, for example, a three-dimensional block including four two-dimensional regions belonging to each of four frames. By the sub-sampling circuit 3 provided before the blocking circuit 4, the pixels in the block are thinned out as shown in FIG. 4, and the number of pixels in one block is set to 16 pixels. 4. In FIG. 4, .largecircle. Indicates a sub-sampled pixel, and X indicates a thinned pixel. The output signal of the blocking circuit 4 is supplied to the dynamic range detection circuit 5 and the delay circuit 6. The dynamic range detection circuit 5 has a dynamic range DR for each block.
And the minimum value MIN. The pixel data PD from the delay circuit 6 is supplied to the subtraction circuit 7, where the pixel data PDI from which the minimum value MIN has been removed is formed. The quantized circuit 8 is supplied with the pixel data PDI and the dynamic range DR that have been subjected to sub-sampling and having undergone the minimum value removal via the subtraction circuit 7. In the quantization circuit 8, the pixel data PDI is quantized according to the dynamic range DR. From the quantization circuit 8, a code signal DT obtained by converting one pixel data into four bits is obtained. The code signal DT from the quantization circuit 8 is supplied to the framing circuit 9. The dynamic range DR (8 bits) and the minimum value MIN (8 bits) are supplied to the framing circuit 9 as an additional code for each block. The framing circuit 9 performs an error correction coding process on the code signal DT and the above-described additional code, and adds a synchronization signal.
Transmission data is obtained at the output terminal 10 of the framing circuit 9,
This transmission data is transmitted to a transmission line such as a digital line. In the case of a digital VTR, an output signal is supplied to a rotary head via a recording amplifier, a rotary transformer, and the like. b. Configuration on the receiving side FIG. 2 shows the configuration on the receiving (or reproducing) side. The data received from the input terminal 11 is supplied to the frame decomposition circuit 12. The frame decomposition circuit 12 separates the code signal DT from the additional codes DR and MIN, and performs an error correction process. The code signal DT is supplied to the decoding circuit 13, and the dynamic range DR is supplied to the decoding circuit 13. The decoding circuit 13 performs a process reverse to that of the quantization circuit 8 on the transmission side. That is, the 8-bit data after the removal of the minimum level is decoded to the representative level, and this data and the 8-bit minimum value MIN are added by the adder circuit 14 to decode the original pixel data. Output data of the adder circuit 14 is supplied to the block decomposition circuit 15. The block decomposing circuit 15 is a circuit for converting the decoded data in the order of blocks into the same order as the scanning of the television signal, contrary to the blocking circuit 4 on the transmission side. An output signal of the block decomposition circuit 15 is supplied to an interpolation circuit 16 to which the present invention is applied. Interpolator 16
In, the data of the thinned pixel is interpolated by the surrounding sub-sample data. The digital color video signal having a sampling frequency of 4 fsc from the interpolation circuit 16 is supplied to the D / A converter 1
Supplied to 7. An analog color video signal is obtained at an output terminal 18 of the D / A converter 17. If a pre-filter is not provided on the transmission side, aliasing may occur at, for example, a point where the luminance level changes sharply. A circuit for removing this distortion may be connected to the output side of the interpolation circuit 16. c. Blocking Circuit The blocking circuit 4 will be described with reference to FIGS. 5, 6, and 7. For simplicity of explanation, it is assumed that a screen of one field has a configuration of (4 lines × 8 pixels) as shown in FIG. 6, and this screen is vertically divided into two as shown by broken lines, Divided into four, (2 lines x 2
The case where eight blocks (pixels) are formed will be described. In FIG. 5, as shown in FIG. 7A, input data A consisting of four lines (Th _{0 to} Th ₃ ) is applied to an input terminal indicated by 21.
And a sampling clock B (FIG. 7B) synchronized with the input data A is supplied to an input terminal indicated by 22. The sample data of numbers (1 to 8) the line Th ₀ indicates respectively show respectively digits (11 to 18) of the sample data of the line Th ₁ s, the number (21 to 28) is the line Th ₂ Sample Data And the numbers ( _{31 to} 38) indicate the line Th ₃
Are shown below. The input data A is supplied to a delay circuit 23 having a delay amount of Th and a delay circuit 24 having a delay amount of 2Ts (Ts: sampling period). Further, the sampling clock B is supplied to the 1/2 frequency dividing circuit 27. The output signal C of the delay circuit 24 (FIG. 7C) is a switch circuit.
25 and 26 are respectively supplied to one input terminal of the delay circuit 23.
(D in FIG. 7) is supplied to the other input terminals of the switch circuits 25 and 26, respectively. The switch circuit 25 is 1 /
The pulse signal E is controlled by an output signal E (FIG. 7E) of the divide-by-2 circuit 27 (FIG. 7E).
It is controlled by the pulse signal inverted by 28. The switch circuits 25 and 26 alternately select an input signal (C or D) every 2Ts. The output signal F from the switch circuit 25 is shown in FIG. 7F, and the output signal G from the switch circuit 26 is shown in FIG. 7G. The output signal F of the switch circuit 25 is the first signal of the switch circuit 29.
And a delay circuit 30 having a delay amount of 4Ts. The output signal G of the switch circuit 26 is supplied to a delay circuit 31 having a delay amount of 2Ts. The output signal H of the delay circuit 30 (H in FIG. 7) is supplied to a third input terminal of the switch circuit 29. The output signal I (I in FIG. 7) of the delay circuit 31 is supplied to a second input terminal of the switch circuit 29 and a delay circuit 32 having a delay amount of 4Ts. The output signal J (FIG. 7J) of the delay circuit 32 is supplied to a fourth input terminal of the switch circuit 29. The output signal of the 1/2 frequency dividing circuit 27 is supplied to the 1/2 frequency dividing circuit 33 to form an output signal K (FIG. 7K). The switch circuit 29 is controlled by this signal K, and the first and the
2, the third and fourth input terminals are sequentially selected. Therefore,
Signal L extracted from switch circuit 29 to output terminal 34
Is as shown in FIG. 7L. That is, the order of data in each field is the order of each block (for example, 1 → 2 → 11
→ Converted to 12). Of course, the actual number of pixels in one field is much larger, unlike the example shown in FIG.
By the same scan conversion as described above, the data is converted into the order of each block shown in FIG. d. Dynamic range detection circuit FIG. 8 shows an example of the configuration of the dynamic range detection circuit 3. As described above, the input terminal indicated by 41 is supplied with the image data of the area that needs to be encoded for each block from the blocking circuit 4 sequentially. The pixel data from the input terminal 41 is supplied to the selection circuit 42 and the selection circuit 43. The selection circuit 42 selects and outputs the higher level between the pixel data of the digital color video signal and the output data of the latch 44. The other selection circuit 43
Selects the smaller one between the pixel data of the input digital color video signal and the output data of the latch 45 and outputs it. The output data of the selection circuit 42 is supplied to the subtraction circuit 46 and is also captured by the latch 44. Output data of the selection circuit 43 is supplied to the subtraction circuit 46 and the latch 48, and is also captured by the latch 45. The latch pulse is supplied from the control unit 49 to the latches 44 and 45. The control unit 49 includes a sampling clock synchronized with the digital color video signal,
A timing signal such as a synchronization signal is supplied from a terminal 50.
The control unit 49 supplies a latch pulse to the latches 44 and 45 and the latches 47 and 48 at a predetermined timing. At the beginning of each block, the contents of latches 44 and 45 are initialized. The latch 44 is initialized with all “0” data, and the latch 45 is initialized with all “1” data. The maximum level of the sequentially supplied pixel data of the same block is stored in the latch 44. The minimum level of the sequentially supplied pixel data of the same block is stored in the latch 45. When the detection of the maximum level and the minimum level is completed for one block, the maximum level of the block is generated at the output of the selection circuit. On the other hand, the minimum level of the block occurs at the output of the selection circuit 43. When the detection for one block is completed, the latches 44 and 45 are initialized again. The output of the subtraction circuit 46 has the maximum level from the selection circuit 42.
The dynamic range DR of each block obtained by subtracting the minimum level MIN from the MAX and the selection circuit 43 is obtained. The dynamic range DR and the minimum level MIN are latched by the latches 47 and 48, respectively, by the latch pulse from the control block 49. The dynamic range DR of each block is obtained at the output terminal 51 of the latch 47, and the output terminal 52 of the latch 48.
Then, the minimum value MIN of each block is obtained. e. Quantization circuit The quantization circuit 8 performs encoding adapted to the dynamic range DR. FIG. 9 shows an example of the quantization circuit 8. In FIG. 9, in the ROM denoted by 55, the pixel data PDI (8 bits) from which the minimum value has been removed has a compressed bit number of, for example, 4 bits.
A data conversion table for converting to bits is stored. The dynamic range DR from the input terminal 56 and the pixel data PDI from the input terminal 57 are supplied to the ROM 55 as address signals. In the ROM 55, a data conversion table is selected according to the size of the dynamic range DR, and a 4-bit code signal DT is extracted from an output terminal 58. In the quantization circuit 8, when the code signal DT is 2 bits (4 bits in the embodiment), the dynamic range DR of the block is divided into four regions as shown in FIG. These four areas are (00) (01) (10) (11)
And the central level L0, L1, L2, L3 is set as the representative level of each area. A 2-bit code signal DT is generated according to the area including the data PDI after the minimum value has been removed. The level of the digital color video signal is correlated within the block even when the digital carrier chrominance signal is superimposed, and the dynamic range DR of each block is concentrated in a narrow range in a stationary part that is not a transient part. I have. Therefore, like 4 bits,
Even if the quantization is performed with the number of bits compressed to, the image quality hardly deteriorates. Also, since each pixel is encoded independently of the other pixels, a sharp level change of the digital color video signal can be reproduced, and the frequency characteristics can be improved as compared with the DPCM. It should be noted that pixel data having each of the minimum level MIN and the maximum level MAX always exists in one block. Therefore, to increase the number of code signals having an error of 0,
As shown in Figure 11, the dynamic range DR is (2 ^m -1)
(Where m is the number of quantization bits)
MIN may be the representative minimum level L0, and the maximum level MAX may be the representative maximum level L3. Further, the quantization circuit 8 may use a configuration including a divider for dividing the dynamic range DR and a comparison circuit for determining the level area to which the data PDI after removing the minimum value belongs, other than the ROM. f. Interpolation Circuit An example of the interpolation circuit 16 provided on the receiving side will be described. This interpolation circuit 16 is a two-dimensional filter designed in the time domain. That is, the interpolation circuit 16 predicts a thinned pixel as a linear combination of a plurality of pieces of reception data (sub-sample data) existing around the thinned pixel to be interpolated. In this case, assuming that the difference signal between the true value and the output signal of the interpolation circuit 16 is an error, the weighting coefficient of the linear combination is uniquely obtained by minimizing the square error.
The identification of the weight coefficient will be described below. The digital color video signal at the pixel of interest is
It can be approximated by a linear combination of data Z1 to Zn of n pixels around the target pixel. _{(I, j) = a 1} · Z1 (i, j) + a 2 · Z2 (i, j) + ...... + a n · Zn (i, j) However,, i-line, estimating the color located j samples shows a video signal, a ₁ ~a _n represents a weighting factor, Zm
(I, j) is a color video signal (hereinafter simply referred to as NTSC signal) near the position of (i, j) (m = 1 to
n). With a video camera, you shoot multiple different pixels,
The imaging signal is converted into a digital signal. Using the thus obtained data, the electronic computer to identify a weighting factor a ₁ ~a _n by the least squares method. In other words, the actual data of a certain pattern imaged by the video camera is applied, the neighboring pixel data is multiplied by the weighting factor, and the estimated NTSC signal synthesized as shown in the above equation is compared with the true value x. And the weighting factor a ₁ to the error (−x) ² that minimizes
a _n is calculated using a computer. In a pattern that is one sheet of data, if the error is e, Data is obtained, the weighting coefficients a ₁ when the error distribution is smallest with these _{data, a 2, a 3 ··· a} n is calculated by the computer. The above equation can be expressed as a vector This error vector Weighting coefficient that minimizes the sum of squares of Error variance To minimize And ask for Here, Z ^T indicates a transposed matrix. In this case, in the case of all pixels in one field, a very large matrix is handled, which is not practical. Therefore, the above equation is processed by converting it into a matrix and a vector of a small order. That is, Is changed to sequential processing using the fact that each is an (n, n) matrix and an nth-order vector, regardless of the number of data. here, Is a vector of the surrounding data at the k-th (i, j), The order n is predetermined in accordance with the size of the IC substrate, the scale of the hardware such as the processing speed, and the like. As an example, in this embodiment, as shown in FIG. 12, the same field around the thinned pixel (indicated by □) to be interpolated is used.
By multiplying each weighting factor a ₁ ~a ₃₀ to 30 of the NTSC signal (substatement sample data) Z1~Z30, thinning pixels are interpolated. An example of the weighting factor identified by the above-described method using the electronic computer is shown below. _{a 1 = 0.047 a 2 = -0.064} a 3 = 0.045 a 4 = -0.007 a 5 = 0.002 a 6 = -0.001 a 7 = -0.003 a 8 = -0.050 a 9 = 0.059 a 10 = -0.050 a 11 = 0.064 a ₁₂ = -0.057 a ₁₃ = 0.036 a ₁₄ = -0.109 a ₁₅ = 0.588 a ₁₆ = 0.588 a ₁₇ = -0.109 a ₁₈ = 0.036 a ₁₉ = -0.056 a ₂₀ = 0.064 a ₂₁ = -0.050 a ₂₂ = 0.058 a ₂₃ = -0.050 a ₂₄ = -0.003 a ₂₅ = -0.001 a ₂₆ = 0.002 a ₂₇ = -0.007 a ₂₈ = 0.044 a ₂₉ = -0.063 a ₃₀ = 0.047 The above weighting factor is an example. The hardware may be simplified by using a coefficient expressed by a fraction as a denominator. As shown in FIG. 12, the hardware of the interpolation circuit 16 includes a plurality of line delay circuits for extracting 30 pieces of data Z1 to Z30 around a thinned pixel to be interpolated from the decoded data from the block decomposition circuit 15. Multiple sample delay circuits and extracted data Z1 to Z3
And a multiplier for multiplying each of the 0s by the weighting coefficients a _{1 to} a ₃₀ as described above. Further, as the peripheral pixels used for interpolation, not only data in the same field but also data in the same frame can be used, and a number other than 30 can be used. g. Modifications The present invention can be applied not only to a fixed-length coding method but also to a variable-length coding method as a coding method adapted to a dynamic range. In the variable-length coding scheme, the dynamic range DR for each block is divided by a quantization step corresponding to a predetermined quantization distortion, that is,
The dynamic range DR is divided into a number of level ranges corresponding to the dynamic range DR, and a code signal corresponding to the level range to which the data after the minimum value removal belongs is formed. In the above description, the code signal DT and the dynamic range
The DR and the minimum value MIN are transmitted. However, instead of the dynamic range DR, the maximum value MAX, the quantization step, or the maximum distortion may be transmitted as the additional code. The sub-sampling may be performed after the input signal is blocked. Further, one block of data may be simultaneously extracted by a circuit combining a frame memory, a line delay circuit, and a sample delay circuit, and the present invention is applicable to processing of only a luminance signal. [Effects of the Invention] Since the present invention is designed in the time domain, it does not require empirical repetitive operations compared to designing an interpolation filter in the frequency domain, and even when the sampling frequency is different. A versatile configuration that can be applied. In particular, the present invention has an advantage that it is possible to interpolate a composite color video signal in which a carrier color signal is superimposed on a luminance signal, which has been difficult with a conventional interpolation filter. Further, according to the present invention, a coefficient that minimizes the error is obtained in advance by learning using various image data. There is an advantage that the resolution is less deteriorated than in the case of interpolation. That is, the present invention has an advantage that it is possible to create a resolution that cannot be obtained by the mean value interpolation.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a transmission side of a color video signal transmission system to which the present invention can be applied, FIG. 2 is a block diagram showing a configuration of a reception side, and FIG. FIG. 4 is a schematic diagram used for explaining a sub-sampling, and FIGS. 5, 6, and 7 are block diagrams showing an example of a blocking circuit. FIG. 8 is a block diagram of an example of a dynamic range detection circuit, FIG. 9 is a block diagram of an example of a quantization circuit, FIG.
FIG. 11 and FIG. 11 are schematic diagrams used to explain one example of quantization and other examples, and FIG. 12 is a schematic diagram used to explain an interpolation circuit to which the present invention is applied. Explanation of main reference numerals in the drawing 1: input terminal of color video signal, 4: blocking circuit, 5:
Dynamic range detection circuit, 7: subtraction circuit, 8: quantization circuit, 13: decoding circuit, 15: block decomposition circuit, 16: interpolation circuit.

Claims (1)

(57) [Claims] A digital image signal interpolating device for interpolating and generating image data of a predetermined pixel between the actual input image data and the corresponding pixel by the actual input image data, receiving the actual input image data, Means for extracting a predetermined number of existing pixel data existing at the peripheral position of the pixel, and pixel data for calculating a coefficient, so as to minimize the sum of squares of the error between the interpolated value and the true value. Means for interpolating and generating the pixel data of the predetermined pixel by a linear linear combination of a predetermined number of coefficients predetermined by a multiplication method and the extracted predetermined number of existing pixel data. Interpolating device for digital image signals.