JP3622209B2 - Image signal processing apparatus and method, and weight coefficient deriving apparatus and method - Google Patents

Description

[0001]
This invention particularly relates to an image signal processing device and how well the weighting factor deriving device and method for adaptive mixing process used in the scanning line interpolation and the like.
[0002]
[Prior art]
The conventional image signal processing apparatus shown in FIG. 5 supplies input signals input from the input terminals 51 to a plurality of different processing means, processing circuits 52 and 53, and a feature amount detection circuit 54, respectively. The processing circuit 52 forms an interpolation pixel by in-field processing. On the other hand, the processing circuit 53 forms an interpolation pixel by inter-field processing.
[0003]
The feature quantity detection circuit 54 multiplies weighting coefficients k and (1−k) obtained from a formula based on a model of an image signal that is theoretically assumed from a feature quantity such as an image correlation value and a motion quantity. To 55 and 56, respectively. The multiplier 55 multiplies the output data of the processing circuit 52 by the weight coefficient k from the feature amount detection circuit 54. The multiplier 56 multiplies the output data of the processing circuit 53 and the weight coefficient (1-k) from the feature amount detection circuit 54. Adder 57 mixes the outputs of multipliers 55 and 56 and outputs the result to output terminal 58. This output signal is obtained by doubling the scanning line of the input image signal.
[0004]
[Problems to be solved by the invention]
In the motion-adaptive signal mixing method as described above, when the assumed model and the actual image signal have different properties, appropriate mixing cannot be performed, and the output signal is deteriorated. There is a possibility that the effect of signal processing such as interpolation may be reduced.
[0005]
Accordingly, an object of the present invention is not simply to mix the signals, by performing learning from known image signal, it obtains a weighting coefficient, the image signal processing device and to mix the signal using the weighting factor mETHODS as well as a weighting factor derivation apparatus and method.
[0006]
[Means for Solving the Problems]
According to the first aspect of the present invention, there is provided class detecting means for determining a class to which the image signal belongs and outputting class information based on the input image signal, and a first image signal processing method for processing the input image signal. According to the class information from the first image processing means, the second image processing means for processing the input image signal by a second image signal processing method different from the first image signal processing method, and the class detection means, The weighting coefficient storage means for outputting the weighting coefficient obtained in advance by learning and the coefficients supplied from the weighting coefficient storage means are used for weighted addition of the output signals of the first and second image processing means and output. A first learning image generating means for processing a known image signal by a signal processing method corresponding to the first image signal processing method and outputting a first learning image signal; , Known image A second learning image generating means for processing the signal by a signal processing method corresponding to the second image signal processing method and outputting a second learning image signal; a known image signal; and the first and second learning signals. An image signal processing apparatus characterized in that the coefficient is generated by a weighting coefficient learning unit including a deriving unit that derives an optimal weighting factor using the image signal.
According to a third aspect of the present invention, there is provided a weighting factor deriving device for deriving a weighting factor used when converting a desired image signal into an output image signal having a higher quality than the desired image signal, based on the input image signal. A class detection means for determining a class to which the image signal belongs and outputting class information; and a signal processing method corresponding to the first image signal processing method used for conversion to a high-quality output image signal. A first learning image generation means for processing an image signal and outputting a first learning image signal, and a signal processing method corresponding to a second image signal processing method used when converting the image signal into a high-quality output image signal The second learning image generating means for processing the input image signal and outputting the second learning image signal, and the output signal and the input image from the first and second learning image generating means according to the class information Based on the signal and And deriving means for deriving the coefficients, the weighting factor derivation apparatus characterized by having a.
Further, the invention of claim 4 determines the class to which the image signal belongs based on the input image signal and outputs the class information, and processes the input image signal by the first image signal processing method. According to class information from a first image processing step, a second image processing step of processing an input image signal by a second image signal processing method different from the first image signal processing method, and a class detection step The weighting coefficient output step for outputting the weighting coefficient obtained in advance by learning and the coefficients supplied from the weighting coefficient output step are used for weighted addition of the output signals of the first and second image processing steps, respectively. And outputting a first learning image signal by processing a known image signal by a signal processing method corresponding to the first image signal processing method and outputting a first learning image signal. Process and A second learning image generating step of processing the image signal by a signal processing method corresponding to the second image signal processing method and outputting a second learning image signal; a known image signal; and the first and second image signals An image signal processing method characterized in that the coefficient is generated by a derivation calculation step of deriving an optimum weighting coefficient using a learning image signal.
According to a fifth aspect of the present invention, there is provided a weighting factor deriving method for deriving a weighting factor used when converting a desired image signal into an output image signal having a higher quality than the desired image signal, based on the input image signal. A class detection step for determining the class to which the image signal belongs and outputting the class information, and a signal processing method corresponding to the first image signal processing method used for conversion to a high-quality output image signal. A first learning image generation step for processing an image signal and outputting a first learning image signal, and a signal processing method corresponding to a second image signal processing method used for conversion to a high-quality output image signal Then, a second learning image generation step that processes the input image signal and outputs a second learning image signal, and an output signal obtained in the first and second learning image generation steps according to the class information Based on the input image signal Is a weighting factor derivation method characterized by comprising: a deriving step of deriving the weighting factor, the [0007]
[Action]
Since the weighting coefficient at the time of mixing is determined in advance by learning, signal mixing based on actual image characteristics can be performed.
[0008]
【Example】
An embodiment in which the present invention is applied to scanning line interpolation will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration at the time of learning according to an embodiment of the present invention. Reference numeral 11 denotes an input terminal which receives a large number of non-interlaced standard still image signals and supplies them to the vertical thinning filter 12 and the least squares arithmetic circuit 16. The vertical decimation filter 12 supplied with the still image signal from the input terminal 11 decimates the non-interlaced still image signal by 1/2 in the vertical direction, and generates a still image equivalent to a normal interlaced image. , 14 and the ADRC circuit 15 respectively.
[0009]
Here, each processing of the processing circuits 13 and 14 will be described. The processing circuit 13 takes out the average value of the pixel data of the upper scanning line and the pixel data of the lower scanning line of the interpolation scanning line as the interpolation pixel of the interpolation scanning line. The processing of the processing circuit 14 outputs the pixel data of the previous field at the same position as the interpolation pixel of the interpolation scanning line of the current field as the interpolation pixel of the interpolation scanning line of the current field.
[0010]
As described above, the processing circuit 13 outputs the interpolation data x ₁ by treating the field interlaced signal. Output data x ₁ is supplied to the arithmetic circuit 16 of the least squares method. Processing circuit 14 outputs the interpolation data x ₂ by processing between fields interlaced signal. After performing the inter-field processing, the output data x ₂ are supplied to the arithmetic circuit 16 of the least squares method.
[0011]
The arithmetic circuit 16 of the least square method, the still image signal supplied from the input terminal 11 and the processing circuit 13 and 14 have been respectively output data x ₁ and x ₂ i.e. the supply from the non-interlaced signals and interlaced input An error of an image obtained by applying a weighting process to a signal is calculated by the method of least squares, and weight coefficients w ₁ and w ₂ are identified from the calculation result. That is,
y ′ = w ₁ x ₁ + w ₂ x ₂ (1)
, The weighted output value y ′ is obtained, and the weighting coefficients w ₁ and w ₂ that minimize the square of the error between the value y ′ and the actual value y are obtained by the least squares arithmetic circuit 16. .
[0012]
The ADRC circuit 15 compresses the interlace signal supplied from the vertical decimation filter 2 by adaptive dynamic range coding (hereinafter referred to as ADRC), which will be described later. The output of the ADRC circuit 15 is supplied to the class code generation circuit 17 to generate a class code c. The weighting coefficient memory 18 holds the weighting coefficients w ₁ and w ₂ supplied from the least squares arithmetic circuit 16 using the supplied class code c as an address.
[0013]
Here, ADRC will be described by taking 1-bit ADRC as an example. Further, 1-bit ADRC will be described by taking pixel data of a (3 × 3) block as an example centering on a pixel of interest on an interlaced image. An example of a block of the ADRC circuit 15 is shown in FIG. In FIG. 2, the detection circuit 22 detects the maximum value MAX and the minimum value MIN for each block with respect to the data converted into the block order from the input terminal 21. MAX and MIN are supplied to the subtracting circuit 23, and a dynamic range DR is generated at the output thereof. The input data and MIN are supplied to the subtraction circuit 24, and the minimum value is removed from the subtraction circuit 24, thereby generating normalized pixel data.
[0014]
The dynamic range DR is supplied to the division circuit 25, the normalized pixel data is divided by the dynamic range DR, and the output data of the division circuit 25 is supplied to the comparison circuit 26. In the comparison circuit 26, it is determined whether the percent calculation power of the eight pixels other than the target pixel is larger or smaller with reference to 0.5. Depending on the result, data DT of ｀ 0 'or ｀ 1' is generated. This comparison output DT is taken out to the output terminal 27. If class division is performed using this 1-bit ADRC, a class code of (3 × 3) blocks is expressed by 9 bits. The pixels used for class division are not limited to (3 × 3) block pixels, and various pixels such as a block including pixels in the previous field can be used.
[0015]
As described above, the learning unit can acquire the weighting coefficient w _i by using the image signal which is actual data, and stores this in the memory. Next, when adaptive mixing is performed, an output image in which scanning line interpolation is performed on an arbitrary input image can be generated. The configuration for this is shown in the block diagram of FIG.
[0016]
Reference numeral 31 denotes an input terminal to which a standard interlaced image signal is input. Signals input from the input terminal 31 are supplied to the processing circuits 13 and 14 and the ADRC circuit 15, respectively. The processing circuit 13 performs in-field processing on the input image signal and supplies output data x ₁ to the multiplier 32. The processing circuit 14 performs inter-field processing on the input image signal and supplies output data x ₂ to the multiplier 33.
[0017]
The input image signal supplied to the ADRC circuit 15 is compressed by ADRC, and its output is supplied to the class code generation circuit 17. The class code generation circuit 17 generates a class code c corresponding to the input image signal subjected to the compression process. The generated class code c is supplied to the weighting coefficient memory 18, and the weighting coefficients w ₁ (c) and w ₂ (c) corresponding to the class code are read out.
[0018]
The read weighting factors w ₁ (c) and w ₂ (c) are supplied to multipliers 32 and 33, respectively. The multiplier 32 multiplies the output data x ₁ of the processing circuit 13 and the weighting coefficient w ₁ (c), and the multiplier 33 multiplies the output data x ₂ of the processing circuit 14 and the weighting coefficient w ₂ (c). . The multipliers 32 and 33 are respectively supplied to the adder 34 and added. The addition result is output from the output terminal 35. Here, the weighted and mixed output data y ′ is shown by a calculation formula. This y ′ constitutes scanning line interpolation, and a non-interlaced image signal is obtained at the output terminal 35.
y ′ = w ₁ (c) x ₁ + w ₂ (c) x ₂ (2)
[0019]
FIG. 4 is a flowchart of the above-described scanning line interpolation process. Control of scanning line interpolation is started from step 41, and in the data block conversion of step 42, processing for extracting an input image in units of processing blocks is performed. At the end of data in step 43, the control shifts to the end of step 47 if the processing of all input data has been completed, and to the class determination of step 44 if the processing has not ended.
[0020]
In step 44, the class is determined from the signal pattern of the input image. In this control, for example, 1-bit ADRC is used to reduce the number of bits as in the learning. In the weighting factor restoration at step 45, the weighting factor corresponding to the class code is restored from the memory. In the weight calculation in step 46, the interpolation value obtained by the intra-field processing and the interpolation value obtained by the inter-field processing are weighted and added to output interpolation data. This series of control is repeated for all data, and when the processing of all data is completed, the control shifts from the end of data in step 43 to the end of step 47, and the scanning line interpolation processing ends.
[0021]
Further, although the ADRC circuit 15 is provided as the information compression means, this is merely an example, and instead of the ADRC circuit 15, for example, a DCT (Discrete Cosine Transform) circuit, a VQ (Vector Quantization) circuit, a DPCM (Prediction) It is possible to appropriately select what is provided as long as it is a means capable of performing data compression, such as providing an encoding) circuit or a BTC (Block Transcoding) circuit.
[0022]
【The invention's effect】
According to the present invention, since a class and a weighting coefficient corresponding to the class are learned on the basis of the actual image properties, the results of different signal processing can be favorably weighted and mixed without causing deterioration in the output signal.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an example of a configuration for obtaining weighting coefficients in an image signal conversion apparatus according to the present invention.
FIG. 2 is a block diagram used to describe an ADRC circuit.
FIG. 3 is a block diagram of an embodiment of scanning line interpolation according to the present invention.
FIG. 4 is a flowchart of an example of scanning line interpolation according to the present invention.
FIG. 5 is a block diagram used for explaining a conventional signal mixing method;
[Explanation of symbols]
13, 14 Processing circuit 15 ADRC circuit 17 Class code generation circuit 18 Weight coefficient memory 32, 33 Multiplier 34 Adder

Claims (5)

Class detection means for determining a class to which the image signal belongs and outputting class information based on the input image signal;
First image processing means for processing the input image signal by a first image signal processing method;
Second image processing means for processing the input image signal by a second image signal processing method different from the first image signal processing method;
A weighting factor storage unit that outputs a weighting factor obtained by learning in advance according to the class information from the class detection unit;
Means for weighting and adding the output signals of the first and second image processing means using the coefficients supplied from the weighting coefficient storage means,
The weighting factor is
First learning image generation means for processing a known image signal by a signal processing method corresponding to the first image signal processing method and outputting a first learning image signal;
Second learning image generation means for processing the known image signal by a signal processing method corresponding to the second image signal processing method and outputting a second learning image signal;
Derivation means for deriving an optimum weighting factor using the known image signal and the first and second learning image signals;
An image signal processing apparatus comprising: a coefficient generated by weight coefficient learning means. The image signal processing apparatus according to claim 1,
The class detection means is
An information compression unit that divides the input image signal into blocks of a predetermined range, compresses information indicating a level distribution pattern of the image signal included in the divided range of the block, and outputs pattern compression information;
Output means for outputting the class detection information in accordance with the pattern compression information supplied from the information compression means;
An image signal processing apparatus comprising: In a weighting factor deriving device for deriving a weighting factor used when converting a desired image signal into an output image signal of higher quality than the desired image signal,
Class detection means for determining a class to which the image signal belongs and outputting class information based on the input image signal;
A first learning that processes the input image signal and outputs a first learning image signal by a signal processing method corresponding to the first image signal processing method used when converting to the high-quality output image signal Image generating means;
A second learning that processes the input image signal and outputs a second learning image signal by a signal processing method corresponding to the second image signal processing method used when converting to the high-quality output image signal Image generating means;
Derivation means for deriving the weighting factor based on the output signals from the first and second learning image generation means and the input image signal according to the class information;
A weighting factor deriving device characterized by comprising: Determining a class to which the image signal belongs based on the input image signal and outputting class information;
A first image processing step of processing the input image signal by a first image signal processing method;
A second image processing step of signal processing the input image signal by a second image signal processing method different from the first image signal processing method;
According to the class information from the class detection step, a weighting factor output step for outputting a weighting factor obtained by learning in advance,
Using the coefficients supplied from the weighting factor output step, the output step of weighting and adding the output signals of the first and second image processing steps, respectively,
The weighting factor is
A first learning image generation step of processing a known image signal by a signal processing method corresponding to the first image signal processing method and outputting a first learning image signal;
A second learning image generation step of processing the known image signal by a signal processing method corresponding to the second image signal processing method and outputting a second learning image signal;
A derivation calculating step of deriving an optimal weighting factor using the known image signal and the first and second learning image signals;
An image signal processing method characterized in that the coefficient is generated by the method. In a weighting factor derivation method for deriving a weighting factor used when converting a desired image signal into an output image signal of higher quality than the desired image signal,
A class detection step of determining class to which the image signal belongs and outputting class information based on the input image signal;
A first learning that processes the input image signal and outputs a first learning image signal by a signal processing method corresponding to the first image signal processing method used when converting to the high-quality output image signal An image generation process;
A second learning that processes the input image signal and outputs a second learning image signal by a signal processing method corresponding to the second image signal processing method used when converting to the high-quality output image signal An image generation process;
A derivation step of deriving the weighting factor based on the output signals obtained in the first and second learning image generation steps and the input image signal according to the class information;
A weight coefficient derivation method characterized by comprising:

Images