JP2000083225A - Predictive arithmetic unit and image information converter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily extract an output which is not subjected toa product-sum operation with general-purpose performance with respect to number of inputs in the case of generating a prediction value by the product-sum operation between a plurality of pixel values and coefficient data. SOLUTION: A product-sum device 301 consists of multipliers 3021-302n and an adder 303 that sums outputs of the multipliers. The multipliers 3021-302n multiply coefficient data with tap data of a prediction tap. Selectors S1-Sn that select the coefficient data or zero are provided to the multipliers 3021-302n respectively. In the case that number of taps is less than a number (n), the zero data are selected. In the case that the product-sum operation is desired to be evaded, the coefficient data with respect to the tap data that are disregarded are set to 1 and the selectors select the zero data as to the other tap data than above. This system is applied to a prediction device for the image information converter adopting classification adaptive prediction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a prediction operation device and an image information conversion device for performing a prediction operation of a linear combination of a plurality of pixel data and a plurality of coefficient data.

[0002]

2. Description of the Related Art As one of image signal conversion apparatuses for converting an input digital image signal into a different scanning line structure, an output image signal is obtained by linearly combining coefficient data obtained in advance and pixel data of the input image signal. Has been proposed. A predictor for performing this prediction operation includes a multiplier that multiplies each pixel data by each coefficient data, and an adder that adds a multiplication output. As the predictor, a versatile configuration that can be used even when the number of pixel data (the number of taps) changes is preferable. In some cases, image data that does not pass through a predictor is required as an output in order to determine the quality of the prediction result.

[0003]

In the conventional predictor, unused taps are prevented from occurring by optimizing the number of taps. Therefore, it generally does not have versatility to support different tap numbers. When image data that has passed through the predictor is desired to be output, a signal line that bypasses the predictor is provided, and a delay circuit for compensating for a delay required for processing of the predictor is provided in the bypass signal path. Therefore, there is a problem that a delay circuit is required.

Accordingly, an object of the present invention is to provide a versatility in the number of taps, and a prediction operation device which does not require a delay circuit to obtain a through output in which no operation is performed, and image information using the prediction operation device. A conversion device is provided.

[0005]

According to a first aspect of the present invention, a plurality of sample data of a digital information signal are multiplied by a plurality of coefficient data, and a multiplication result is added to output a predicted value or an interpolated value. The above-mentioned prediction operation device is a prediction operation device provided with control means for setting one of sample data and coefficient data input to a plurality of multipliers to 0.

According to an eighth aspect of the present invention, there is provided an image information conversion apparatus for forming a plurality of output image signals having different scanning line structures from an input image signal. First data selecting means for selecting a plurality of first pixels of the image signal, and second data for selecting a plurality of second pixels of the input image signal located around a pixel to be generated with the output image signal Selection means, third data selection means for selecting a plurality of third pixels of the input image signal located around a pixel to be generated with the output image signal, and memory for storing the estimation equation coefficients obtained in advance Means, a plurality of first pixels selected by the first data selection means, a signal generation means for generating pixels of the output image signal by a linear estimation formula of the estimation equation coefficients, and a selection means by the second data selection means Multiple A class in which a space class is formed based on the second pixel, a motion class is formed based on the plurality of third pixels selected by the third data selection means, and the space class and the motion class are integrated. A class determining unit that supplies the estimated expression coefficients to the signal generating unit in accordance with the information; and a scan converting unit that is connected to the signal generating unit and converts the converted image into a specified scanning line structure. , The signal generating means multiplies a plurality of first pixels by a plurality of estimation formula coefficients, and adds the multiplication results, wherein the first pixel and the estimated An image information conversion apparatus characterized by having control means for setting one of the expression coefficients to 0.

[0007] When the number of input taps changes and a tap that suspends operation occurs, predicted tap data or coefficient data relating to the tap is set to zero. Further, a coefficient relating to tap data to be passed through is set to 1, and a coefficient relating to other tap data is set to 0. Thereby, desired tap data can be passed through.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 schematically shows a flow of a prediction process to which the present invention can be applied. The central part is a product-sum unit 313 which performs an operation represented by a linear combination of a prediction tap 311 and a prediction coefficient 312 cut out from an input image signal. Commands are provided to the system to set the functions of the system. Three
The encoder denoted by 14 generates a command interpretation and an instruction to set an internal state based on the result of the interpretation.

The n prediction coefficients 312 to be used in the prediction coefficient group 315 are selected by a prediction coefficient switching instruction 316. Further, n prediction taps 311 used in the filter taps 317 are set by the prediction tap switching instruction 318. Further, the input width of the accumulator 313 is n
It is. Number of prediction taps and number of prediction coefficients n
Is kept n or m by the input optimization processing 319.
(N> m). The input optimization processing 319 is performed according to an instruction based on the interpretation of the command formed in the encoder 314. Prediction calculation is performed by the product-sum unit 313, and the calculation output is processed by the gain adjustment and limiter 320 to generate a final prediction calculation output.

As described above, after the input optimization processing 319, if the number of the prediction taps 311 and the prediction coefficients 312 is equal to the input width n of the multiply-accumulator 313, the predictor 3
At 13, no inactive taps occur. However, in a conversion device that converts an image signal, as can be seen from an embodiment of an image signal conversion device described later, a pixel in an input image signal and a positional relationship between a pixel to be predicted and
Difficulty of prediction differs, and as a result, the input optimization processing 31
9 later, the prediction tap 311 and the prediction coefficient 312
May be m (n> m). On the other hand, in order to reduce the scale of hardware, it is preferable to use the product accumulator 313 in common. Therefore, when the number of taps is m,
The number of nm taps of the accumulator 313 suspends operation. Further, there is a case where the pixel value of a predetermined tap not multiplied by the prediction coefficient is desired to be output as it is.

FIG. 2 shows an example of a configuration conventionally used when any of the tap data is output without performing a prediction operation. Reference numeral 301 indicated by a broken line indicates a product-sum device. The accumulator 301 includes n multipliers 302 ₁ , 30
2 _2, ..., and 302 _n, multipliers 302 ₁ to 302 _n
And an adder 303 for adding the outputs of the above. Multipliers 302 ₁ to 30
2 _n as one input, registers R11, R12,.
.., coefficient data COEF1, COEF2,..., CO from R1n
EF _n are supplied respectively. Multipliers 302 _{1 to} 302 _n
, Registers R21, R22,.
-, tap data from R2n DT1, DT2, · · ·, the DT _n are supplied.

The output of the adder 303 of the accumulator 301 is supplied as one input to a selector 305 via a gain and limiter 304. The other input of the selector 305 is supplied with the output of the selector 306 via the delay circuit 307. The selector 305 selectively outputs one input. The selectors 306 include registers R21, R
22,..., Tap data DT1, DT2,.
, DT _n are supplied, and one data to be passed through is selected.

As shown in FIG. 2, in a configuration in which a signal line bypassing the product-sum device 1 is provided and a delay circuit 307 for a time corresponding to the delay generated in the product-sum device 301 is provided, the bypass signal line, the selector 306 and the delay It is necessary to add the circuit 307, and the circuit scale increases. Also, the input width and the number of taps of the accumulator 301 are usually selected to be equal, and no measure has been taken when the number of taps decreases from n to m.

An embodiment of the prediction arithmetic unit according to the present invention shown in FIG. 3 can solve these problems. As in the configuration shown in FIG.
and n number of multipliers 302 ₁ to 302 _n, multipliers 302 ₁ to
It is constituted by an adder 303 for adding the output of 302 _n, having n taps as the width of the input.

The outputs of the selectors S1, S2,..., Sn are supplied as one input of the multipliers 302 _{1 to} 302 _n . The selector S1, S2, ···, as one input of Sn, registers R11, R12, ···, coefficients from R1n data COEF1, COEF2, ···, COEF _n are supplied. As the other inputs of the selectors S1 to Sn,
Zero data is provided. The selectors S1 to Sn select coefficient data or zero data in response to the control signals CNT1 to CNTn, respectively. Further, the multipliers 302 _1-
As the other input of 302 _n , registers R21 to R2n
Tap data DT1, DT2 from, ..., is DT _n are supplied.

The input digital image signal is supplied to the prediction tap selection unit (not shown), n taps data DT1, DT2 to be used for prediction, ···, DT _n is taken at the same time. On the other hand, predetermined coefficient data COEF1, COEF2,
·, COEF _n is read from a memory unit (not shown).
The outputs of the multipliers 302 _{1 to} 302 _n are added in an adder 303, and a product-sum operation output is taken out from the adder 303. The output of the accumulator 301 is a gain and limiter 3
The signal is output as a prediction calculation output via the line 04. The gain and limiter 304 corrects the gain for the operation result and limits the number of bits of the operation result, and is controlled by the control signal cnt.

The operation of the embodiment of the present invention will be described. First, n tap data DT1, DT2,.
・, DT _n and n coefficient data COEF1, COEF2, ..., COEF
Linear primary coupling with _n (COEF1 × DT1 + COEF2 × DT2 +
· · + COEF _{when n} × DT _n) by forming a prediction calculation output, as the selector S1~Sn supplies the coefficient data to the multiplier 2 ₁ to 2 _n, respectively, the control signals CNT1
To CNTn control the selectors S1 to Sn.

Next, when a prediction operation output is generated by m (n> m) taps and m coefficient data, among the selectors S1 to Sn, m selectors select coefficient data, and The control signals CNT1 to CNTn are set so that the remaining nm selectors select zero data.
The selectors S1 to Sn are controlled by the selector. Zero data is, for example, a ground level. Therefore, the multiplier to which the zero data is input does not substantially perform the multiplication operation, thereby preventing wasteful power consumption. In addition, if necessary,
Gain and limiter 304 according to the difference in the number of taps
Is optimized by the control signal cnt.

Further, a case will be described in which a through operation is performed in which the pixel values of one or more input taps are output as they are, instead of the prediction calculation output. For example, tap data DT
When it is desired to output _n , the coefficient data COEF _n is set to 1, and the selector Sn is controlled by the control signal CNTn so that the selector Sn selects the 1 coefficient data. Therefore, the output of the multiplier 302 _n is the data DT _n .

Other selectors than the selector Sn are controlled by a control signal so as to select all zero data. Therefore, all outputs of the multipliers other than the multiplier 302 _n become zero data, and the adder 303 outputs data DT _n
Is output. The gain and limiter 304 sets the gain to 1 and outputs the number of input bits as it is,
It is controlled by the control signal cnt. In this way, the output can be obtained tap data DT _n. Since the through data passes through the accumulator 301, a delay circuit for time alignment is not required.

In the present invention, as shown in FIG. 4, a configuration in which AND circuits G1 to Gn are connected instead of the selectors S1 to Sn is also possible. That is, the AND circuits G1 to Gn
Are supplied with coefficient data COEF1 to COEFn as one input, and control signals CNT1 to CNTn as the other input.
Is supplied. When it is desired to prohibit the input of coefficient data to the multiplier, the corresponding one of the control signals CNT1 to CNTn may be set to “0”. Other configurations and operations are the same as those of the predictor shown in FIG.

In the structure shown in FIG. 3, selectors S1 to Sn are used to select coefficient data and zero data. In the structure shown in FIG. 4, AND circuits G1 to G1 are used to invalidate coefficient data. Gn is used. However, no processing may be performed on the coefficient data, and tap data and zero data may be selected or tap data may be set to zero data.

The above-described motion judging device according to the present invention comprises:
The present invention can be applied to generation of a motion class in an image signal conversion device. This image signal conversion device uses SD
(Standard Definition) signal is input and HD (High
Definition) Outputs a signal. Also, HD
When generating a pixel, it is in the vicinity of the HD pixel to be generated,
By classifying SD pixels into classes and acquiring prediction coefficient values for each class by learning, HD pixels closer to the true value are obtained.
Get the pixels. FIG. 9 shows an image signal conversion apparatus using such a method.

Before describing an example of the image information conversion apparatus, an image information conversion process as a premise thereof will be described. This process converts a standard resolution digital image signal (hereinafter, referred to as an SD signal) into a high resolution image signal (which may be referred to as an HD signal) and outputs the same. As the SD signal at this time, for example, an image signal of an interlace system (hereinafter referred to as 5
25i signal). As a high-resolution image signal, for example, an output video signal of a progressive system with 525 lines (hereinafter referred to as a 525p signal) or the like is used. Further, the number of pixels in the horizontal direction in the output image signal is twice the number of pixels in the horizontal direction in the input image signal.

In such image information conversion processing, the resolution is to be increased by the classification adaptive processing proposed by the present applicant. The classification adaptive processing is different from the one that forms a high-resolution signal by the conventional interpolation processing. That is, in the class classification adaptive processing, class division is performed according to the three-dimensional (spatio-temporal) distribution of the signal level of the input SD signal, and the prediction count value obtained by learning in advance for each class is stored in a predetermined storage unit. This is a process of outputting an optimum estimated value by an operation based on a prediction formula. Through the classification adaptive processing, it is possible to obtain a resolution higher than the resolution of the input SD signal.

FIG. 5 shows an example of the arrangement of pixels in an image information conversion process for converting a 525i signal as an input SD signal into a 525p signal as an output image signal by enlarging a part of an image of one field. It is shown. Large dots indicate pixels of the 525i signal, and
Small dots indicate pixels of the 525p signal. Here, FIG. 5 shows a pixel arrangement in an odd field of a certain frame. In other fields (even fields), 52
The line of the 5i signal is spatially shifted by 0.5 line. As can be seen from FIG. 5, the line data y1 at the same position as the line of the 525i signal (shown as small black dots) and the line data y2 at the middle position between the upper and lower lines of the 525i signal (shown as small white dots) Is formed to predict and generate a 525p signal.

Here, while y1 is an existing point, y2 is a point generated by newly predicting. Therefore, predictive generation of y2 is more difficult than predictive generation of y1. Therefore, y2
Needs to be assigned a larger number of classes than when predicting and generating y1. Further, for example, when the memory resources provided in the apparatus do not have a sufficient amount, or when performing efficient image information conversion processing using a smaller number of classes, the total number of classes may be reduced. It is necessary to perform the conversion of a simple class value. From these viewpoints, the image information conversion apparatus described below allocates the number of classes appropriately so that prediction and generation of y1 and y2 can be performed efficiently and finely.

Hereinafter, an example of such an image information conversion apparatus will be described with reference to the drawings as appropriate. An example of this is input S
The image information conversion processing for converting a 525i signal as a D signal into a 525p signal as an output image signal is performed. FIG. 6 is a block diagram illustrating an example of the configuration of the image information conversion processing system. The input SD signal (525i signal) is supplied to the tap selection circuits 1, 10, and 20.

The tap selection circuit 1 includes line data y1,
Calculation processing for predicting and estimating y2 (formula (1) described later)
Area including a plurality of pixels required for the calculation processing according to
SD pixels necessary for predicting and estimating 1, y2 (hereinafter, referred to as
(Referred to as prediction tap). The selected prediction tap is supplied to the estimation prediction calculation circuits 4 and 5. Here, as the prediction tap, the same as a space class tap described later can be used. However, in order to improve prediction accuracy, a prediction tap may be selected based on prediction tap position information corresponding to a class.

The estimation prediction calculation circuits 4 and 5 are supplied with prediction coefficients necessary for predicting and estimating the line data y1 and y2 from a coefficient memory 41 described later. Based on the prediction taps supplied from the tap selection circuit 1 and the prediction coefficients supplied from the coefficient memory 41, the estimation prediction calculation circuits 4 and 5 sequentially predictively generate pixel values y according to the following equation (1).

Y = w ₁ × x ₁ + w ₂ × x ₂ + ‥‥ + w _n × x _n (1) where x ₁ , ‥‥, x _n are each prediction tap,
w ₁ , ‥‥, and w _n are respective prediction coefficients. That is, Expression (1) is an expression for predictively generating a pixel value y using n prediction taps. As a column of pixel values y, the line data y1 is predicted and generated in the estimated prediction operation circuit 4, and the line data y2 is predicted and generated in the estimated prediction operation circuit 5. The prediction coefficients w ₁ , ‥‥, w _n are y1,
y2 are different from each other.

Here, as will be described later, five prediction taps are used to predict line data y1 (that is, n = 5), and six prediction taps are used to predict line data y2. (Ie, n =
6). Since the number of taps is different and the prediction coefficient is different, the two estimated prediction calculation circuits 4 and 5 are shown in FIG.
It is shown. These estimation and prediction calculation circuits 4 and 5
However, the above-described prediction operation unit according to the present invention can be applied. The prediction arithmetic unit according to the present invention can cope with the difference in the number of prediction taps, so that the hardware configuration can be the same. As described above, since the prediction arithmetic unit according to the present invention has versatility, the labor for designing an integrated circuit of the estimated prediction arithmetic circuit can be reduced.

The estimation prediction operation circuits 4 and 5 supply the line data y1 and y2 to the line order conversion circuit 6, respectively.
The line order conversion circuit 6 performs a line double speed process on the supplied line data to generate a high-resolution signal. This high resolution signal is used as the final output image signal of the image information conversion processing system. Although not shown, an output image signal is supplied to a CRT display. The synchronization system of the CRT display is configured so that an output image signal (525p signal) can be displayed. As the input SD signal, a broadcast signal or a reproduction signal of a reproduction device such as a VTR is supplied. That is, the image information conversion device can be built in a television receiver or the like.

On the other hand, the tap selection circuits 10 and 11 respectively select SD pixels (hereinafter, referred to as space class taps) required to detect a space class for the line data y1 and y2 from the input SD signal. . Also,
The tap selection circuit 20 selects an SD pixel (hereinafter, referred to as a motion class tap) motion class tap necessary for detecting a motion class from the input SD signal. The outputs of the tap selection circuits 10 and 11 are output to the space class detection circuits 12 and 1
3 and the output of the tap selection circuit 20 is supplied to the motion class detection circuit 21.

The space class detection circuits 12 and 13 respectively determine y1 and y2 based on the supplied space class tap.
Find the spatial class value for. Then, the detected space class values are supplied to the class synthesis circuits 30 and 31, respectively. The motion class detection circuit 21 detects a motion class value based on the supplied motion class tap, and supplies the detected motion class value to the class synthesis circuits 30 and 31.

The class combining circuits 30 and 31 combine the supplied space class value and motion class value. The outputs of the class synthesis circuits 30 and 31 are supplied to class value conversion circuits 32 and 33, respectively. Class value conversion circuits 32, 3
3 performs a class value conversion process described later on the outputs of the class synthesis circuits 30 and 31, respectively. The class value conversion process reduces the total number of classes.

The converted class value is supplied to the coefficient memory 41. The coefficient memory 41 stores prediction coefficients determined in advance by learning, which will be described later, and, among the stored prediction coefficients, the one specified by the class value supplied from the class value conversion circuits 32 and 33 and the estimated prediction calculation circuit 4 , 5 are output. To enable such an operation,
For example, a method of storing a prediction coefficient in accordance with an address specified by a class value obtained by class value conversion may be used.

Here, the space class detection will be described in more detail. In general, a space class detection circuit detects a spatial pattern of a level distribution of image data based on a level distribution pattern of a space class tap, and generates a space class value based on the detected spatial pattern. In this case, in order to prevent the number of classes from becoming enormous, a process of compressing 8-bit input pixel data into data of a smaller number of bits is performed for each pixel. As an example of such information compression processing, ADRC (Adaptive Dynamic Ra
nge Coding) can be used. Further, DPCM (predictive coding), VQ (vector quantization), or the like can be used as the information compression processing.

ADRC is originally designed for VTR (Video Tape Re-
This is an adaptive requantization method developed for high-efficiency coding for coder) .However, in this example, ADRC classifies ADRC as spatial class classification because local patterns at the signal level can be expressed efficiently with a short word length. Is used to generate the code. ADRC
Is the dynamic range of the spatial class tap as DR, the bit allocation as n, the data level of the pixel of the spatial class tap as L, and the requantization code as Q, by the following (2), the maximum value MAX and the minimum value MIN Are equally divided by the designated bit length and requantization is performed.

DR = MAX−MIN + 1 Q = {(L−MIN + 0.5) × 2 / DR} (2) where {} indicates a truncation process.

As described above, the number of classes for the line data y2 is larger than the number of classes for the line data y1. This point will be specifically described. First, FIG. 7 shows an example of an arrangement of space class taps used for space class detection for y1. FIG. 7 shows pixels in each field arranged in the vertical direction when time progresses in the horizontal direction (left to right).

The fields shown are denoted as F-1 / o, F-1 / e, F / o, and F / e in order from the left.
Note that F-1 / o is a notation indicating that it is "a field including the odd-numbered (odd) -th scanning line of the (F-1) -th frame". Similarly, F-1 / e means "a field consisting of even-numbered (even) -th scanning lines of the (F-1) -th frame", and F / o means "an odd-numbered scanning line of the F-th frame." F / e means “a field consisting of even-numbered scanning lines of the F-th frame”.

In one such example, the field F / o
Are used as space class taps for the line data y1 in the field F-1 / e, and T1, T2, and T3 in the field F / o are all five pixel values. However, the arrangement of the space class taps is not limited to this. For example, a plurality of input pixels in the horizontal direction may be used as space class taps.

Similarly, FIG. 8 shows an example of the arrangement of space class taps used for space class detection for y2. The notation in FIG. 8 is the same as that in FIG. In such an example, the field F-1 / e is used as a space class tap for the line data y2 in the field F / o.
, And T2, T3, T in the field F / o.
4, T5 and a total of 6 in T1 in field F / e
Pixel values are used. However, the arrangement of the space class taps is not limited to this. For example, a plurality of input pixels in the horizontal direction may be used as space class taps. Note that the same prediction taps as the space class taps shown in FIGS. 7 and 8, respectively, are used to predict line data y1 and y2.

Next, the motion class detection will be described in more detail. The motion class detection circuit 21 calculates an average value param of the inter-frame difference absolute value based on the supplied motion class tap according to the following equation (3). Then, a motion class value is detected based on the calculated param value.

[0046]

(Equation 1)

In equation (3), n is the number of motion class taps, and can be set to, for example, n = 6 (see FIG. 9). Then, by comparing the value of param with a preset threshold value, a motion class value that is a motion index is determined. For example, when param ≦ 2, the motion class value is 0. When 2 <param ≦ 4, the motion class value is 1. When 4 <param ≦ 8, the motion class value is 2. When param> 8, the motion class value. A motion class value is generated, such as class value 3. The motion class value 0 indicates that the motion is the minimum (still), and the motion class values 1, 2, and 3 are determined to be large as the motion becomes. Note that a motion vector may be detected, and a motion class may be detected based on the detected motion vector.

FIG. 9 shows an example of the arrangement of motion class taps used for motion class detection. In such an example, the motion class tap used when predicting the line data y1 and y2 in the field F / o is the field F / o.
Pixels n1, n3, n5 in / o and field F /
e, pixels n2, n4, n6 and field F-1
/ E in pixels m2, m4, m6 and field F
Pixels m1, m3, and m5 within −1 / o. Here, pixel m1 and pixel n1, pixel m2 and pixel n2,.
6 and the vertical position of the pixel n6 coincide with each other. However,
The arrangement of the motion class taps is not limited to this.

When classifying by 1-bit ADRC using the spatial class tap arrangement as shown in FIGS. 7 and 8, the motion class values are common to y1 and y2, and the number of motion class value classes Is 4, the number of classes is as follows.

The number of classes for y1 = ²⁵ × 4 = 128 The number of classes for y2 = ²⁶ × 4 = 256 In this case, the total number of classes is 384. However, for example, a memory resource provided in the image information conversion processing system (FIG. 6)
In the case where the coefficient memory 41) does not have a sufficient storage capacity to correspond to this number of classes, or when the image information conversion process is efficiently performed with a smaller number of classes, the total number of classes is reduced. There is a need to. Therefore,
By performing the class value conversion processing, the number of classes for y1 and y2 is reduced. FIG. 10 schematically shows the reduction of the total number of classes in this case. The provisional prediction coefficients in FIG. 10 will be described later.

Next, learning, that is, processing for setting an appropriate prediction coefficient will be described below. First, the first of learning
FIG. 11 shows an example of the configuration of the processing system related to the creation of the class value conversion table, which is the stage. A known signal (for example, a 525p signal) having the same signal format as the output image signal is converted to a thinning filter 51 and normal equation addition circuits 61 and 62.
Supplied to The thinning filter 51 generates an SD signal (for example, a 525i signal) having a number of pixels of で each in the horizontal direction and the vertical direction, and having 数 of the number of signals supplied as a whole.

As such processing, for example, pixels are decimated by a vertical decimating filter so that the vertical frequency of the input image signal is halved, and further, horizontal decimating is performed so that the horizontal frequency is halved. Processing such as thinning out pixels is performed by the filter. By changing the characteristics of the thinning filter 51, the characteristics of the learning can be changed, and thereby the image quality of the image obtained by the conversion can be controlled.

FIG. 12 shows a spatial relationship of pixels between a 525p signal as an example of an input SD signal to the thinning filter 51 and a 525i signal as an example of an output image signal from the thinning filter 51. In the odd-numbered fields of the 525p signal, even-numbered lines are thinned out, and pixels in the odd-numbered lines are alternately thinned out in the horizontal direction. The state of this thinning process is shown in FIG.
FIG. 2 illustrates the first field and the third field. On the other hand, in odd-numbered fields of the 525p signal, odd-numbered lines are thinned out, and pixels in even-numbered lines are alternately thinned out in the horizontal direction. FIG. 12 shows the state of the thinning process for the second field.

In FIG. 11, the interlaced image signal generated by the thinning filter 51 is supplied to the tap selection circuit 101,
110, 111 and 120. The tap selection circuit 101 selects a prediction tap that can be used in calculation of a normal equation described later, and supplies the selected prediction tap to the normal equation addition circuits 61 and 62. On the other hand, the tap selection circuits 110 and 111 respectively output the line data y
1 and y2 are selected, and the selected space class taps are respectively assigned to the space class detection circuit 1
12 and 113. Further, the tap selection circuit 12
0 selects a motion class tap, and supplies the selected motion class tap to the motion class detection circuit 121.

The space class detection circuits 112 and 113 compress the data of the space class tap by ADRC, and independently detect the space class values for y1 and y2. Further, the motion class detection circuit 121 detects a motion class value. The output of the space class detecting circuit 112 and the output of the motion class detecting circuit 121 are
0 is supplied. The output of the space class detection circuit 113 and the output of the motion class detection circuit 121 are supplied to a class synthesis circuit 131. Class synthesis circuits 130 and 131
Combines the spatial class value and the motion class value for y1 and y2, respectively. Class synthesis circuits 130 and 131
Are supplied to normal equation addition circuits 61 and 62.

The normal equation addition circuit 61 obtains data relating to a normal equation using a prediction coefficient for predicting y1 as a solution based on the input SD signal, the output of the tap selection circuit 101, and the output of the class synthesis circuit 130. To perform addition. Similarly, the normal equation adding circuit 62 outputs the input SD
Based on the signal, the output of the tap selection circuit 101, and the output of the class synthesis circuit 131, an addition is performed to obtain data relating to a normal equation having a solution of a prediction coefficient related to prediction generation of y2.

The normal equation adding circuits 61 and 62 supply the calculated data to the provisional prediction coefficient determining circuit 63, respectively. The tentative prediction coefficient determination circuit 63 performs a calculation process for solving a normal equation based on the supplied data, and calculates y1, y2
Tentative prediction coefficients for the prediction generation of. Then, the calculated temporary prediction coefficient is supplied to the proximity coefficient class integration processing unit 64. Here, the temporary prediction coefficient is referred to because the final prediction coefficient is determined as a result of class integration, as described later. The calculation process for solving the normal equation is the same as the calculation process in the prediction coefficient determination circuit 263 (see FIG. 13) described later.

The proximity coefficient class integration processing unit 64 determines the distance d between the provisional prediction coefficients to be supplied, for example, by the following equation (4).
Calculate according to

[0059]

(Equation 2)

Here, k _l and k _l ′ represent prediction coefficients of different classes. Also, l indicates the number of the prediction tap. Further, n represents the total number of prediction taps.

Further, the proximity coefficient class integration processing section 64
Compares the distance between the prediction coefficients between the classes obtained as described above with a predetermined threshold value, and integrates a plurality of classes whose distances are smaller than the threshold value into one class. By integrating classes, the total number of classes can be reduced. At this time, if the distances between a certain class and some other classes are both smaller than the threshold, the classes having the smallest distance may be integrated. The larger the above-mentioned predetermined threshold value is set, the more the number of classes can be reduced. Therefore, by varying the threshold value, the total number of classes obtained by the class value conversion processing can be adjusted.

At this time, an appropriate number of classes is assigned to y1 and y2 in accordance with conditions such as the storage capacity of a memory resource (coefficient memory 41 in FIG. 6) provided in the image information conversion processing system. Accordingly, it is possible to obtain an effect that the memory resources can be used more effectively, or the image information conversion processing can be efficiently performed with a smaller number of classes.

Then, the proximity coefficient class integration processing section 64
Supplies information on class integration to the class value conversion table generation unit 65. Class value conversion table generator 6
5 generates a class value conversion table based on the supplied information. This class value conversion table is supplied to and stored in the above-described class value conversion circuits 32 and 33 (see FIG. 6) and class value conversion circuits 81 and 82 (see FIG. 13) described later. And it is referred to in the operation of these components.

Next, the calculation process of the prediction coefficient as the second stage of the learning will be described in detail. FIG. 13 illustrates an example of a configuration of a prediction coefficient calculation processing system that calculates a prediction coefficient. A known signal (for example, a 525p signal) having the same signal format as the output image signal is supplied to the decimation filter 251 and the normal equation addition circuits 261 and 262. Thinning filter 2
51 indicates that the number of pixels in each of the horizontal direction and the vertical direction is 1
/ 2, and generates an SD signal (for example, a 525i signal) having 1/4 the number of pixels of the signal supplied as a whole.

As such processing, for example, pixels are decimated by a vertical decimating filter so that the vertical frequency of the input image signal is halved, and further horizontal decimating is performed so that the horizontal frequency is halved. Processing such as thinning out pixels is performed by the filter. By changing the characteristics of the thinning filter 51, the characteristics of the learning can be changed, and thereby the image quality of the image obtained by the conversion can be controlled. S generated by the thinning filter 251
The D signal is applied to the tap selection circuits 201, 210, 211, 22
0 is supplied. By changing the characteristics of the thinning filter 251, the characteristics of learning can be changed, thereby controlling the image quality of the image obtained by the conversion.

The tap selection circuit 201 selects a prediction tap, and adds the selected prediction tap to the normal equation addition circuit 26.
1, 262. On the other hand, the tap selection circuit 210,
The space class taps for y1 and y2 selected by 211 are supplied to space class detection circuits 212 and 213, respectively. Further, the prediction tap selected by the tap selection circuit 220 is supplied to the motion class detection circuit 221.

The space class detection circuits 212 and 213 respectively supply y1 and y1 based on the supplied space class tap.
Detect the space class value for y2. Further, the motion class detection circuit 221 detects a motion class value based on the supplied motion class tap. The output of the space class detection circuit 212 and the output of the motion class detection circuit 221 are supplied to the class synthesis circuit 230. Further, the output of the space class detection circuit 212 and the output of the motion class detection circuit 221 are supplied to the class synthesis circuit 231. The class combining circuits 230 and 231 combine class values for y1 and y2, respectively, and convert each class value to the class value conversion processing unit 8.
1, 82.

The class value conversion processing units 81 and 82 store the class value conversion tables generated as described above, and refer to the class value conversion tables to output the class synthesis circuits 230 and 231. Class value conversion is performed on them. Then, the result of the class value conversion is supplied to the normal equation adding circuits 261 and 262, respectively.

The normal equation addition circuits 261 and 262 calculate data used for a calculation process for solving a normal equation using the prediction coefficient as a solution. That is, the normal equation adding circuit 261 performs an adding process based on the input SD signal, the output of the tap selecting circuit 201, and the output of the class value converting circuit 81, and thereby solves the prediction coefficient related to the prediction generation of y1. Calculate the data required to solve the normal equation. The normal equation addition circuit 262 receives the input SD
By performing an addition process based on the signal, the output of the tap selection circuit 201, and the output of the class value conversion circuit 82, data necessary for solving a normal equation having a prediction coefficient related to prediction generation of y2 is calculated. I do.

The data calculated by the normal equation addition circuits 261 and 262 are supplied to the prediction coefficient determination circuit 263. The prediction coefficient determination circuit 263 performs a calculation process for solving a normal equation based on the supplied data, and calculates a prediction coefficient related to prediction generation of the line data y1 and y2. The calculated prediction coefficients are supplied to the coefficient memory 84 and stored.

The processing related to the determination of the prediction coefficient, that is, the processing performed by the normal equation addition circuits 261 and 262 and the prediction coefficient determination circuit 263 will be described in more detail below. First, the normal equation will be described. As described above, using the n prediction taps, the line data y1,
Each pixel constituting y2 can be sequentially predicted and generated by equation (1).

In equation (1), before learning, the prediction coefficients w ₁ , ‥‥, and w _n are undetermined coefficients. The learning is performed on a plurality of signal data for each class. When the total number of signal data is expressed as m, the following equation (5) is used according to equation (1).
Is set.

Y _k = w ₁ × x _k1 + w ₂ × x _k2 + ‥‥ + w _n × x _kn (5) (k = 1,2, ‥‥, m) When m> n, the prediction coefficients w ₁ , Since ‥‥ and w _n are not uniquely determined, the element e _k of the error vector e is defined by the following equation (6), and the prediction coefficient is set so as to minimize the error vector e defined by the equation (7). To be determined. That is, the prediction coefficient is uniquely determined by the so-called least square method.

E _k = y _k − {w ₁ × x _k1 + w ₂ × x _k2 + Δ + w _n × x _kn } (6) (k = 1, 2, Δm)

[0075]

(Equation 3)

As a practical calculation method for obtaining the prediction coefficient minimizing e ² in equation (7), partial differentiation of e ² with the prediction coefficient w _i (i = 1, 2 ‥‥) (8)) It is sufficient to determine each prediction coefficient w _i so that the partial differential value becomes 0 for each value of _i .

[0077]

(Equation 4)

A specific procedure for determining each prediction coefficient w _i from equation (8) will be described. When X _ji and Y _i are defined as in equations (9) and (10), equation (8) can be written in the form of a determinant of equation (11).

[0079]

(Equation 5)

[0080]

(Equation 6)

[0081]

(Equation 7)

Equation (11) is generally called a normal equation. Each of the normal equation addition circuits 261 and 262 includes a class value supplied from the class value conversion circuits 81 and 82, a prediction tap supplied from the prediction tap selection circuit 201, and a known image having the same signal format as the input image signal. Using the signal, the normal equation data, ie,
The values of X _ji and Y _i according to equations (9) and (10) are calculated.
Then, the calculated normal equation data is transmitted to the prediction coefficient determination unit 4.
Supply 3 Based on the normal equation data, the prediction coefficient determination unit 43 performs a calculation process for solving the normal equation according to a general matrix solution method such as a sweeping-out method, and calculates the prediction coefficient w.
Calculate _i .

The prediction coefficients generated as described above are calculated based on the data subjected to the class value conversion, whereas the tentative prediction coefficients described above with reference to FIG.
It is calculated based on data that has not been subjected to class value conversion. Except for this point, both are calculated by completely the same calculation processing as the solution of the normal equation. Therefore,
Processing related to the determination of the tentative prediction coefficient, that is, the normal equation addition circuits 61 and 62 and the prediction coefficient determination circuit 6 in FIG.
The processing performed by No. 3 is exactly the same.

That is, in FIG. 11, the class synthesis circuit 1
A normal equation adding circuit using a class value supplied from the prediction taps 30 and 131, a prediction tap supplied from the prediction tap selection circuit 101, and a known input image signal (for example, a 525p signal) having the same signal format as the output image signal. 61,
Each of 62 calculates normal equation data, that is, the values of X _ji and Y _i according to equations (9) and (10). Then, the calculated normal equation data is transferred to the provisional prediction coefficient determination unit 6.
Supply 3 The tentative prediction coefficient determination unit 63 calculates a tentative prediction coefficient by performing a calculation process for solving a normal equation according to a general matrix solution method such as a sweeping method based on the normal equation data.

As described above, the tentative prediction coefficients are calculated, the class value conversion table is generated based on the tentative prediction coefficients, and further, the prediction coefficients are calculated while performing the class value conversion with reference to the class value conversion table. Is performed.
As a result, the prediction coefficient memory 84 stores, for each class,
A prediction coefficient that can be estimated to be statistically closest to the true value is stored. The prediction coefficients stored in the prediction coefficient memory 84 are:
It is loaded into the prediction coefficient memory 41 in the image information conversion processing system described above with reference to FIG.

The image signal conversion processing system as shown in FIG. 9, the processing system for creating the class value conversion table as shown in FIG. 11, and the processing system for determining the prediction coefficients shown in FIG. May be provided separately, or a component may be commonly used by appropriately providing a switch or the like. However, when these processing systems are separately provided, it is necessary to use components having the same operation characteristics as components having the same function in order to ensure the effectiveness of learning. For example, tap selection circuits 1, 51,
201 need to have the same operating characteristics. When components are commonly used, the processing of the entire device becomes complicated due to switching or the like, but it can contribute to the reduction of the circuit scale, cost, and the like relating to the entire device.

The number of prediction taps output by the tap selection circuit 201 is larger than the number of prediction taps used in the image information conversion processing system. Therefore, the prediction coefficient determination unit 263 in FIG. 13 obtains more coefficients for each class. Among the prediction coefficients obtained in this manner, the prediction coefficients to be used are selected in order from the one having the largest absolute value.

Next, the line double speed processing performed by the line order conversion circuit 6 will be described. As described above, the horizontal period of the 525p signal generated by the estimation / prediction calculation circuits 4 and 5 is the same as the horizontal period of the 525i signal before the image information conversion processing is performed. The line order conversion circuit 6 has a line memory and performs line double speed processing for doubling the horizontal period. FIG.
4 shows the line double speed processing using an analog waveform. The line data y1 and y2 are simultaneously predicted and generated by the estimated prediction operation circuits 4 and 5.

The line data y1 includes a1, a2,
a3 # line is included, and the line data y2 includes lines b1, b2, b3 # in order. The line order conversion circuit 500 includes a line doubler (not shown) and a switching circuit (not shown). The line doubler compresses the data of each line by half in the time axis direction and supplies the compressed data to the switching circuit. The switching circuit alternately selects and outputs the supplied data. As described above, the line sequential output (a
0, b0, a1, b1,...) Are formed.

In the above configuration, the output pixel value (line data L1) on the existing line and the output pixel value (line data L2) on the creation line are created in a parallel configuration. On the other hand, adding memory,
If the operation speed of the circuit is high, the line data y
1 and y2 may be generated in order and the line double speed processing may be performed.

Further, an example of the above-described image information conversion apparatus is for converting a 525i signal into a 525p signal. However, the number of 525 lines is an example, and another number of lines may be used. For example, an output image signal including a line including a number of pixels other than twice the number of pixels in the horizontal direction in the input SD signal may be predicted and generated. More specifically, a 1050i signal as an output image signal, that is, an interlace-type image signal having 1050 scanning lines may be predicted and generated. FIG.
Shows an arrangement of pixels in a 525i signal and a 1050i signal by enlarging a part of an image of one field. Large dots indicate pixels of the 525i signal, and small dots indicate pixels of the 1050i signal. Note that what is indicated by the solid line in FIG. 15 is a pixel arrangement in an odd field of a certain frame. Lines in other fields (even-numbered fields) are spatially shifted by 0.5 lines as indicated by dotted lines, and pixels of line data y1 'and y2' are formed.

The prediction operation device according to the present invention is not limited to the image information conversion device using the classification adaptive processing, but may be applied to another device that generates a prediction value or an interpolation value by linearly combining a plurality of pixel data. Can be applied to

[0093]

According to the present invention, by setting one of the coefficient data and the tap data to zero data, the operation for the tap can be suspended, and the change in the number of taps can be flexibly handled. Further, tap data that has passed through the accumulator can be extracted without providing a bypass signal line and a delay circuit.

The image information conversion device using the prediction operation device according to the present invention is provided with a plurality of types of points of interest (specifically, located on the scanning line of the input image signal) having different positional relationships with the scanning line of the input image signal. The number of classes determined for each of y1 and y2 located between the scanning lines of the input image signal is determined in accordance with the positional relationship of the target point with respect to the scanning line of the input image signal (for example, prediction generation becomes more difficult). (E.g., assigning a larger number of classes to difficult y2).

Therefore, the number of classes can be set according to the storage capacity of the memory resources in the apparatus, so that accurate processing can be performed even when the storage capacity of the memory resources is small. In addition, it is effective to apply the present invention when performing efficient processing with a smaller number of classes.

Further, depending on the difficulty of predictive generation (specifically, between y1 and y2) due to the difference in the positional relationship between the point of interest and the scanning line of the input image signal, each point of interest is An appropriate number of classes will be assigned to them. For this reason, even when the storage capacity of the memory assets is limited, it is possible to perform fine and efficient image information conversion processing.

[Brief description of the drawings]

FIG. 1 is a schematic diagram schematically showing a predictor to which the present invention can be applied.

FIG. 2 is a block diagram of an example of a predictor used for reference of the description of the present invention.

FIG. 3 is a block diagram of an embodiment of a predictor according to the present invention.

FIG. 4 is a block diagram of another embodiment of the predictor according to the present invention.

FIG. 5 is a schematic diagram illustrating an example of an arrangement of pixels in an image information conversion process performed by an example of an image information conversion apparatus to which the present invention is applied;

FIG. 6 is a block diagram showing an example of a configuration of an image information conversion processing system in an example of an image information conversion device to which the present invention is applied.

FIG. 7 is a schematic diagram illustrating an example of a space class tap arrangement for line data y1 in an example of an image information conversion device to which the present invention is applied.

FIG. 8 is a schematic diagram illustrating an example of a space class tap arrangement for line data y2 in an example of an image information conversion device to which the present invention is applied.

FIG. 9 is a schematic diagram illustrating an example of a motion class tap arrangement in an example of an image information conversion device to which the present invention is applied.

FIG. 10 is a schematic diagram illustrating class integration in an example of an image information conversion apparatus to which the present invention is applied.

FIG. 11 is a block diagram illustrating an example of a configuration of a class value conversion table generation processing system in an example of an image information conversion device to which the present invention is applied.

FIG. 12 is a schematic diagram for explaining processing by a thinning filter used in a class value conversion table generation processing system and a prediction coefficient calculation processing system.

FIG. 13 is a block diagram illustrating an example of a configuration of a prediction coefficient calculation processing system in an example of an image information conversion device to which the present invention is applied.

FIG. 14 is a schematic diagram illustrating a line double speed process.

FIG. 15 is a schematic diagram illustrating an example of an arrangement of pixels in another example of the image information conversion device to which the present invention is applied.

[Explanation of symbols]

1 ... prediction tap selection circuit, 4, 5 ... estimation prediction calculation circuit, 32, 33 ... class value conversion circuit, 41
..Coefficient memory, 64 ... Proximity coefficient class integration, 65
... Class value conversion table generators, 81, 82 ...
Class value converting circuit, 18 and 19 ... class value conversion circuit, 301 ... product-sum unit, 302 ₁ to 302 _n ... multiplier, 303 ... adder, S1 to Sn ... selector, G1 to Gn: AND circuit

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Yasushi Tatehira 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Tsutomu Watanabe 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. 72 Inside Sony Corporation (72) Nobuyuki Asakura, Inventor 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Masashi Uchida 6-35, 7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni -Inside the Corporation (72) Inventor Takuo Morimura 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation Inside (72) Inventor Kazutaka Ando 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni Inside (72) Inventor Ken Ken Inoue 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Wataru Niizuma 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F Over-time (reference) 5C063 AC01 BA04 BA09 CA01 CA34

Claims (6)

[Claims] 1. A prediction operation device in which a plurality of sample data of a digital information signal is multiplied by a plurality of coefficient data, and a multiplication result is added to output a prediction value or an interpolation value. A predictive calculation device provided with control means for setting one of the values of sample data and coefficient data which are respectively input to the control unit to 0. 2. The method according to claim 1, further comprising setting a value of a part of the coefficient data or a part of the sample data input to each of the plurality of multipliers to 0.
And a prediction calculation device configured to set values of other parts to 1. 3. An image information conversion apparatus which forms a plurality of output image signals having different scanning line structures from an input image signal, wherein a plurality of output image signals located around a pixel where an output image signal is to be generated are provided. First data selection means for selecting a first pixel; second data selection means for selecting a plurality of second pixels of an input image signal located around a pixel to be generated with an output image signal; A third data selection unit that selects a plurality of third pixels of the input image signal located around a pixel where an image signal is to be generated; a memory unit that stores a previously acquired estimation formula coefficient; A plurality of first pixels selected by the first data selection unit and a signal generation unit that generates pixels of an output image signal based on the linear estimation expression of the estimation expression coefficient; and a selection unit selected by the second data selection unit. plural A space class is formed based on the second pixel, and a motion class is formed based on the plurality of third pixels selected by the third data selection means. Class determining means for supplying the estimation formula coefficients to the signal generating means in accordance with the integrated class information; and scanning for converting the converted image into a designated scanning line structure, connected to the signal generating means. Conversion means, wherein the signal generation means multiplies a plurality of the first pixels by a plurality of the estimation formula coefficients and adds a multiplication result. An image information conversion apparatus, comprising: a control unit that sets one of the input first pixel and the estimation formula coefficient to 0. 4. The image information conversion device according to claim 3, wherein a progressive output image signal is formed from the interlace input image signal. 5. The image information conversion device according to claim 3, further comprising generating an output image signal having twice as many pixels as the input image signal in the horizontal direction. 6. The prediction coefficient according to claim 3, wherein, when the pixel of the output image signal is generated by the linear estimation formula, an error between a generated value and a true value of the pixel is minimized. An image information conversion apparatus characterized in that the class information is previously obtained by learning for each class information.