JPH0856335A - Converting device for digital image signal - Google Patents

Abstract

PURPOSE:To improve the precision of prediction or clustering when an input digital image signal is converted by classification adaption processing into a digital image signal having moire pixels. CONSTITUTION:An SD signal of standard resolution is corrected through the classification adaption processing of a predictor 2A. The corrected SD signal is stored in a memory 6. A predictor 2B predicts HD pixels by using the corrected SD signal. The corrected SD signal and predicted HD pixels are stored in the memory 6. A predictor 2C predicts HD pixels by using the corrected SD signal and predicted HD pixels obtained in the previous stage. A predictor 2D predicts all HD pixels finally. An aimed pixel to be corrected or predicted and peripheral pixels used for the correction or prediction are corrected or predicted stepwise in the increasing order of their mutual distances. Consequently, the precision of the prediction or classification can be improved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image signal conversion apparatus capable of receiving an input digital image signal such as a digital television signal and outputting a digital image signal having a larger number of pixels.

[0002]

2. Description of the Related Art Taking a television signal as an example, a high resolution (referred to as HD) in which the resolution in the horizontal and vertical directions is approximately twice that of an existing standard resolution signal (referred to as SD). A signal is proposed. SD method and H
When the D system is mixed, there is a case where a so-called up-conversion process for converting an SD image signal into an HD image signal is necessary. For example, when an SD image signal is displayed on an HD monitor, such signal conversion is necessary.

Conventionally, a horizontal interpolation filter and a vertical interpolation filter are prepared, and an SD image signal is serially passed through these filters to form an HD image signal. Since the HD image signal thus obtained creates insufficient pixels by interpolation, it cannot have a resolution higher than that of the SD image signal before conversion.

As one method for solving such a problem, the applicant of the present application has proposed a signal conversion device which performs up-conversion by class classification adaptive processing. This is to predict the value of the HD pixel of interest by linear linear combination of the value of the SD pixel of the periphery and the coefficient, and in this case, it corresponds to the pattern of the level distribution of the SD pixels of the periphery of the HD pixel of interest. Classify and use the coefficient for each class. This coefficient minimizes the prediction error, and is obtained by learning using a standard image in advance.

[0005]

Here, SD pixel and H
The time relationship of D pixels will be described with reference to FIG. 14
Indicates a plurality of pixels aligned in the vertical direction in each of the kth field and the (k + 1) th field. Each field of HD pixels (shown by white dots) shown in FIG. 14A is thinned out line by line. As a result, the SD pixel (indicated by a black dot) shown in FIG. 14B is obtained. In FIG. 14B, the thinned-out line positions are indicated by x.
Indicate. The line of the (k + 1) th field in FIG. 14B is the same as the upper and lower lines of the same field and the previous k
It is formed by an interpolation process using the line at the same position in the second field.

Another example of a temporal relationship is shown in FIGS. 14C and 14D.
Shown in In this example, in the kth and (k + 1) th fields, the number of lines is thinned to half, and lines of the SD image are formed at positions where interlaced scanning is maintained.

The spatial positional relationship between the HD image and the SD image is shown in FIGS. 15A and 15B. FIG. 15A shows an array of HD pixels in the same field. 2 adjacent to each other in the vertical direction
One line is thinned out, and one pixel of two adjacent pixels is thinned out in the horizontal direction. As a result, an SD image having the pixel array shown in FIG. 15B is formed.

[0008]

As described above, when the SD image is formed from the HD image, the signal conversion apparatus uses the pixels existing in the SD image to perform the class classification adaptive processing to perform the HD pixel conversion. (X) is formed. In this case, the time difference and the spatial distance between the target HD pixel and a plurality of surrounding SD pixels used for prediction or classification are not in a fixed relationship.

Such a time difference and a difference in spatial distance between the target HD pixel and a plurality of peripheral SD pixels used for prediction or classification result in a difference in prediction or classification accuracy. Let In general, it can be said that prediction using SD pixels having a closer temporal difference or spatial distance, or classification is higher in accuracy than those using SD pixels having a farther distance. The previously proposed signal conversion device for class classification adaptive processing does not consider the fact that the accuracy of prediction is affected by such a time difference and a difference in spatial distance. As a result, high-precision prediction and low-precision prediction are mixed, and there is a problem that the quality of the converted HD image is insufficient as a whole.

Therefore, the object of the present invention is to improve the accuracy by predicting or classifying the original image data and the image data after conversion, which are closer to each other in spatial position, in order. An object of the present invention is to provide a possible digital image signal converter.

[0011]

According to a first aspect of the present invention, there is provided a digital image signal converting apparatus for converting an input digital image signal into a digital image signal having a larger number of pixels, the input digital image signal being supplied. Is
A classification circuit for designating the class of the pixel of interest,
When the value of the pixel of interest is created by a linear linear combination of the values and the coefficients of a plurality of pixels spatially and / or temporally adjacent to the pixel of interest included in the input digital image signal,
A coefficient generation circuit for generating a coefficient for each class so as to minimize the error between the created value and the true value of the pixel of interest, and a plurality of spatially and / or temporally neighboring pixels of the coefficient and the pixel of interest. And an arithmetic circuit for generating a predicted value of the pixel of interest by linear linear combination with the pixel value of the pixel of interest, and the pixel of interest is stepwise in order from the one having a smaller distance between the pixel of interest and the pixel used for prediction. It is a digital image signal conversion device characterized by making predictions.

According to a fourth aspect of the present invention, there is provided a digital image signal converting apparatus for converting an input digital image signal into a digital image signal having a larger number of pixels, the input digital image signal being supplied. , A classifying circuit for designating the class of the pixel of interest, and a representative value acquired by learning in advance is stored for each class, and a representative value corresponding to the class determined by the classifying circuit is output as the value of the pixel of interest. The digital image signal conversion device is characterized in that the pixel of interest is predicted step by step in the order of decreasing distance between the pixel of interest and the pixel used for prediction.

[0013]

The input digital image signal (SD signal) is supplied, and the output digital image signal (H
D signal) is formed. H that does not exist in the input signal
When creating a D pixel, the target pixel (the HD pixel of interest) is classified into classes based on the level distribution pattern of surrounding pixels. The prediction coefficient is obtained in advance for each class. The linear primary combination of the prediction coefficient and the surrounding pixels forms the value of the HD pixel of interest. The distance between the HD pixel of interest and the peripheral pixels used for prediction or classification is not constant. The HD pixels of interest are predicted in order from those with a short distance to those with a long distance. This makes it possible to improve the accuracy of prediction or classification.

According to the invention described in claim 4, the pixel value is obtained in advance for each class and is stored in the memory. The class of the HD pixel of interest is determined, and the value of the class is set as the value of the HD pixel of interest. As a result, the accuracy of classification can be improved. When using normalized values, the accuracy of the prediction is also improved.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, the number of lines is decimated by half according to the relationship shown in FIGS. 14A and 14B, and the number of pixels is decimated in the horizontal direction by the relationship shown in FIGS. 15A and 15B. It is an example.
Furthermore, in actual television images, the line spacing is generally larger than the horizontal pixel spacing.

In FIG. 1, the input terminal denoted by 1 is S
A D (standard resolution) digital video signal is supplied. Specifically, a transmission signal by broadcasting or the like, a reproduction signal from a VTR or the like is supplied to the input terminal 1. The value of each pixel is 8
It is represented by a bit code. The input SD signal is supplied to the predictor 2A and the classification circuit 3A. In this example,
Four-step prediction is performed in the order of increasing distance from the target pixel to be processed and the surrounding pixels used for correction or prediction. The predictor 2A performs the first stage prediction operation. In the first stage prediction, the input SD signal is corrected, and in the second, third and fourth stages, the prediction for creating the pixels of the HD signal is performed. The prediction results of the previous step can be used in the prediction and classification of each step.

When the SD signal is generated by the thinning process of the HD signal, the transmitted SD signal itself is also subjected to the filtering process for the thinning process, and as a result, the waveform is deteriorated. Therefore, the SD signal itself is corrected. That is, the classification circuit 3A classifies the target pixel of the SD signal to be corrected using the SD pixels around the target pixel. A class code designating the class of the pixel of interest is supplied to the memory 4A as an address. The prediction coefficient read from the memory 4A is supplied to the predictor 2A. Memory 4A
As will be described later, the prediction coefficient stored in advance is stored therein. This coefficient is needed to correct the value of the pixel of interest.

The predictor 2A generates a correction value for the pixel of interest by linear linear combination of the coefficient from the memory 4A and the values of surrounding SD pixels. Specifically, the predictor 2A includes a time series conversion circuit for synchronizing a plurality of SD pixels used for correction, a plurality of synchronized SD pixels, and a memory 4.
It is composed of a multiplier that multiplies each of the plurality of coefficients from A and an adder that aggregates the multiplication outputs. The corrected SD pixel is written in the memory 6 from the predictor 2A via the selector 5.

The read output of the memory 6 is supplied to the selector 7. The selector 7 outputs the read output to the predictors 2B and 2B.
It is possible to switch between C and 2D and a state applied to the classification circuits 3B, 3C and 3D, and a state of outputting to the output terminal 8. The selection operation of the selectors 5 and 7 and the write / read operation of the memory 6 are controlled by a control signal from the controller 9. The selector 5 includes predictors 2A, 2B,
Select the 2C and 2D outputs in order. After the result of the final prediction is written in the memory 6, an HD image composed of the corrected SD pixel and the created HD pixel is taken out from the memory 6 to the output terminal 8 via the selector 7.

The predictor 2A performs the first stage prediction (correction of SD pixels), and the predictor 2B performs the second stage prediction.
The predictor 2B receives the read output of the memory 6 through the selector 7 and generates HD pixels by the class classification adaptive processing. Basically, similarly to the predictor 2A, the classification circuit 3A, and the memory 4A, in the predictor 2B, the value of the HD pixel of interest is linearly linearly calculated from the prediction coefficient from the memory 4B and the values of surrounding SD pixels. Generated by combining. The prediction of HD pixels in the third stage is performed by the configuration of the predictor 2C, the classification circuit 3C, and the memory 4C.
The prediction of the HD pixel in the fourth stage is performed by the configuration of the predictor 2D, the classification circuit 3D and the memory 4D.

The prediction operation in this embodiment will be described with reference to FIG. 2 to 8, each pixel is shown by two types of circles having different sizes, and the larger circle is SD.
It represents a pixel, and the smaller circle represents an HD pixel. FIG. 2A shows a first stage correction operation performed by the predictor 2A, the classification circuit 3A and the memory 4A. The SD pixel of interest indicated by A has eight SD pixels a to h around it.
Is corrected using. That is, the pixels above and below and to the left and right of the SD pixel A of interest and the pixels located diagonally thereof are used for correction. When the first stage is completed, in FIG. 3A,
SD pixels are corrected as indicated by the diagonal lines. S after correction
The D pixel is stored in the memory 6 via the selector 5.

At the second stage, which is performed by the predictor 2B, the classification circuit 3B and the memory 4B, as shown in FIG. 2B, the left and right SD pixels c and d on the same line as the HD pixel B of interest and its upper line. , And the pixels e and f on the lower line thereof are used to generate the value of the HD pixel B. These SD pixels are all corrected in the first stage. When the first and second steps are completed, as indicated by hatching in FIG. 3B, SD
The pixels are corrected and HD pixels are created. Correction S
The D pixel and the created HD pixel are stored in the memory 6 via the selector 5.

At the third stage, which is performed by the predictor 2C, the classification circuit 3C and the memory 4C, as shown in FIG. 2C, the pixels a, b and c on the upper line of the HD pixel C of interest and the lower side thereof. The pixels d, e, f of the line are used to generate the value of the HD pixel C. SD pixels b and e
Has been corrected in the first stage, and HD pixel a,
c, d, and f are created in the second stage. When the first, second and third steps are completed, SD pixels are corrected and HD pixels are created, as shown by hatching in FIG. 3C. The corrected SD pixel and the created HD pixel are stored in the memory 6 via the selector 5.

In the fourth step performed by the predictor 2D, the classification circuit 3D and the memory 4D, as shown in FIG. 2D, the pixels a, b and c on the upper line of the HD pixel D of interest are on the same line. The value of the HD pixel D is generated using the left and right pixels d and e and the pixels f and h on the lower line. SD pixels a, c, f, and h are those corrected in the first stage, and HD pixels b, d, e, and g are those produced in the second and third stages. First,
When the second, third and fourth steps are completed, the SD pixel is corrected and the HD is corrected as shown by hatching in FIG. 3D.
Pixels are created. Corrected SD pixel and created HD
Pixels are stored in the memory 6 via the selector 5. As can be seen from FIG. 3D, the desired HD image is finally stored in the memory 6. This HD image is read from the memory 6 and taken out to the output terminal 8 via the selector 7.

The HD pixel to be created always belongs to any of the patterns of the second, third and fourth stages. HD
When predicting a pixel, as can be seen from FIG. 2, prediction is performed in order from a pixel closer to the pixel of interest to a pixel farther from it.
That is, as shown in FIG. 2B, the pixel B is first predicted using a plurality of pixels including the left and right SD pixels c and d having the shortest distance. Next, as shown in FIG. 2C, the pixel C is predicted using a plurality of pixels including the upper and lower SD pixels b and e having the next shortest distance. Then, as shown in FIG. 2D, the pixel D is predicted using a plurality of pixels including the SD pixels a, c, f, and h in the diagonal direction.

The correlation between adjacent pixels is strong, and the closer the distance between adjacent pixels is, the stronger the correlation is. Therefore, the closer the distance between the predicted pixel and the pixel used for prediction is, the higher the accuracy of prediction is. . Of course, the stepwise prediction in this embodiment uses the pixels predicted in the previous step. Therefore, as described above, it is possible to improve the accuracy of the prediction by performing the stepwise prediction starting from the one with a short distance.

The present invention can thus improve not only the accuracy of prediction but also the accuracy of classification. The classification is a process of determining the class of the pixel of interest based on the pattern of the level distribution of pixels around the pixel of interest. Therefore, when the created HD pixel is also used as a peripheral pixel used for classification, the accuracy of the prediction of the HD pixel is high, so that the classification can be performed with high accuracy. In addition, as described above, SD
Since the pixels are corrected, the accuracy of classification is improved by using the corrected SD pixels.

Several methods of classification will be described below. In this description, the peripheral pixels used when classifying the target pixel are the same as the peripheral pixels used for the correction or prediction described above. However, this relationship is not always necessary, and pixels used for correction or prediction and classification may be different. Further, the classification processing is slightly different between the SD pixel and the HD pixel to be created. First, classification of SD pixels will be described.

The classification circuit 3A classifies SD pixels into classes. As an example, the classification of SD pixels is
The class of this SD pixel of interest is determined based on the pattern of the level distribution of SD pixels near the SD pixel of interest. As shown in FIG. 4A, the pattern of the level distribution of the SD pixels (b, d, e, g) at the closest distances in the vertical and horizontal directions of the target SD pixel A is determined as a class. As an example, the average value Av of the four referenced pixels is obtained, and the surrounding pixels are compressed from 8 bits to 1 bit according to the magnitude relationship with the average value Av. That is, as shown in an example in FIG. 4B, `1 'is assigned to a value larger than the average value Av, and` 0' is assigned to a value smaller than the average value Av. FIG.
In the example of B, a 4-bit code of (0101) is obtained.

As the class code generated by the classification circuit 3A, not only the peripheral pixels but also the value A of the SD pixel of interest.
(Here, the reference code indicating a pixel also represents the value of the pixel itself.) Is used. For example, 5 bits obtained by adding 1 bit obtained by comparing the pixel value A with the average value Av can be used. In this case, the value A of the SD pixel of interest is set to ADRC instead of 1 bit for the SD pixel of interest.
It is also possible to use a combination of a quantized value of several bits compressed by. That is, the ADRC detects the dynamic range DR and the minimum value MIN of a plurality of pixels, subtracts the minimum value MIN from the value of each pixel, divides the value obtained by subtracting the minimum value by the dynamic range DR, and calculates the quotient as an integer. It is the process of converting.

For example, in the case of 1-bit ADRC, the maximum value MAX and the minimum value MI of the five pixels will be described.
N is detected, and the dynamic range DR (= MAX-M
IN) is calculated. The minimum value MIN is subtracted from the value of each pixel, and the value after removal of the minimum value is divided by the dynamic range DR. The quotient of this division is compared to 0.5,
In the above case, it is set to "1", and when the quotient is less than 0.5, it is set to "0". The 1-bit ADRC gives substantially the same result as comparing the above average value with the value of each pixel. In case of 2-bit ADRC, DR / 2
^The value after removal of the minimum value is divided by the quantization step size calculated in ² .

FIG. 5 shows another example of classification of SD pixels. This is the vertical direction (Fig. 5A), the diagonal direction (Fig. 5B,
5C) and the horizontal direction (FIG. 5D), the minimum absolute value of the difference is obtained. For two SD pixels adjacent in the vertical direction shown in FIG. 5A, the absolute value of the next difference is obtained. | A-d |, | b-A |, | A-g |, | e-h |

The minimum value among these absolute value differences is detected. A 2-bit code is assigned to each of the four absolute value differences, and 2 corresponding to the detection minimum value is used.
The bit code is selected. As shown in FIG. 5B, the absolute value of the next difference is obtained for two SD pixels that are adjacent to each other in the diagonally upward right direction. | B-d |, | c-A |, | A-f |, | e-g |

The minimum value among these absolute value differences is detected, and the 2-bit code corresponding to the detected minimum value is selected. As shown in FIG. 5C, the following absolute value of the difference is obtained for two SD pixels that are adjacent to each other in the diagonally upward left direction. | D-g |, | a-A |, | A-h |, | b-e |

The minimum value among these absolute value differences is detected, and the 2-bit code corresponding to the detected minimum value is selected. As shown in FIG. 5D, the following absolute value of the difference is obtained for two SD pixels adjacent in the horizontal direction. | A-b |, | d-A |, | A-e |, | g-h |

The minimum value among these absolute value differences is detected, and the 2-bit code corresponding to the detected minimum value is selected. Finally, (2 × 4 = 8) bits corresponding to the minimum value detected in each of the four directions are obtained. These 8 bits are used as a class code indicating the class of the SD pixel of interest. Furthermore, for these 8 bits, attention S
You may combine the value which compressed the value of D pixel itself by quantization.

The method of classifying the SD pixels is not limited to the two methods described above, and other classification is possible. For example, the following difference values D0 to D7 are calculated centering on the SD pixel of interest. D0 = A-b, D1 = A-a, D2 = A-d, D
3 = Af D4 = Ag, D5 = Ah, D6 = Ae, D
7 = A-c

The minimum value among these difference values D0 to D7 is detected. The direction in which this minimum value exists is indicated by a 3-bit direction code. A 7-bit class code is a combination of the 3-bit direction code, 1 bit indicating the polarity of the difference value, and the quantized value (for example, 3 bits) of the value of the target SD pixel A itself.

Next, the classification for creating HD pixels will be described. For example, the classification of HD pixels B (see FIG. 2B) created in the second stage processing will be described. The following classification method is also applicable to the classification of HD pixels (FIGS. 2C and 2D) created in the third and fourth steps of processing. However, in the second stage, the pixels that can be used for classification are the corrected SD.
In contrast to the pixels, in the third and fourth steps, not only the corrected SD pixels but also the HD pixels created in the previous step can be used for classification. Therefore, in the third and fourth steps, the same classification as that for the SD pixel described above can be applied to the HD pixel of interest.

As shown in FIG. 6A, six corrected SD pixels a to f exist around the target HD pixel B. The average value Av of these is calculated. Then, each pixel is compared with the average value Av, and a class code is generated according to the result. In the example shown in FIG. 6B, 6 bits of (011010) are generated.

Another example of classification of the target HD pixel B will be described with reference to FIG. As shown in FIG. 7A, the HD of interest
Around the pixel B, 4 pixels on the upper line, 4 pixels on the same line, and 12 pixels on the lower line are used.
Twelve pixels are corrected SD pixels. Then, the average value a ′ of the four pixels located on the left side and the upper left side of the target HD pixel B
(= 1/4 (a + c + g + i) is formed. Similarly, the average value c ′ (= 1/4 (c + e + i + k) of the four pixels located on the left side and the lower left side of the HD pixel B of interest is formed.
The average value b '(= 1/4 (b + d + h + j)) of the four pixels located on the right and upper right of the target HD pixel B is formed, and
Average value c of 4 pixels located on the right side and the lower right side of D pixel B
'(= 1/4 (d + j + f + 1) is formed.

These average values are obtained by estimating HD pixels by using SD pixels on the upper, lower, left and right sides. Therefore, at the HD pixel positions shown in FIG. 7B, average values a ′, b ′, c ′,
It is considered that the pattern has d '. This FIG. 7B
By applying the same idea as in FIG. 6 to the pattern shown in FIG. 6, a 10-bit class code can be formed.

Another example of classifying the HD pixel B of interest will be described with reference to FIG. In an array similar to the pixel array shown in FIG. 7A, two SD pixels adjacent to the target HD pixel B, for example, pixels c and d located on the left and right are defined, and the directionality of each of the pixels c and d is examined. FIG. 8A shows a state where the pixel c on the left side is used to check the directionality, and FIG. 8B shows a state where the directionality is checked using the pixel d on the right side.

As shown in FIG. 8A, with the pixel c at the center, left and right pixels (i, d), upper and lower pixels (a, e), diagonally upper pixels (g, b), diagonally lower pixels (k). , F), the next difference values D0 to D7 are calculated. D0 = c-a, D1 = c-g, D2 = c-i, D
3 = c-k D4 = c-e, D5 = c-f, D6 = c-d, D
7 = c-b

The minimum value among these difference values D0 to D7 is detected. The direction in which this minimum value exists is indicated by a 3-bit direction code, as shown in FIG. 8C. For example, when the difference value (D3) is the minimum, the direction code is (01
1). As shown in FIG. 8B, using the pixel d on the right side of the HD pixel B of interest, the next difference value D
0'-D7 'are formed. D0 ′ = d−b, D1 ′ = d−a, D2 ′ = d−
c, D3 '= d-e D4' = df, D5 '= dl, D6' = d-
j, D7 ′ = d−h

Then, the minimum value among the difference values D0 'to D7' is detected, and the direction in which the minimum value exists is indicated by a 3-bit direction code. A total of 6 bits of the direction code generated for the left pixel and the direction code generated for the right pixel are adopted as the class code of the HD pixel B of interest. If necessary, a code bit (2 bits) indicating the minimum polarity of the two difference values may be added to the class code to form a class code of 8 bits in total.

As a method of classifying the HD pixel of interest, a method other than the above-mentioned method can be used. For example, HD pixels can be classified by using a method similar to the classification of SD pixels as shown in FIG.

In the classification into SD pixels or HD pixels described above, the class of the pixel of interest can be determined with a small number of bits, in other words, a small number of classes. If the 8-bit data of the peripheral pixels is used as it is, the number of classes becomes enormous, and the memory capacity and the scale of hardware such as a memory control circuit become too large. The classification described above can solve such a problem.

Now, the predictor 2A generates a correction value by linear linear combination of the coefficient from the memory 4A and the values of the peripheral pixels. The predictors 2B, 2C and 2D have memories 4B and 4
HD pixels are created by linearly combining the prediction coefficients from C and 4D and the values of the perimeter pixels. Memory 4A ~
The coefficients respectively stored in 4D are obtained by learning in advance.

FIG. 9 shows the structure at the time of learning for determining the prediction coefficient stored in the memory 4A. The HD signal is supplied to the input terminal indicated by 11, and the thinning filter 12 forms the SD signal. The thinning filter 12 is
Thinning processing is performed in each of the horizontal direction and the vertical direction to form an SD signal having a pixel array as shown in FIG. 2A. The output signal of the thinning filter 12 is the time series conversion circuit 1
3 is supplied.

The time series conversion circuit 1 is connected to the input terminal 11.
4 are connected. Time series conversion circuits 13 and 14
Rearranges the data from raster scan order to block order. The output signal of the time series conversion circuit 13 is supplied to the coefficient determination circuit 15 and the classification circuit 16. Similar to the classification circuit 3A, the classification circuit 16 uses the surrounding pixels to determine the class of the SD pixel of interest. The class code from the classification circuit 16 is the coefficient determination circuit 1
5 and the memory 17 respectively.

The coefficient determining circuit 15 determines a prediction coefficient that minimizes the sum of squares of the error between the predicted value generated by linear linear combination and its true value. The original data supplied to the input terminal 11 passes through the time series conversion circuit 14 and the coefficient determination circuit 1
5 is supplied as the true value of the pixel of interest. The coefficient determination circuit 15 determines the best prediction coefficient by the least square method. The determined prediction coefficient is stored in the memory 17. The storage address is designated by the class code from the classification circuit 16.

The operation of determining the coefficient by software will be described with reference to FIG. First, step 4
Control of the process is started from 1, and in the learning data formation of step 42, learning data corresponding to a known image is formed. At the end of the data in step 43, if the processing of all the input data, for example, one frame of data has been completed, the control proceeds to the prediction coefficient determination of step 46, and if not completed, the control proceeds to the class determination of step 44.

The class determination in step 44 is performed by the above-mentioned S.
This is a step for forming a class code indicating a class by performing a class determination process for D pixels. In the normal equation generation in the next step 45, a normal equation described later is created. After the processing of all the data is completed from the end of the data in step 43, the control proceeds to step 46, and step 46
In determining the prediction coefficient, the coefficient is determined by solving the equation (8) described later using the matrix solution method. The predictive coefficient store in step 47 stores the predictive coefficient in the memory 17, and
At 8, the control of the learning process ends.

The processing of step 45 (normal equation generation) and step 46 (prediction coefficient determination) in FIG. 10 will be described in more detail. At the time of learning, the true value y of the target SD pixel is known. When the correction value of the SD pixel of interest is y ′ and the values of the surrounding pixels are x ₁ to x _n , the coefficient w _{1 for} each class
N-tap linear first-order combination by ˜w _n y ′ = w ₁ x ₁ + w ₂ x ₂ + ... + w _n x _n (1) is set. Before learning, w _i is an undetermined coefficient.

As described above, learning is performed for each class,
When the number of data is m, according to the equation (1), y _j ′ = w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (2) (where j = 1, 2, ...

When m> n, w _{1 to} w _n are not uniquely determined, so the elements of the error vector E are e _j = y _j − (w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (3) (However, j = 1, 2, ..., M) is defined, and a coefficient that minimizes the following expression (4) is obtained.

[0058]

[Equation 1]

This is a so-called least squares method. Here, the partial differential coefficient by w _i of the equation (4) is obtained.

[0060]

[Equation 2]

Since each w _i may be determined so that the equation (5) becomes 0,

[0062]

(Equation 3)

If a matrix is used as

[0064]

[Equation 4]

It becomes This equation is generally called a normal equation. If this equation is solved for w _i using a general matrix solution method such as a sweeping method, the prediction coefficient w _i is obtained, and this prediction coefficient w _i is stored in the memory 17 using the class code as an address.

Next, learning for obtaining the prediction coefficient stored in the memory 4B will be described. This is done by the configuration shown in FIG. In this case, the SD signal corrected using the prediction coefficient obtained as described above is used. That is, the HD signal is supplied to the input terminal 11, and the thinning filter 12 forms the SD signal. This SD signal is supplied to the classification circuit 16 via the time series conversion circuit 13, and the class code from the classification circuit 16 is supplied to the memory 17 as an address.

The memory 17 stores the coefficients acquired by the configuration of FIG. The coefficient and the SD signal from the time series conversion circuit 13 are supplied to the predictor 18.
Corrected SD pixels are obtained from the predictor 18. The predictor 18 performs the first stage prediction (correction) similarly to the predictor 2A. The corrected SD signal has waveform deterioration corrected as compared with the SD signal output from the thinning filter 12.

The corrected SD signal is supplied to the coefficient determination circuit 21 and the classification circuit 22 via the time series conversion circuit 19. The classification circuit 22 determines the class of the HD pixel B of interest in the pattern shown in FIG. 2B, similarly to the classification circuit 3B. The true value of the target HD pixel B is supplied to the coefficient determination circuit 21 via the time series conversion circuit 20.
Like the coefficient determination circuit 15 described above, the coefficient determination circuit 21 determines a prediction coefficient that minimizes the error between the true value and the predicted value of the HD pixel B of interest by the least square method. The determined prediction coefficient is stored in the memory 23.

The prediction coefficient stored in the memory 4C is shown in FIG.
This is for the target HD pixel C in the pattern shown in C. It is determined in the same manner as the prediction coefficient for the target HD pixel B described above. In that case, the corrected SD pixel and the HD pixel B predicted as described above are used. Furthermore, the prediction coefficient stored in the memory 4D is as shown in FIG.
This is for the target HD pixel D in the pattern shown in D. Even in this case, the corrected SD pixel and the predicted HD pixel B and C are used. For a description of the determination of prediction coefficients for these pixels C and D, see
Omit it to avoid duplication.

Although FIG. 10 shows a software structure for learning, learning can be performed by a hardware structure or a combination of software and hardware. Further, correction of SD pixels and prediction of HD pixels are not limited to linear linear combination using prediction coefficients, and the values themselves of these data may be created in advance by learning and stored in a memory. .

FIG. 12 is a flow chart for explaining a learning process for creating a data value, for example, an SD pixel value in advance. The control start step 51, the learning data formation step 52, the data end step 53, and the class determination step 54 are the same as the steps 41, 42, 43, and 44 in the learning for determining the prediction coefficient described above. Is the step of performing.

The representative value determining step 55 is a step of obtaining an average value of true values for each class and determining the average value as a representative value. That is, the representative value is obtained by dividing the cumulative value of the true value obtained in the learning process by the cumulative frequency. The method of obtaining such a representative value is called the center of gravity method. In addition, when obtaining the representative value, the cumulative amount of data increases when the data values themselves are accumulated, so the reference value in the block (the value for relatively defining the sizes of multiple pixels in the block And the minimum value MI
N, maximum value MAX, average value, etc.) and a value normalized by the dynamic range DR of the block may be obtained as a representative value.

That is, when the reference value of the block is B (for example, the minimum value of pixels in the block) and the dynamic range is DR, the normalized representative value G is G = (y−B) / DR Stipulated. In step 56, the determined representative value is stored in the memory, and the learning ends.

When the thus normalized value is obtained by learning, the configuration shown in FIG. 13 is used, for example, for SD pixel correction. As shown in FIG. 13, the SD signal from the time series conversion circuit 31 is supplied to the classification circuit 32 and the detection circuit 34. The normalized representative value is read from the address of the memory 33 designated by the class code from the classification circuit 32. In addition, the detection circuit 34
Detects a plurality of SD pixels around the target pixel, that is, the dynamic range DR of the block and the minimum value MIN.

The normalized representative value from the memory 33 is supplied to the multiplication circuit 35, and the normalized representative value is multiplied by the detected dynamic range DR. The output of the multiplication circuit 35 is supplied to the addition circuit 36, and is added to the detected minimum value MIN. The output signal of the adder circuit 36 is the correction value.

When using the representative value obtained by the centroid method, it cannot be said that the accuracy of the prediction is improved by the stepwise prediction. However, the accuracy of classification can be improved. When the normalized representative value is used, the dynamic range DR of the block and the minimum value MI
N etc. are required. Therefore, when the block includes pixels that have been corrected or predicted in the previous stage, the accuracy of these corrections or predictions is improved, so that the accuracy of prediction and classification can be improved.

Further, for classifying or predicting calculation in the present invention, the value of the pixel around the pixel of interest is not limited to being used spatially, but a pixel close to the pixel of interest in the time direction (for example, the same pixel in the previous frame). Pixels) can also be used.

Further, in the above-described embodiment of the present invention,
Although separate configurations are provided for prediction of each stage, these configurations can be common.

[0079]

According to the present invention, since the target pixel is predicted in a stepwise manner from the shortest distance to the farthest pixel between the target pixel and the peripheral pixels used for prediction, the accuracy of prediction or classification is improved. be able to. Therefore, the quality of the converted digital image signal can be improved.

Further, in this embodiment, not only the HD pixel but also the SD pixel value is corrected, so that the high frequency component lost by the filtering process can be compensated. Therefore, it is possible to compensate the rounding of the waveform of the converted signal,
The quality of the output image can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram for explaining an operation of stepwise prediction of SD pixels.

FIG. 3 is a schematic diagram for explaining an operation of stepwise prediction of SD pixels.

FIG. 4 is a schematic diagram for explaining an example of a method of classifying SD pixels.

FIG. 5 is a schematic diagram for explaining another example of classification of SD pixels.

FIG. 6 is a schematic diagram for explaining an example of a method of classifying HD pixels.

FIG. 7 is a schematic diagram for explaining another example of classification of HD pixels.

FIG. 8 is a schematic diagram for explaining still another example of classification of HD pixels.

FIG. 9 is a block diagram of an example of a configuration at the time of learning for obtaining a prediction coefficient regarding SD pixels.

FIG. 10 is a flowchart when learning for obtaining a prediction coefficient is performed by software processing.

FIG. 11 is a block diagram of an example of a configuration at the time of learning for obtaining a prediction coefficient regarding HD pixels.

FIG. 12 is a flowchart when learning for obtaining a representative value is performed by software processing.

FIG. 13 is a block diagram of an example of a configuration for generating an interpolation value from a normalized representative value.

FIG. 14 is a schematic diagram for explaining a thinning process of the number of lines.

FIG. 15 is a schematic diagram for explaining a pixel number thinning process.

[Explanation of symbols]

 2A to 2D Predictor 3A to 3D Classifying circuit 4A to 4D Memory storing prediction coefficients

Claims (7)

[Claims] 1. A digital image signal conversion device for converting an input digital image signal into a digital image signal having a larger number of pixels, wherein the input digital image signal is supplied and a class for indicating a class of a pixel of interest. The value of the pixel of interest is created by a dividing means and a linear linear combination of the values of a plurality of pixels spatially and / or temporally adjacent to the pixel of interest included in the input digital image signal and the coefficient. Sometimes, coefficient generating means for generating a coefficient for each class so as to minimize an error between the created value and the true value of the pixel of interest, and the spatial and / or spatial relationship between the coefficient and the pixel of interest. And a calculation unit for generating the predicted value of the pixel of interest by linear linear combination with the values of a plurality of temporally neighboring pixels. That in order of distance to the pixel is small, the conversion device of the digital image signal, characterized in that predicting the target pixel in stages. 2. The digital image signal conversion apparatus according to claim 1, wherein the coefficient generating means determines the coefficient by a least square method. 3. The digital image signal converting apparatus according to claim 1, wherein when a pixel of interest is predicted stepwise, a plurality of pixels used for prediction or classification are predicted in the previous step. A device for converting a digital image signal, comprising one or more pixels. 4. A digital image signal converting apparatus for converting an input digital image signal into a digital image signal having a larger number of pixels, wherein the input digital image signal is supplied and a class for indicating a class of a pixel of interest. Classifying means, and a memory means for storing a representative value acquired by learning in advance for each class, and outputting the representative value corresponding to the class determined by the class dividing means as the value of the pixel of interest. A digital image signal conversion device, comprising: step-by-step prediction of the target pixel in order of increasing distance from the target pixel to the pixel used for prediction. 5. The digital image signal conversion apparatus according to claim 4, wherein when a pixel of interest is predicted stepwise, a plurality of pixels used for prediction or classification are predicted in the previous step. A device for converting a digital image signal, comprising one or more pixels. 6. The digital image signal conversion apparatus according to claim 4, wherein the representative value stored in the memory means is a value obtained by averaging the true value of the pixel of interest given during learning. For converting digital image signals. 7. The digital image signal converting apparatus according to claim 4, wherein the representative value stored in the memory means is a reference value of a plurality of pixels in a block including a target pixel, and a dynamic range of the block. According to the above, the digital image signal conversion device is a value obtained by normalizing the true value of the pixel of interest.