JPH1166298A - Interpolation processor of digital image signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make an edge sharp and to improve apparent resolution by executing an in-area average difference correction processing. SOLUTION: The interpolation processor of a digital image signal is constituted so as to improve appearance resolution by correction a signal which is interpolation-processed. Here, an interpolation processing 1 is executed by considering an interpolation sampling phase, so as to permit an area where the sample point of the signal before the interpolation processing presents (range in the horizontal direction and/or the longitudinal direction of the image) to match with the area where plural sample points corresponding to the area after the interpolation processing represent in most of the parts of the both areas, and correction processings 2 and 3 are executed so as to make the average 4 of a sample value in the area after the interpolation processing equal to the sample value before the interpolation processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image signal interpolation processing device, and more particularly, to a digital image signal up-conversion processing device.

[0002]

2. Description of the Related Art The interpolation processing of a digital image signal is realized by digital signal processing theory. As a method of performing the interpolation processing, any filter that satisfies the conditions of the interpolation filter may be used. For example, a nearest neighbor interpolation method, a bilinear interpolation method (nearest neighbor interpolation), For the famous interpolation filters such as bi-linear interpolation and cubic convolution interpolation, see "Image Analysis Handbook (First Edition)" (Takagi and Shimoda, published by The University of Tokyo Press, pp.441)
−444, issued on Jan. 17, 1991).

[0003]

The interpolation processing of the digital image signal is realized by performing an interpolation filter processing on the image signal by, for example, the above-mentioned method. However, the interpolation filter is realized with a finite number of taps. As far as possible, the characteristics of the passband do not become the ideal characteristics, but the high-frequency components are attenuated, and the resolution of the image is reduced. In order to solve this problem, conventionally, in order to improve the resolution, the image after the interpolation processing is separately and uniformly enhanced independently of the interpolation processing.

[0004]

SUMMARY OF THE INVENTION The present invention relates to a digital image signal interpolation processing device which is configured to improve the apparent resolution by correcting a signal subjected to interpolation processing. is there. Here, as a correction method, an area (a range in the horizontal direction and / or the vertical direction of the image) represented by the sample points of the signal before the interpolation processing, and a plurality of samples after the interpolation processing corresponding to the area. Interpolation processing is performed in consideration of the interpolated sample phase so that the area represented by the point coincides with the majority of both areas, and the average of the sample values in the area after the interpolation processing is calculated. The correction processing is performed so as to be equal to the previous sample value.

That is, the digital image signal interpolation processing device of the present invention is a digital image signal interpolation processing device, wherein an area represented by a sample point of an input digital image signal before interpolation and a plurality of samples after interpolation are provided. Interpolation processing means for interpolating the input digital image signal so that the area represented by the point is substantially the same in both areas; and the interpolation processing means corresponding to the area represented by the sample point of the input digital image signal. Average value detection means for detecting an average value of the sample values of the plurality of sample points after the interpolation processing; and the corresponding internal signal detected by the average value detection means from the sample values of the sample points of the input digital image signal. Subtraction means for subtracting an average value of sample values of a plurality of sample points after the interpolation processing; and the interpolation processing which is an output of the interpolation processing means. Adding means for adding an output corresponding to each area in the output of the subtraction means to each sample value of a plurality of subsequent sample points, including the sign of the output, and adding means. is there.

Further, the present invention provides a digital image signal interpolation processing apparatus, comprising: a digital image signal interpolation processing apparatus comprising: an area represented by sample points of an input digital image signal before interpolation; and a plurality of samples after interpolation. Interpolation processing means for interpolating the input digital image signal so that the area represented by the point is substantially the same in both areas; and the interpolation processing means corresponding to the area represented by the sample point of the input digital image signal. Average value detecting means for detecting an average value of sample values of a plurality of sample points after the interpolation processing,
Subtraction means for subtracting the sample value of the sample point of the input digital image signal from the average value of the sample values of the corresponding plurality of sample points after the interpolation processing detected by the average value detection means; and Adding means for inverting the sign of the output and adding an output corresponding to each area in the output of the subtraction means to each sample value of the plurality of sample points after the interpolation processing, which is an output of the processing means, It is characterized by comprising.

Further, in the digital image signal interpolation processing apparatus according to the present invention, the interpolation processing means is an interpolation processing means for performing a 1: 2 interpolation processing in a horizontal and / or vertical direction of the image. It is a feature.

[0008]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on embodiments of the present invention with reference to the accompanying drawings. Although the sampled value of the digital image signal is theoretically considered as a value of one point in a space having no area, it is actually a value sampled by a sensor (for example, a CCD) having a small area. . In the present invention, this idea is extended to consider that the sample value of the digital image signal is an average value in an area surrounded by a line segment equally dividing each sample point.

As described above, the digital image signal interpolation processing apparatus according to the present invention operates based on the principle that an area represented by a sample point of a signal before interpolation processing and an area corresponding to that area after interpolation processing are used. The area represented by a plurality of sample points (for example, in the case of 1: 2 interpolation, the sum of the areas represented by two adjacent sample points after interpolation) matches in most of both areas. Thus, interpolation processing is performed in consideration of the interpolation sample phase, and correction is performed so that the average of the sample values after the interpolation processing in the area becomes equal to the sample value before the interpolation processing. .

In the above, depending on the interpolation ratio and the sample phase before the interpolation processing, the area represented by the sample points before the interpolation processing and the area represented by a plurality of sample points after the interpolation processing are completely completed. Will not match. In that case, for the sample points after the interpolation process, of all the areas represented by all the sample points after the interpolation process included in the area represented by the sample points before the interpolation process, At the sample phase where the ratio of the part completely included in the area represented by the sample point is as large as possible, that is, the area represented by the sample point before the interpolation processing and the sample included therein after the interpolation processing Interpolation is performed at the sample phase that minimizes the difference from the entire area represented by the point, and represents the area after the interpolation that is almost completely included in the area represented by the sample point before the interpolation. The in-area average difference correction processing described below is performed on the sample points. Here, the sample phase after the interpolation processing is determined in advance based on the positional relationship between the sample points before and after the interpolation processing.

FIG. 1A shows sample points of an input digital image signal (each sample point before the interpolation processing; in the drawing, interpolation processing is omitted and only interpolation is described). In order to facilitate understanding, it is assumed here that the image is sampled only in the horizontal direction, and the horizontal axis is the horizontal position of each sample point (in the drawing, only the horizontal position is indicated. The vertical axis represents the magnitude, that is, each sample value in an analog manner.

Next, the interpolation process will be described. As the most typical example of the interpolation processing, an interpolation processing as shown in FIG. The interpolation processing shown in FIG. 1B calculates the signal magnitude at each center point of the horizontal position of each sample point in FIG. 1A by, for example, averaging the sample values of both adjacent sample points. This point (Fig. 1
(Indicated by a cross in (b)) and each sample point before interpolation (○
(Indicated by a mark) together with the sample points after the interpolation process.

However, in the interpolation processing shown in FIG.
Each sample point (for example, A,
B, C, etc.) are considered to be points representing areas indicated by double-headed arrows such as A, B, C, etc. below the horizontal axis in FIG.
The size of the representative area of each sample point indicated by a cross in (b) is half that of FIG. 1 (a), but the area covered by a pair of adjacent sample points of a circle and a cross is the inside. The sample point before the insertion processing (see FIG. 1A) does not coincide with the area (the area in FIG. 2) which is covered.

In the digital image signal interpolation processing apparatus of the present invention, as described above, the interpolation processing means includes an area represented by the sample points of the input digital image signal before the interpolation processing and an input digital image signal. The present invention cannot be configured unless the input digital image signal is subjected to interpolation processing so that the area represented by the plurality of sample points after interpolation matches in most of both areas. The interpolation processing described above with respect to (b) cannot be applied to the present invention.

The interpolation processing which can be applied to the present invention
As an example, the interpolation (1: 2 interpolation) shown in FIG.
I can do it. The interpolation processing shown in FIG.
(B) has the same number of sample points,
The position is shown at each sample point in FIG.
Interpolation point). Was
However, in the interpolation processing shown in FIG.
Pull points do not constitute sample points after interpolation.
You. FIG. 3 shows a sample obtained by performing the interpolation processing shown in FIG.
Which area the pull point is representative of
In order to show this, A below the horizontal axis of FIG.₁, A_Two, B ₁,
B_Two, C₁, C_TwoAnd the like. 2 and 3
As shown in the figure, the input digital image signal before interpolation is processed.
Areas A, B, and C represented by sample points are input digital
Multiple (two in this example) after interpolation processing to the total image signal
Area representative of the sample points of₁, A_Two, B₁,
B_Two, C₁, C_Two(For example, A becomes A₁, A_TwoMatches
Meaning). It should be noted that FIG.
The interpolation filter that performs the interpolation process has a delay of multiple delay elements.
By adjusting the amount and the coefficients of the interpolation filter.
Can be manifested.

Next, focusing on the area of the sample point A in FIG. 2, the sample points corresponding to the area of the sample point A in FIG. 3 are A ₁ and A ₂ . According to FIG. 2, in the area corresponding to the sample point A, the size per unit area is A
However, in practice, the average magnitude of the sample points A ₁ and A ₂ in FIG.
(Smaller than the size of sample point A). Therefore, in the present invention, the size of the sample point A and the sample point A are set so that the average of the size of the sample points A ₁ and A ₂ in FIG. 3 becomes equal to the size of the sample point A in FIG.
_1, performs a process for adding to each sample value of the sample points A _1, A ₂ the difference of the average size of the A _2. Hereinafter, this process is referred to as an intra-area average difference correction process.

This is, specifically, the sampling point A in FIG.
The average of the sizes of the sample points A ₁ and A ₂ in FIG. 3 is subtracted from the size of the sample points A ₁ and A ₂ , and the subtraction result (the difference amount) is added to the sample values of the sample points A ₁ and A ₂ in FIG. Or-)
This is performed by adding together. Further, in the above, the size of the sample point A in FIG. 2 may be subtracted from the average of the size of the sample points A ₁ and A ₂ in FIG. 3, but in this case, The sign (+ or-) of the subtraction result (difference amount) is inverted and added to the sample values of the sample points A ₁ and A ₂ in FIG.

FIGS. 4A to 4C show the operation of the intra-area average difference correction processing according to the present invention, which will be described below. Also in this figure, the horizontal axis indicates the horizontal position and the vertical axis indicates the signal magnitude. First, FIG.
The waveform shown in (a) is the original analog signal waveform. This signal is sampled in the horizontal one-dimensional space at the phase shown in FIG. 1A, and then the analog signal waveform restored from the signal subjected to the interpolation processing at the phase shown in FIG.
(B). It should be noted that the waveform shown in FIG. 4B also theoretically matches the analog signal waveform restored from the sample point of the phase shown in FIG. On the other hand, the signal waveform obtained by performing the intra-area average difference correction processing on the sample points after the interpolation processing is schematically shown in FIG.
FIG. 4C shows that the signal has a high-frequency component closer to the original signal than the signal waveform shown in FIG. 4B (the signal waveform before the intra-area average difference correction processing). In other words, the edge is sharpened by the mean difference correction processing in the area, and the apparent resolution is improved.

FIG. 5 shows the interpolation of the digital image signal of the present invention.
One embodiment of the processing device is shown in a block diagram. input
Digital image signal (sampled as shown in FIG.
Signal) is, for example, an interpolation processing of 1: 2 in the horizontal direction.
The signal is supplied to the insertion processing unit 1 and the subtracted terminal of the subtractor 2,
From the output side of the processing unit 1, interpolation processing as shown in FIG.
A controlled signal is obtained. This interpolated signal is 2
One is supplied to one of the input terminals of the adder 3
However, the other is supplied to the average value detection unit 4. Average value detection
In part 4, A in FIG.₁, A_TwoAverage value of B₁, B_Twoof
Average, C₁, C_TwoThe average of _----2 sequentially
The average value of the sample values is calculated and obtained. I was asked
The average value (the output of the average value detection unit 4) is a subtraction terminal of the subtractor 2.
From the size of the sample point A in FIG.
A₁, A_TwoSubtraction of the average of the magnitudes of

Next, in the adder 3, as described above, one of its input terminals is supplied with the output of the interpolation processing unit 1, while the other input terminal is provided with the subtractor 2 of the subtractor 2. Outputs are supplied and signals are added to each other. In this case, the subtraction output of the subtracter 2 including the sign (+ or-) is supplied to the other input terminal of the adder 3, and as a result, the average of the sample values in the area after interpolation is obtained. Becomes equal to the sample value before interpolation.

In another embodiment of the device of the present invention, the signals supplied to the subtracted terminal and the subtracted terminal of the subtractor 2 (see FIG. 5) may be interchanged. In this case, the subtraction output of the subtracter 2 has its sign (+ or-) inverted and the other input terminal of the adder 3 (see FIG. 5) (one or the other expression is the same as that of FIG. 5). Shall be supplied).

In the above description, in order to facilitate understanding of the present invention, an image is sampled only in the horizontal direction (one-dimensional space). The digital image signal interpolation processing apparatus according to the present invention when configured for a dimensional space will be described.

FIGS. 6A and 6B show input digital image signals (FIG. 6) sampled in a two-dimensional space using a square lattice.
(A)), the sample points before (FIG. 6 (a)) and after (FIG. 6 (b)) the interpolation processing are performed when the interpolation processing of 1: 2 is performed in both the horizontal and vertical directions. The symbol before the insertion process is indicated by 、, and the value after the interpolation process is indicated by ×. In FIG. 6A, the area represented by the central sample point is surrounded by a square frame, and a plurality of (four) sample points X after interpolation processing corresponding to the area are represented by circles in FIG. 6B.
It is shown enclosed by. Thus, the signal subjected to the interpolation processing has four times as many sample points as the input signal. Also here, the sum of the areas represented by the four sample points after the interpolation processing and the area represented by the sample points before the interpolation are to be subjected to the interpolation processing so as to match in most of both areas. I do. Such interpolation processing can be easily realized by appropriately setting the delay amount of a general digital filter and the coefficients of the interpolation filter.

FIGS. 7A and 7B show an example in which the sample points before the interpolation processing are line-offset. In this example, the area represented by the line-offset sample points before the interpolation processing indicated by the mark ○ and the entire area represented by the corresponding plurality of sample points after the interpolation processing indicated by the mark 完全 are completely It is assumed that the sample phase after the interpolation processing is selected so as not to match, but to minimize the difference.

Next, in performing the intra-area average difference correction processing, the average value detection section (corresponding to the average value detection section 4 in FIG. 5) performs interpolation processing belonging to the area represented by the sample point of the input digital image signal. The average of four sample values of the output signal is calculated, and the difference detection unit (corresponding to the subtracter 2 in FIG. 5) outputs the sample value of the input digital image signal and the output signal of the average value detection unit corresponding to that point. (Average of four sample values) is calculated.

Further, the difference correction unit (corresponding to the adder 3 in FIG. 5) adds or subtracts the average difference in the area by the difference amount obtained by the difference detection unit to the interpolated four sample values. Correction is performed by subtraction (addition or subtraction depending on the configuration of the difference detection unit), and the sample values of the input digital image signal and the average of the sample values of the interpolation-processed sample points in the area corresponding to those sample points are calculated. To eliminate the difference.

The present inventors have calculated the average of four sample values in two horizontal pixels and two vertical pixels of a luminance signal, and obtained a digital image having a pixel size of 1/4.
A comparison between the improvement of the conventional interpolation processing (perform uniform enhancement after the interpolation processing) and the result of performing the average difference correction processing in the area according to the present invention by computer simulation shows that the present invention has much better resolution. Was obtained.

As described above, one-dimensional space (only in the horizontal or vertical direction) and two-dimensional space (in both the horizontal and vertical directions)
The description of the configuration and operation of the digital image signal interpolation processing apparatus of the present invention is finished for the case (1), but by adopting the above configuration, depending on the magnitude of the interpolation signal interpolated by the interpolation processing unit, Overflow or underflow may occur in the output (the signal corrected for the average difference within the area). Finally, a method of correcting this will be described.

As an example of a method of correcting overflow or underflow, in the case of the above-described two-dimensional space, for example, when an overflow occurs at one sample value among the corresponding four sample values, the remaining A process of resetting the difference amount so as to absorb the overflow of the three sample values is performed so that the average values of the corresponding pixels match. FIG. 8 shows a flowchart of this process. When it is particularly required to reduce the scale of the hardware, there is not much problem even if the overflow and the underflow are not corrected as the simple processing. However, in this case, since the overflow or underflow portion of the sample value in which the overflow or underflow has occurred is ignored, the sample point of the input digital image signal is representative in the final output signal (corrected signal). The average of the corrected signal in the area of interest is the magnitude of the original input digital image signal,
It may be somewhat different from time to time.

[0030]

According to the conventional interpolation processing alone, there is a problem that the high-frequency component is attenuated due to the finite number of taps of the interpolation filter and the resolution of the image is reduced. According to the method, by performing the intra-area average difference correction processing, the edge becomes sharp, and the apparent resolution can be improved.

[Brief description of the drawings]

FIGS. 1 (a), 1 (b) and 1 (c) show sample points ○ of an input digital image signal before interpolation processing and sample points の of a digital image signal interpolated therefrom, respectively. What is not applicable to the invention and what can be applied (interpolation of 1: 2) are shown.

FIG. 2 shows an area represented by a sample point ○ in FIG.

FIG. 3 shows an area represented by a sample point x in FIG.

FIG. 4 shows the operation of the intra-area average difference correction processing according to the present invention.

FIG. 5 is a block diagram showing an embodiment of a digital image signal interpolation processing apparatus according to the present invention.

FIGS. 6 (a) and 6 (b) show sample points ○ of an input digital image signal when the apparatus of the present invention is configured for a two-dimensional space, and digital points obtained by performing 1: 2 interpolation processing on the sample points. A sample point x of the image signal is shown.

FIGS. 7 (a) and 7 (b) show sample points ○ of the input digital image signal when the apparatus of the present invention is configured as a line-offset input digital image signal (two-dimensional space), respectively. : 2 indicates a sample point x of the digital image signal subjected to the interpolation processing of 2.

FIG. 8 is a flowchart illustrating a process of correcting an overflow or an underflow.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Interpolation processing part 2 Subtractor 3 Adder 4 Average value detection part

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yasumasa Ito 1-10-11 Kinuta, Setagaya-ku, Tokyo Japan Broadcasting Research Institute (72) Inventor: Hajinobu Mizutani 1-1-10 Kinuta, Setagaya-ku, Tokyo 11 Japan Broadcasting Corporation Broadcasting Research Institute

Claims (3)

[Claims] In a digital image signal interpolation processing apparatus, an area represented by sample points of an input digital image signal before interpolation and an area represented by a plurality of sample points after interpolation are larger than both areas. Interpolation processing means for performing interpolation processing on the input digital image signal so as to coincide with each other, and sample values of a plurality of sample points after the interpolation processing corresponding to an area represented by sample points of the input digital image signal Average value detection means for detecting an average value of the input digital image signal, and the sample values of the corresponding plurality of sample points after the interpolation processing detected by the average value detection means from the sample values of the sample points of the input digital image signal. Subtraction means for subtracting an average value; and subtraction means for subtracting each sample value of a plurality of sample points after the interpolation processing, which is an output of the interpolation processing means. A digital image signal interpolation processing apparatus comprising at least an adding means for adding an output corresponding to each area in the output including a sign of the output. 2. An interpolation processing apparatus for a digital image signal, wherein an area represented by sample points of the input digital image signal before interpolation and an area represented by a plurality of sample points after interpolation are larger than both areas. Interpolation processing means for performing interpolation processing on the input digital image signal so as to coincide with each other, and sample values of a plurality of sample points after the interpolation processing corresponding to an area represented by sample points of the input digital image signal Average value detecting means for detecting an average value of the input digital image signal from the average value of the sample values of the plurality of sample points after the interpolation processing detected by the average value detecting means. Subtraction means for subtracting a sample value; and output of the subtraction means to each sample value of the plurality of sample points after the interpolation processing, which is the output of the interpolation processing means. A digital image signal interpolation processing device comprising at least an adding means for adding an output corresponding to each area in the output while inverting the sign of the output and adding the output. 3. An interpolation processing apparatus for digital image signals according to claim 1, wherein said interpolation processing means performs 1: 2 interpolation processing in a horizontal and / or vertical direction of the image. A digital image signal interpolation processing device.