JP3277696B2 - Digital signal processing apparatus and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for processing digital signals such as digital image signals and digital audio signals.

[0002]

2. Description of the Related Art When signal processing of a conventional digital image signal, digital audio signal or the like is performed, it is common to perform signal processing in either the time domain or the frequency domain. The processing in the frequency domain can express the stationary characteristics of the signal well, but is not suitable for expressing the transient characteristics.
On the other hand, the processing in the time domain is suitable for expressing the transient characteristics, but is not suitable for expressing the stationary characteristics. Here, the steady characteristic means a stable repetitive change, and the transient characteristic means an isolated one-time change.

As an example, FIG. 13 shows a case of time domain processing. As shown in FIG. 13A, the transient characteristic has a waveform (impulse-like waveform) that changes greatly with respect to the time axis, and this can be sufficiently processed by using, for example, about several samples. it can. The dots in the waveform indicate sampling positions, and in the case of digital signals, are discrete signal sequences having a sampling value corresponding to the level of each sampling position. However, in the figure, the same applies to the following, but it is represented by an analog signal waveform. On the other hand, the steady-state characteristic has a waveform (flat waveform) having a gradual change on the time axis as shown in FIG. 13B. This is because even if several samples are used, the characteristics of the waveform are not sufficiently understood. Processing cannot be performed.

Next, when considered in the frequency domain, the stationary characteristic has an impulse-like waveform as shown in FIG. on the other hand,
The transient characteristic has a flat waveform as shown in FIG. 14B. As described above, the impulse-like waveform is more suitable for capturing the characteristics of the signal.

[0005] A general signal waveform is expressed as follows with respect to the time axis.
As shown in FIG. 15, the portion FL of the steady characteristic (flat)
1, FL2, FL3, ... and transient characteristics (impulse)
Are mixed. Therefore, by performing only one of the time-domain processing and the frequency-domain processing, it is difficult to perform processing that correctly reflects the characteristics of the signal. For this reason, it is necessary to perform time domain processing and frequency domain processing on the same signal, and there has been a problem that the processing time becomes longer and the scale of hardware for processing becomes larger.

Accordingly, an object of the present invention is to process the stationary characteristic portion of a digital signal in the frequency domain,
It is an object of the present invention to provide digital signal processing capable of processing the transient characteristic portion in the time domain and shortening the processing time and reducing the scale of hardware for processing.

[0007]

SUMMARY OF THE INVENTION The first aspect of the present invention, a first analyzing means for dividing <br/> analyze the stationarity of the input digital signal in the time domain, the input in the frequency domain Di
At least one of second analysis means for analyzing the continuity of the digital signal, and a stationary component having a steadiness based on an output of the at least one analysis means.
Classifying means for classifying into a transient component having no stationary state ,
Based on the characteristics of the input digital signal,
Classification means for classifying signals into a plurality of classes
And the input digital signal classified by the classification means.
Digital signal processing for stationary and transient components in the time domain and frequency domain according to each of multiple classes
And a synthesizing means for synthesizing an output digital signal based on the outputs of the first and second processing means.

According to a second aspect of the present invention, there is provided an analyzing means for analyzing a frequency of an input digital signal, a separating means for separating an impulse-like component and a flat component in a frequency domain from an output of the analyzing means, First processing means for supplying an impulse-like component of the output of the means and processing the same in the frequency domain; first converting means for converting the flat component of the output of the separating means into a time-domain signal; A second processing unit for processing the output of the first conversion unit in the time domain, a second conversion unit for converting the output of the first processing unit to the time domain, and a second conversion unit. This is a digital signal processing device comprising a synthesizing means for synthesizing an output and an output of the second processing means.

[0009]

An input digital signal, for example, a digital video signal is converted into coefficient data by DCT, and by analyzing this coefficient data, an impulse component and a flat component are separated in the frequency domain. The impulse-like signal in the frequency domain is processed in the frequency domain by the first processing means. Since the flat component in the frequency domain becomes an impulse signal in the time domain, it is processed in the time domain by the second processing means. As described above, since the processing is performed in the form of an impulse signal, the result of the processing can be improved.

[0010]

An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, the present invention is applied to the resolution compensation of a digital video signal. The resolution compensation means that in FIG.
2B by compensating for the narrowing of the band as shown by the frequency characteristic of 20b by a filtering process or the like on a wideband video signal as shown by the frequency characteristic of FIG. Is to convert it to a broadband video signal as shown.

In FIG. 1 showing the overall configuration of this embodiment, a standard resolution digital video signal (referred to as an SD video signal) is supplied to an input terminal 1. A high-resolution digital video signal is called an HD video signal. An example of an input SD video signal is SDVT
R playback signal, broadcast signal, and the like. The input SD video signal is supplied to the blocking circuit 2, and the video signal in the order of television raster is scan-converted into a signal having a (8 × 8) block structure, for example.

For the blocking circuit 2, DCT (Disc)
Rete Cosine Transform) circuit 3 is connected, and from the DCT circuit 3, corresponding to one block, coefficient data DC of one DC component and coefficient data AC of 63 AC components are provided.
1, AC2,..., AC63 occur. As an example, starting from DC, coefficient data is output by zigzag scanning in which higher-order AC coefficients are sequentially output. DCT is one means of frequency analysis of an input video signal, and may use FFT, Hadamard transform, or the like.

The coefficient data from the DCT circuit 3 is supplied to a classification circuit 5 via a coefficient analysis circuit 4. The coefficient analysis circuit 4 and the classification circuit 5 are provided to separate a steady component and a transient component of the digital video signal converted into the frequency domain. The classification circuit 5 outputs a flat component (ie, a transient component) 6a in the frequency domain.
And an impulse-like component (ie, a stationary component) 6b appear separately.

For easy understanding, an example of the value of the coefficient data is (DC = 50, AC1 = 48, AC2 = 46,
AC3 = 44, AC4 = 42, AC5 = 60, ...
・). The coefficient analysis circuit 4 analyzes the coefficient data and determines that AC5 is in the form of an impulse.
That is, AC5 should be 40 due to the change tendency of AC1, AC2, AC3, and AC4. This stands out by a value of 20, as it has a value of 60.
The classification circuit 5 outputs a flat component (transient component in the frequency domain, DC = 50, AC1 = 48, A
C2 = 46, AC3 = 44, AC4 = 42, AC5 = 4
0,...) 6a and an impulse-like component in the frequency domain (a steady component, and in the above example, DC = 0, AC1
= 0, AC2 = 0, AC3 = 0, AC4 = 0, AC5 =
,...) 6b are output separately.

The flat component 6a from the classification circuit 5 has an inverse D
The signal is supplied to the CT circuit 7, returned to a time-domain signal, and supplied to the block decomposition circuit 8. From the block decomposition circuit 8, a digital video signal returned in the order of television raster scanning is obtained. This digital video signal is supplied to a class classification adaptive processing circuit 9 as a second processing circuit. This circuit 9 is a processing circuit for increasing the resolution in the time domain, as described later. The flat component 6a is suitable for processing in the time domain, and the circuit 9 can satisfactorily compensate for the resolution.

The impulse component 6b from the classification circuit 5 is supplied to a gain conversion circuit 10. The output signal of the blocking circuit 2 is supplied to the gain conversion circuit 10 for class classification. The gain conversion circuit 10 is provided with a memory in which gain conversion ratio information previously acquired by learning is stored as described later. As described above, by adjusting the gain of the coefficient data according to the conversion ratio information, the high frequency component is enhanced in the frequency domain. An output signal of the gain conversion circuit 10 is supplied to the inverse DCT circuit 11. The signal returned to the time domain by the inverse DCT circuit 11 is supplied to the block decomposition circuit 12 and converted into data in the order of television raster scanning.

The output signal of the block decomposing circuit 12 is supplied to a synthesizing circuit 14 via a phase compensating circuit 13, and the synthesizing circuit 14 synthesizes the output signal of the class classification adaptive processing circuit 9 described above. This synthesis is a simple multiplexing process. Then, a digital video signal whose resolution is compensated, that is, an HD video signal is obtained from the combining circuit 14 to the output terminal 15.

FIG. 3 shows an example of the classification adaptive processing circuit 9. A digital video signal from the block decomposition circuit 8 is supplied to an input terminal 21. This digital video signal is a flat component (transient component) of the SD video signal, and is an impulse signal in the time domain. This digital video signal is output to the synchronization circuit 22.
Supplied to The output data of the synchronization circuit 22 is supplied to the classification circuit 23. A memory 2 in which the output of the classifying circuit 23 stores mapping tables M1 to M4, respectively.
4a to 24d are supplied as address signals.

FIG. 4 partially shows the relationship between the SD image and the HD image. In FIG. 4, the pixel data of ○ is for an SD image, and the pixel data of X is for an HD image.
For example, pixel data y1 to y4 of four HD images are generated from pixel data al of twelve SD images. The mapping table M1 of the memory 24a is for generating pixel data y1, and the mapping tables M2, M3, M4 of the memories 24b, 24c, 24d are for generating pixel data y2, y3,
y4, respectively.

The read outputs of the memories 24a to 24d are supplied to a selector 25. The selector 25 is controlled by the output of the select signal generation circuit 26. The select signal generating circuit 26 is supplied with an HD image sample clock from an input terminal 27. By the selector 25, 4
The pixel data y1 to y4 are sequentially selected, and these pixel data are supplied to the scan conversion circuit 28. The scan conversion circuit 28 generates the pixel data of the HD image at the output terminal 29 in the order of raster scan. The number of pixels of the output image is four times the number of pixels of the input SD video signal.

The mapping tables M1 to M4 stored in the memories 24a to 24d are generated in advance by learning. FIG. 5 shows an example of a configuration for generating mapping tables M1 to M4.
Shown in In FIG. 5, a digital HD video signal is supplied to an input terminal indicated by 31. This HD video signal
It is preferable that the signal is a standard signal in consideration of generation of the mapping table. In practice, a standard image is taken by an HD video camera, or
By recording on a DVTR, an HD video signal can be obtained.

The HD video signal is supplied to the synchronization circuit 32. The synchronizing circuit 32 simultaneously outputs pixel data a to l and y ₁ to y ₄ having the positional relationship shown in FIG. The pixel data a to l are supplied to the class classification circuit 33. The class classification circuit 33 uses the HD
The pixel data y _{1 to} y ₄ are classified. The output of the classifying circuit 33 is output from the mapping table generating circuits 34a to 34a.
34d are commonly supplied.

The pixel data y _{1 to} y from the synchronization circuit 32
₄ is supplied to the mapping table generation circuits 34a to 34d. The mapping table generation circuits 34a to 34d have the same configuration. Two types of mapping tables are possible. One of them is HD pixel values y ₁ , y ₂ , y ₃
Or y ₄ intended to predict a linear combination of values a~l coefficient w ₁ to w ₁₂ of the SD pixel, in this case, the coefficient w ₁ to w ₁₂ is determined for each class. The other is the value of the HD pixel itself predicted for each class.

The mapping table creating circuits 34a-3 in FIG.
4d stores a mapping table indicating the correlation between the HD video signal and the SD video signal. In other words, when a plurality of data of the SD video signal is given, a mapping table for outputting the pixel data of the HD video signal that is in average correspondence with the plurality of data classes can be formed.

The class classification circuit 33 classifies the pixel data of interest in the same manner as the class classification circuit 23 of FIG. 3 to generate class information. As the class classification, class classification based on gradation, class classification based on a pattern, and the like can be used. When gradation is used, if the pixel data is 8 bits, the number of classes becomes extremely large. Therefore, it is preferable to reduce the number of bits of each pixel by high-efficiency coding such as ADRC. When a pattern is used, a plurality of patterns (for example, flat, the value increases at the upper right, the value decreases at the lower right, etc.) composed of four pixels are prepared, and the output data of the synchronization circuit 32 is converted to the plurality of patterns. Classify into one of them.

[0026] Taking the mapping table generating circuit 34a for obtaining the HD pixel data y ₁ as an example, the memory is provided class information from classification circuit 33 is supplied as an address. At the time of training (learning), an SD video signal is formed by thinning out the original HD video signal. A horizontal thinning process (subsampling) and a vertical thinning process (subline) are performed.
An HD video signal of one frame or more, for example, a still image is used. The memory for each address corresponding to the class information, the sample values of the pixel data a~l and y ₁ is written. For example, at address AD0 of the memory, (a
_{10, a 20, ···, a} n0) (b 10, b 20, ···,
b _n0 )... (l ₁₀ , l ₂₀ ,..., l _n0 ) (y ₁₀ ,
y ₂₀ ,..., y _n0 ) are stored.

The learning data thus stored is read from the memory, and an HD pixel (corresponding to y ₁ ) predicted value obtained by a linear linear combination of SD pixel values al and coefficients w _{1 to} w ₁₂ The coefficient that minimizes the error between the true value and the true value is obtained by the least square method. Focusing on the learning data stored at one memory address, the following simultaneous equations hold for this address.

[0028] _{y 10 = w 1 a 10 +} w 2 b 10 + w 3 c 10 + ······ + w 12 l 10 y 20 = w 1 a 20 + w 2 b 20 + w 3 c 20 + ····· _{· + w 12 l 20 y 30} = w 1 a 30 + w 2 b 30 + w 3 c 30 + ······ + w 12 l 30 · · · y n0 = w 1 a n0 + w 2 b n0 + w 3 c n0 + ... + w ₁₂ l _n0

Here, y ₁₀ -y _n0 , a ₁₀ -a _n0 , b _10-
Since b _n0 , c _{10 to} c _n0 ,..., l _{10 to} l _n0 are known, the coefficient w ₁ that minimizes the square of the error of the predicted value with respect to y _{10 to} y _n0 (true value). it is possible to find the ~w _12. Coefficients can be determined for other classes (addresses) in the same manner. The coefficients determined in this way are stored in a memory and used as a mapping table.

The value of the data of the HD video signal is not limited to the coefficient but may be obtained by training for each class and stored in the memory. For example, FIG. 6 shows a configuration for that purpose.
A data memory 40 and a frequency memory 41 to which class information from the class classification circuit 33 is supplied as addresses are provided.

The read output of the frequency memory 41 is applied to an adder 42.
Is supplied to the memory 41, and the output of the adder 42 is stored in the memory 41.
At the same address. Memory 40 and 41
, The contents of each address are cleared to zero as an initial state.

The data read from the data memory 40 is supplied to the multiplier 43 and is multiplied by the frequency read from the frequency memory 41. The output of the multiplier 43 is supplied to the adder 44, where the output is added to the input data y. The output of the adder 44 is supplied to a divider 45 as a divisor.
The output (quotient) of the divider 45 is used as input data in the data memory 40.

In the configuration shown in FIG. 6, when a certain address is accessed first, memories 40 and 41 are accessed.
For reading the output of a 0, the data y ₁₀ is directly written in the memory 40, the value of the corresponding address in the memory 41 is set to 1. Wakashi, thereafter, when the address is accessed once again, the output of the adder 42 is 2, which is the output of the adder 44 (y ₁₀ + y _20). Therefore, the output of the divider 45 is (y ₁₀ + y ₂₀ ) / 2, which is written to the memory 40. Thereafter, when the above-mentioned address is accessed, the memory 40 is operated in a similar manner.
Data is changed to _{(y 10 + y 20 + y} 30) / 3, frequency is also updated to 3.

By performing the above operation for a predetermined period, a mapping table is stored in the memory 40 such that when a class is specified by the output of the class classification circuit 33, data at that time is output. In other words, when a plurality of pixel data of the input video signal is given, a mapping table can be formed that outputs data that has an average correspondence with a result of classifying the pixel data.

The class classification adaptive processing circuit 9 will be described in more detail. As described above, the class classification adaptive processing circuit 9 performs training on the coefficients of the linear linear combination by training.
Determine in advance. During this training, the configuration shown in FIG. 7 is used. In FIG. 7, an input terminal 51 receives a number of standard HD signal still images, which are supplied to a vertical thinning filter 52 and a learning unit 54. The vertical decimation filter 52 decimates the HD image by half in the vertical direction. Connected to the vertical thinning filter 52, the horizontal thinning filter 53 performs horizontal thinning by １ ／, and S
The still image of the pixel equivalent to the D signal is supplied to the learning unit 54. The memory 55 stores the class code created by the learning unit 54 and the learning result.

In this example, as shown in FIG. 8, the positional relationship between HD pixels and SD pixels is defined. As shown in FIG. 8, when an SD pixel (3 × 3) block is used, SD pixels a to i and HD pixels A, B, C, and D form a set of learning data. There are a plurality of sets of learning data for one frame, and an extremely large number of sets of learning data can be used by increasing the number of frames.

FIG. 9 is a flowchart showing the operation of the learning section 54 when determining the coefficient of the linear linear combination by software. Control of the learning unit is started from step 61, and step 62
In the corresponding data block formation, the HD signal and the SD signal are supplied, and a process of extracting the HD pixels and the SD pixels having the arrangement relationship as shown in FIG. At the end of the data in step 63, if the processing of all the input data, for example, data of one frame has been completed, the control is shifted to the prediction coefficient determination in step 66, and if not, the control is shifted to the class determination in step 64.

In step 64, the class is determined from the signal pattern of the SD signal. In this control, ADRC can be used to reduce the number of bits.
In the normal equation addition in step 65, an equation as described later is created.

After the processing of all data is completed from the end of data in step 63, the control moves to step 66, and
In the prediction coefficient determination of No. 6, an equation described later is solved using a matrix solution method to determine a prediction coefficient. In step 67, the prediction coefficients are stored in the memory.
At 8, the control of the learning section ends. The prediction coefficient of the class is stored in the memory with the class determined by the SD signal as an address. The class and the prediction coefficient correspond to the mapping table described above.

The processing for obtaining a coefficient for defining the relationship between the HD pixel and the SD pixel in FIG. 8 will be described in more detail. In general, when the SD pixel level is x ₁ to x _n and the HD pixel level is y, _n by the coefficients w ₁ to w _n for each class
Setting the linear estimation equation of tap _{y'= w 1 x 1 + w} 2 x 2 + ‥‥ + w n x n (1). Before learning, _wi is an undetermined coefficient.

As described above, learning is performed for a plurality of Hs for each class.
Performed for D data and SD data. If the number of data is m, according to Equation _{1, y j '= w 1} x j 1 + w 2 x 2 2 + ‥‥ + w n x jn (2) ( where, j = 1,2, ‥‥ m)

[0042] m> For n, w ₁ so to w _n are not uniquely determined, elements of an error vector _{e e j = y j - (} w 1 x j1 + w 2 x j2 + ‥‥ + w n x jn (3) (where j = 1, 2, ‥‥ m) and a coefficient that minimizes the following Expression 4 is obtained.

[0043]

(Equation 1)

This is a solution by the so-called least square method. Here, the partial differential coefficient by w _i in Equation 3 is obtained.

[0045]

(Equation 2)

Since it is sufficient to determine each w _i so that Equation 6 is set to 0,

[0047]

(Equation 3)

Using a matrix

[0049]

(Equation 4)

By solving Equation 8 using a general matrix solution such as a sweeping-out method, a prediction coefficient w _i is obtained, and the prediction coefficient w _i is stored in a memory using a class code as an address.

As described above, the learning unit uses the actual data HD
The prediction coefficient w _i can be obtained using the signal, and this is stored in the memory. And any input S
The class information is formed from the D signal, the prediction coefficient corresponding to the class information is read from the memory, and the value of the pixel of interest can be formed by linear linear combination of the value of the SD pixel around the pixel of interest and the prediction coefficient. , An output HD image can be generated for an arbitrary input SD image.

When the learning section 54 determines not the prediction coefficient but the representative value for each class, the processing shown in the flowchart of FIG. 10 is performed. The starting step 71, the learning data forming step 72, the data ending step 73, and the class determining step 74 are the same as those in FIG.
It is the same as steps 61, 62, 63 and 64 in FIG.

In the normalization step 75, the pixel values are normalized. That is, assuming that the value (input value) of the HD pixel is y, the input data is normalized by the calculation of (y-base) / DR. Here, DR is the difference (dynamic range DR) between the maximum value and the minimum value of the pixels in one block when a to i are one block in the pixel array shown in FIG. Further, base is a reference value of the block, for example, a minimum value of the pixels of the block. An average value of the pixel values in the block other than the minimum value may be used. With this normalization, the relative level of the pixel can be noted.

In the representative value determination step 76, the cumulative frequency n (c) of the class is determined in the same manner as in FIG. 6, and the representative value g (c) is determined. That is, the newly formed representative value g (c) ′ is g (c) ′ = {(y−base) / DR + n (c) × g (c)} / n (c + 1) (9) is there. The representative value for each class obtained in this way is stored in the memory.

As information compression means for classifying, for example, DCT (Disc) is used instead of the ADRC circuit.
rete Cosine Transform), VQ (vector quantization),
Alternatively, what is provided can be appropriately selected as long as it is means capable of performing data compression, such as providing a DPCM (prediction coding) circuit.

As described above, the classification adaptive processing circuit 9 learns the correspondence between the SD signal and the HD signal in the time domain based on the properties of the actual image, and from the learning, the HD signal corresponding to the SD signal. Can be generated. In addition, since the class is adaptively selected according to the level distribution of the SD signal, up-conversion that follows the local properties of the image can be performed. Furthermore, unlike the one using the interpolation filter, it is possible to obtain an HD signal whose resolution is compensated.

Returning to FIG. 1, an impulse-like component 6b is supplied from the classification circuit 5 in the frequency domain.
The gain conversion circuit 10 as a processing circuit for compensating the resolution in the frequency domain. That is, as shown in FIG. 11, the gain conversion is for compensating that the high-frequency gain of a signal whose frequency characteristic originally expanded to a high frequency is reduced by signal processing. The gain conversion circuit 10, like the classification adaptive processing circuit 9, has a memory in which a mapping table for compensating for high frequencies by learning in advance is stored. As in the case of the above-described time domain class classification adaptive processing circuit 9, there are two types of mapping tables, one for outputting a gain conversion ratio and the other for outputting a predicted value of gain.

FIG. 12 shows a configuration at the time of learning for creating a mapping table in the gain conversion circuit 10. HD video data used for learning is supplied to an input terminal 81 and supplied to a sub-line / sub-sampling circuit 82. This circuit 82 performs vertical thinning (sub-line).
And horizontal thinning (sub-sampling). Accordingly, a video signal having the same resolution as the SD video signal is generated from the sub-line / sub-sampling circuit 82.

A delay circuit 83 and a D / A converter 90 are connected to the sub-line / sub-sampling circuit 82.
The delay circuit 83 is for delaying input data and adjusting timing until the classification is performed. The blocking circuit 84 is connected to the delay circuit 83, and, for example, data having a (4 × 4) block structure is synchronized. The output of the blocking circuit 84 is supplied to the DCT circuit 85, where the cosine transform is performed. Starting from the DC component coefficient data, the DCT circuit 85 generates coefficient data in the order of low-order to high-order AC component coefficient data (zigzag scanning).

The coefficient data from the DCT circuit 85 is supplied to the division circuit 86. The division circuit 86 is provided to obtain a gain conversion ratio for coefficient data, which is required for compensating a high band. The gain conversion ratio signal from the division circuit 86 is supplied to the memory 87. The memory 87 has a plurality of memories for storing the gain conversion ratio corresponding to each of the plurality of DCT coefficients.

In order to check the high frequency degradation of the SD video signal resulting from the signal processing, the SD video signal converted into an analog signal by the D / A converter 90 is converted into an analog signal by the analog transmission system 9.
1 is supplied. The analog transmission system 91 is, for example, a recording and reproducing process of an analog VTR. The video signal transmitted through the analog transmission system 91 is converted into a digital signal by the A / D converter 92 and supplied to the blocking circuit 93.

The blocking circuit 93 forms digital video data having the same block structure as the output data of the blocking circuit 84. Output data of the blocking circuit 93 is supplied to the DCT circuit 94 and the class classification circuit 95. The coefficient data from the DCT circuit 94 is supplied to the division circuit 86. A division process is performed on the coefficient data of the same order, and a gain conversion ratio signal relating to the coefficient data is generated by the division circuit 86. That is, when the signal passes through the analog transmission system 91, the high-frequency component is lost. The gain conversion ratio signal indicates how the gain (value) of each component of the DCT coefficient data changes. .

For example, DC, AC 1 to DC 1
AC15 coefficient data is generated, and the DCT circuit 94
Consider a case where coefficient data of C 'and AC1' to AC15 'is generated. The division circuit 86 forms the gain conversion ratio signals G ₀ , G ₁ ,..., G ₁₅ by the following operation. G ₀ = DC / DC ′, G ₁ = AC / AC ′,..., G ₁₅ = AC ₁₅ / AC ₁₅ ′

Although not shown in FIG. 12 for simplicity, a plurality of gain conversion ratio signals generated for each coefficient are averaged to obtain a final gain conversion ratio signal. It is memorized.

Such a gain conversion ratio signal is multiplied by the coefficient data of the video data in which the high frequency has been attenuated, thereby making it possible to generate the coefficient data of the video data in which the high frequency is compensated. . The gain conversion circuit 10 in FIG. 1 has a memory in which a gain conversion ratio signal obtained in advance by learning is stored, and changes the value of the coefficient data by multiplying the coefficient data by the gain conversion ratio signal. . As a result, high-frequency compensation can be performed.

The classifying circuit 95 includes the blocking circuit 9
Classification according to the level distribution of the block data from No. 3 is performed. For this classification, it is preferable to perform data compression such as ADRC, as described above.
The class information obtained by the class classification circuit 95 is stored in the memory 87.
Is supplied as an address in the memory. The memory 87 includes a plurality of memories corresponding to the coefficient data of the DC and the coefficient data of the ACs of all orders, and the gain conversion ratio signal for the coefficient data corresponding to each of the plurality of memories. Is stored.

An address for switching a plurality of memories is formed by an address counter 88 in correspondence with the coefficient data. The address counter 88 is connected to the input terminal 8
9 to generate addresses that change sequentially. In this case, the address changes in synchronization with the coefficient data from the blocking circuit 84. Then, a plurality of types of HD video signals are supplied to the input terminal 81,
An optimum gain conversion ratio signal is formed for each class, and is stored in the memory 87.

Instead of the gain conversion ratio, the value of the predicted DCT coefficient can be obtained by learning.

The same signal as the gain conversion ratio signal stored in the memory 87 is stored in the memory provided in the gain conversion circuit 10 of FIG. The output signal of the blocking circuit 2 is supplied to the gain conversion circuit 10 for class classification. In the gain conversion circuit 10, each component of the DCT coefficient data is multiplied by a gain conversion ratio signal to perform gain adjustment. As a result, high frequency band compensation is performed. Here, an impulse-like component 6b is supplied to the gain conversion circuit 10 in the frequency domain. The reason is that if a signal composed of various components including a flat component is to be converted, a non-linear component is mixed in, the accuracy is deteriorated, and a problem arises that a correct gain conversion cannot be performed. For the same reason, an impulse-like signal is used also during the learning shown in FIG.

In the above embodiment, the present invention is applied to resolution compensation of a video signal, more specifically, up-conversion from an SD signal to an HD signal. However, the present invention can be applied to other signal processing. For example, the present invention can be applied to a noise removal circuit.

As noise, impulse noise (white noise) and pink noise can generally be present. Impulse noise can be removed by passing through a median filter in the time domain. The median filter compares the values of three consecutive sample data on the time axis. When the value of the central sample data does not exist between the values of the preceding and following sample data, the central sample data is regarded as noise. Judgment is performed, and a process of replacing the value of the central sample data with data having a closer value in the preceding and following sample data is performed (for example, see Japanese Utility Model Publication No. 3-19094).

The impulse noise in the time domain can be removed by this median filter. However, it is difficult to remove flat noise (pink noise) in the time domain. According to the present invention, the signal is converted from the time domain to the frequency domain together with the noise removal processing in the time domain using the median filter, and the impulse-like component is separated in the frequency domain. Noise is removed by a digital low-pass filter. As a result, it is possible to remove a flat noise component in the time domain. As a result, various noises in the input digital signal can be removed.

The present invention can be applied not only to the resolution compensation and noise removal circuits but also to other digital signal processing.

[0074]

As described above, the present invention divides a digital information signal into a stationary component and a transient component, processes them separately in the time domain and the frequency domain so that each can be appropriately represented, and then combines them again after the processing. I do. Therefore, compared to performing the time domain and the frequency domain in two stages,
Advantages such as a reduction in processing time, a reduction in hardware scale, and an improvement in signal processing accuracy can be obtained.

[Brief description of the drawings]

FIG. 1 is an overall block diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram for explaining resolution compensation performed according to an embodiment of the present invention.

FIG. 3 is a block diagram illustrating an example of a class classification adaptive processing circuit according to an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating an arrangement of pixels between an SD image and an HD image.

FIG. 5 is a block diagram of an example of a configuration for creating a mapping table in which prediction coefficients are stored.

FIG. 6 is a block diagram illustrating an example of a configuration for creating a mapping table in which predicted values are stored.

FIG. 7 is a block diagram illustrating an example of a configuration at the time of learning for forming a prediction coefficient or a prediction value;

FIG. 8 is a schematic diagram illustrating another example of an array of pixels between an SD image and an HD image.

FIG. 9 is a flowchart showing processing at the time of learning for forming a prediction coefficient.

FIG. 10 is a flowchart showing processing at the time of learning for forming a predicted value.

FIG. 11 is a schematic diagram for explaining high-frequency compensation in the frequency domain.

FIG. 12 is a block diagram for learning a gain conversion ratio for high frequency compensation in a frequency domain.

FIG. 13 is a schematic diagram showing an impulse component and a flat component in a time domain.

FIG. 14 is a schematic diagram illustrating an impulse component and a flat component in a frequency domain.

FIG. 15 is a schematic diagram of a signal waveform including both an impulse component and a flat component in a time domain.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 input terminal of high-resolution digital image signal 3 DCT circuit 5 classification circuit for separating flat component and impulse-like component in frequency domain 7, 11 inverse DCT circuit 9 class classification adaptive processing circuit 10 gain conversion circuit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI H04N 7/30 H04N 7/133 Z

Claims (20)

(57) [Claims] 1. A of the input digital signal between regions when
A first analyzing means for analyzing the stationarity, at least one of the second analysis means for analyzing the stationarity of the input digital signal in the frequency domain, the based on the output of the analysis means comprising said at least one Steady-state stationary component and stationary digital signal
A classification means for classifying the input digital signal into a transient component having no characteristic, and the input data based on characteristics of the input digital signal.
Classes for classifying digital signals into multiple classes
Classifying means and the input digital signal classified by the classifying means.
The above-mentioned stationary component and the above-mentioned transient component of the signal in each of the plurality of classes in the time domain and the frequency domain.
First and second processing means for performing corresponding digital signal processing; and an output based on the output of the first and second processing means.
Synthesizing means for synthesizing a digital signal, a digital signal processing apparatus comprising a. 2. An analyzing means for frequency-analyzing an input digital signal; a separating means for separating an impulse-like component and a flat component in a frequency domain from an output of the analyzing means; and an impulse output from the separating means. First processing means for supplying a shape component and processing the same in the frequency domain; first converting means for converting a flat component of the output of the separating means into a time domain signal; Second processing means for processing the output of the conversion means in the time domain; second conversion means for converting the output of the first processing means to the time domain; and the output of the second conversion means. And a synthesizing means for synthesizing the output of the second processing means. 3. The digital signal processing device according to claim 1, wherein the input digital signal is a digital video signal. 4. The digital signal processing device according to claim 1, wherein said frequency domain analysis means is an orthogonal transform. 5. The digital signal processing device according to claim 4, wherein said orthogonal transform is a discrete cosine transform. 6. The digital signal processing device according to claim 4, wherein said orthogonal transform is a fast Fourier transform. 7. The digital signal processing device according to claim 4, wherein said orthogonal transform is a Hadamard transform. 8. The digital signal processing apparatus according to claim 1, wherein the processing means includes a digital filter having a finite number of taps. 9. The digital signal processing device according to claim 1, wherein the processing means is a noise removing circuit. 10. The digital signal processing device according to claim 9, wherein said noise removing circuit comprises a median filter and a low-pass filter. 11. entry between domain or frequency domain when
And analyzing the constancy of the force digital signal, based on the analysis results of the stationarity of the input digital signal
A step of classifying a certain stationary component and steadiness to the transient components without, on the basis of the characteristics of the input digital signal, a classification step of classification of said input digital signal into a plurality of classes, classified the input The above stationary component of the digital signal and
First and second processing steps of performing digital signal processing on the transient component in the time domain and the frequency domain according to each of the plurality of classes; and Based on the result ,
Synthesizing an output digital signal ; and a digital signal processing method. 12. A step of frequency-analyzing an input digital signal, a step of separating an impulse-like component and a flat component in a frequency domain based on the analysis result, and the impulse-like component is supplied and processed in a frequency domain. A first processing step of converting the flat component into a time domain signal; and a second processing of processing the signal converted in the first conversion step in the time domain. A processing step; a second conversion step of converting the result of the first processing step into a time domain; and a signal obtained as a result of the conversion in the second conversion step and a signal obtained as a result of the second processing step. Digital signal processing method comprising the steps of combining. Claim 13 The digit according to claim 11 or 12.
Signal processing method, The input digital signal is a digital video signal
A method comprising: 14. The digit according to claim 11 or 12.
Signal processing method, The frequency domain analysis is an orthogonal transform.
Way. 15. The digital signal processor according to claim 14,
In the processing method, The orthogonal transform is a discrete cosine transform.
And how. 16. The digital signal processor according to claim 14,
In the processing method, The orthogonal transform is a fast Fourier transform.
how to. 17. The digital signal processor according to claim 14,
In the processing method, The orthogonal transform is a Hadamard transform.
Way. 18. The digit according to claim 11 or 12.
Signal processing method, The processing means includes a digital filter with a finite number of taps.
And the method characterized by the above. (19) The digit according to claim 11 or 12.
Signal processing method, The processing step is a noise removal step.
how to. 20. The digital signal processor according to claim 19.
In the processing method, The above noise removal step is performed by the median filter and the
-A method characterized by being processed by a pass filter.