JPH0888849A - Picture signal processing method and picture signal transmitter - Google Patents

Abstract

PURPOSE: To improve the picture quality of restored pictures by performing different processings corresponding to the class of thinned-out picture elements and obtaining the picture element value of the thinned-out picture elements. CONSTITUTION: An average interpolation value generation circuit 51 obtains pictures thinned out by a sub sampling circuit by average value interpolation in a horizontal line direction. That is, the average interpolation value generation circuit 51 supplies input data through delay elements (D) 51A and 51B for respectively delaying the input data for one picture element in a horizontal direction to an addition circuit 51D and also supplies the input data to the addition circuit 51D as they are. As a result, in the addition circuit 51D, the picture element value of the thinned-out picture elements during the time is calculated from sampling picture elements. Then, an edge emphasis filter 53 emphasizes an edge near a horizontal interpolation picture element by performing a two-dimensional edge emphasis processing. Thus, edge information and texture information lost by average value interpolation are recovered to the level close to the original pictures.

Description

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Industrial Application Conventional Technology (FIGS. 10 and 11) Problem to be Solved by the Invention (FIGS. 10 and 11) Means for Solving the Problem (FIGS. 1 to 3 and 11) Operation (FIG. 1) -FIG. 3) Example (1) Overall configuration (FIGS. 1 and 2) (2) Class classification circuit (FIG. 3) (3) Detailed configuration (FIGS. 4-9) (4) Operation of example (FIG. 1) (FIG. 3) (5) Effect of the embodiment (FIG. 3) (6) Other embodiment (FIG. 11) Effect of the invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing method and an image signal transmitting apparatus, and is particularly suitable for application to an image signal transmitting apparatus for reducing the amount of information of original image data by thinning processing and transmitting the information. Is.

[0003]

2. Description of the Related Art Conventionally, in a so-called image signal transmission system for transmitting an image signal to a remote place such as a video conference system, or an apparatus for recording an image signal into a video tape recorder or a video disc recorder for reproduction. In order to efficiently use the transmission path and the recording medium, the correlation between the digitized image signals is used to efficiently encode the significant information to reduce the transmission information amount and the recording information amount, thereby reducing the transmission efficiency and It is designed to improve recording efficiency.

Specifically, a method of reducing the amount of data to be transmitted by highly efficient compression coding of image data is widely used. In this high efficiency encoding, in general,
The number of pixels is reduced by performing sub-sampling (thinning-out) on the original image data in advance spatially or spatiotemporally, and then compression encoding processing is performed to further reduce the amount of information to be transmitted. An example of spatiotemporal subsampling is MUSE (Multiple Sampling Encode)
There is a method.

On the other hand, on the receiving side, the resolution is increased by obtaining the pixels that have not been transmitted, that is, the thinned pixels, by interpolation. Generally, interpolation is performed by a fixed tap or a filter having a fixed coefficient. However, in such an interpolation method, the interpolated pixel value is obtained by averaging the pixel values of a plurality of pixels adjacent to the interpolated pixel. Therefore, depending on the type of the image, image blurring and spatio-temporal fluctuation (that is, jerkiness) , Edge business, etc.), resulting in deterioration of image quality of the restored image.

As one method for solving such a problem,
A digital data converter as described in Japanese Patent Application No. 5-201913 has been proposed. As shown in FIG. 10A, the transmitter 1 in this digital data converter
Generates the compression coded data D2 by passing the input image data D1 through the sub-sampling circuit 2 and the encoder 3, and supplies this to the output terminal 4.

Further, the transmitter 1 decodes the compression coded data D2 by the local decoder 5 and then supplies it to the least square method arithmetic circuit 6. The least squares method operation circuit 6 inputs the decoded data D3 and the input image data D1, sets the pixel (thus removed pixel) decimated by sub-sampling as the pixel of interest, and sets the pixel of interest included in the input image data D1. A linear linear combination model is established with the pixel value and the pixel values of the peripheral pixels included in the decoded data D3, and the coefficient of the linear linear combination model is calculated by the least squares method. As a result, the least squares method arithmetic circuit 6 outputs coefficient data D4 corresponding to the pixels thinned out by the sub-sampling circuit 2, and the coefficient data D4 is given to the output terminal 7.

As described above, in the transmitter 1, the compressed coded data D2 obtained by compressing and coding the image data after the sub-sampling as well as the coefficient representing the correlation between the thinned pixels and the transmission pixels in the vicinity thereof are obtained. The data D4 is transmitted. .

The receiver 10 of the digital data converter.
Is configured as shown in FIG. 10B, and the compression encoded data D2 is input to the decoder 12 via the input terminal 11. The decoded data D5 decoded by the decoder 12 is supplied to the time series conversion circuit 13 and the interpolation calculation circuit 14. Further, the receiver 10 receives the coefficient data D4 at the input terminal 1
It is input to the interpolation calculation circuit 14 via 5.

The interpolation calculation circuit 14 uses the decoded data D5 and the coefficient data D4 to obtain the interpolation data D6 from the linear linear combination equation and sends it to the time series conversion circuit 13. The time series conversion circuit 13 uses the decoded data D5 and the interpolation data D6.
Are arranged in the same manner as the original image (that is, the input image data D1) to obtain the restored image data D7.

Thus, in the digital data conversion device composed of the transmitter 1 and the receiver 10, the transmitter 1
Side, compression-encoded data D obtained by high-efficiency encoding
Instead of transmitting only 2, the coefficient data D4 representing the correlation between the thinned pixels and the transmission pixels in the vicinity thereof is transmitted together, and the coefficient data D4 is transmitted on the receiver 10 side.
By generating thinned pixels that are not actually transmitted by using, the image quality deterioration on the receiver 10 side can be reduced even when the amount of transmission information is reduced by thinning. .

In addition, in Japanese Patent Application No. 5-201913, in addition to this, the transmitter 1 side classifies the interpolation target pixel into classes according to the distribution state of transmission pixels around the interpolation target pixel (target pixel). , A method of obtaining the above-mentioned coefficient data D4 by the least-squares arithmetic circuit 6 for each class has been proposed.
According to this method, the accuracy of the coefficient data D4 can be improved, so that the image quality of the restored image in the receiver 10 can be improved. In this case, the receiver side also performs the same class classification as the transmitter side, and selects the coefficient data for the class to obtain the interpolated pixel value.

As another method, as disclosed in Japanese Unexamined Patent Publication No. 5-328185, a memory for storing a prediction coefficient or a prediction value previously obtained by learning is provided on the receiving side, and thinning processing is performed. The interpolation pixel value is generated based on the prediction coefficient read from the memory according to the transmitted transmission image data, or the prediction value read from the memory according to the transmission image data is set as the interpolation pixel value. A digital data conversion device that reduces image quality deterioration has been proposed.

This digital data converter is shown in FIG.
In the transmitter 20, the compression coded data D10 obtained through the sub-sampling circuit 21, the encoder 22 and the transmission processing circuit 23 is sent to the transmission line 25 via the output terminal 24 as shown in FIG. The receiver 30 receives the compression coded data D10 input through the input terminal 31 from the reception processing circuit 3
2 and the decoder 33 to pass the decoded data D11
And sends it to the synchronization circuit 34. Synchronization circuit 3
Reference numeral 4 designates a thinned pixel as a target pixel, and simultaneously synchronizes a plurality of peripheral transmission pixels for each target pixel to cluster the clustering circuit 3
5

The clustering circuit 35 classifies the input transmission image data into classes according to gradations and patterns, and sends the class classification result to the memory 36 as class information D12 representing the class of each pixel of interest. The memory 36 outputs the prediction coefficient D13 corresponding to the input class information D12 among the prediction coefficients that are obtained and stored for each class in advance by learning using the class information D12 as an address.

The interpolation data creation circuit 37 uses the prediction coefficient D13.
Interpolation pixel data D14 is created by performing an operation by linear linear combination using the peripheral pixel values synchronized with. The interpolated pixel data D14 and the decoded data D11 are combined by the subsequent combining circuit 38 to generate restored image data D15.

Thus, the receiver 30 can restore high frequency components that do not exist in the transmitted image data D10, and as a result, it is possible to suppress image quality deterioration due to thinning processing, and to obtain a high resolution restored image. Obtainable.

[0018]

By the way, as described above, the pixels to be interpolated are classified into classes using the transmission pixels around the pixels to be interpolated, and the prediction coefficient or prediction value is obtained for each class and transmitted. Equipment (Japanese Patent Application No. 5-201913),
The prediction coefficient or prediction value is obtained in advance by learning and stored in the memory of the receiving side, and the receiving side classifies the pixels to be interpolated into classes and outputs the prediction coefficient or predicted value corresponding to the class to interpolate the object to be interpolated. Device for obtaining pixel value
5-328185 gazette), the larger the number of classes,
Since the accuracy of the prediction coefficient or the prediction value can be improved, the quality of the restored image is improved.

However, if the number of classes is increased, the amount of calculation for obtaining the prediction coefficient or the predicted value increases, and in the receiver 30 (FIG. 11) described above, the amount of data that must be stored in the memory 36 increases. Therefore, it is practically impossible to take an infinite number of classes, and it cannot be said that sufficient classification is done. As a result, for example, in a portion such as an edge or a texture where a sufficient resolution is required, image quality is deteriorated due to insufficient class classification, and there is a problem that sufficient resolution cannot be obtained.

The present invention has been made in consideration of the above points, and when classifying pixels to be interpolated, image signal processing capable of improving the image quality of a restored image by performing more accurate class classification processing. A method and an image signal transmission device are proposed.

[0021]

In order to solve such a problem, according to the present invention, thinned pixels are classified into classes according to the states of the surrounding pixels, and different processing is performed for each class. In an image signal processing method for obtaining an interpolated value of a thinned pixel, an interpolation step for generating an interpolated pixel value corresponding to the thinned pixel using the remaining pixels that have not been thinned, and an interpolated pixel value and a thinned pixel The edge enhancement step that applies edge enhancement processing to the image that has the remaining pixel values, and for the image after edge enhancement, classify the thinned pixels according to the states of the interpolation pixels and the pixels around the interpolation pixels. A class classification step for classifying is provided, and different processing is performed depending on the class of the thinned pixel obtained by the class classification step to obtain the pixel value of the thinned pixel.

In the present invention, the input image data D
The thinning means 41 for generating the transmission pixel data D2 in which the number of pixels is reduced by thinning out a predetermined number of pixels 1 and the interpolation for generating the interpolated pixel value corresponding to the thinned pixel by using the remaining pixels which are not thinned out. A value generation unit 51, an edge enhancement unit 53 that performs edge enhancement processing on an image that includes the interpolated pixel value and the pixel value that is left unthinned, and the interpolated pixel and the periphery of the interpolated pixel for the image after the edge enhancement. A class classification unit 55 that classifies each thinned pixel according to the state of the pixel, and a plurality of transmission pixels around the thinned pixel and the thinned pixel for each class classified by the class classification unit. The transmission pixel data D is provided with the parameter generation means 44 for generating the parameter D4 representing the correlation with
2 and parameter information D4 are transmitted.

Further, in the present invention, the transmission pixel data D2 in which the number of transmission pixels is reduced by thinning, and each thinned pixel classified according to the distribution state of a plurality of transmission pixels around the thinned pixel. In the image signal transmission device 60 that receives the parameter D4 obtained for each of the classes and generates a restored image, the interpolation value generation that generates the interpolation pixel value corresponding to the pixel thinned using the transmission pixel data D2. Means 51 and edge enhancing means 53 for performing edge enhancement processing on an image composed of interpolated pixel values and transmission pixel values.
In the edge-enhanced image, a class classification unit 55 that classifies the thinned pixels according to the states of the interpolation pixels and the pixels around the interpolation pixels, and the transmission pixel data D2.
And the parameter D corresponding to each thinned pixel class
4 and the interpolation data generating means 63 for generating the interpolation data D32 corresponding to each thinned pixel.

Further, in the present invention, the transmission pixel data D10 in which the number of transmission pixels is reduced by thinning is received,
In an image signal transmission device 30 that generates a restored image by interpolating thinned pixels, an interpolation value generation unit 51 that generates an interpolated pixel value corresponding to a thinned pixel using a transmission pixel, and an interpolated pixel. Edge enhancing means 53 for performing edge enhancement processing on an image consisting of values and transmission pixel values, and for the image after edge enhancement, classify thinned pixels according to the states of interpolation pixels and pixels around the interpolation pixels. Using the class classification means 55 for classification and the class D12 classified by the class classification means 55 as addresses, the coefficient data D13 representing the correlation between the thinned pixels and a plurality of transmission pixels around the thinned pixels is output. Memory means 36
And coefficient data D13 output from the memory means 36
And the transmission pixel data D11 are used to provide the interpolation data generating means 37 for generating the interpolation data D14 corresponding to each thinned pixel.

[0025]

The pixel which remains without being thinned is used to generate an interpolated pixel value corresponding to the thinned pixel, and edge enhancement processing is performed on the image formed by the interpolated pixel and the pixel left without being thinned. Then, the thinned pixels are classified into classes. As a result, the edge enhancement processing allows the classification processing to be performed on the basis of the image in which the edge information and the texture information are returned to values close to the original image, so that accurate classification can be performed particularly in the edge portion and the texture portion. Thus, it is possible to significantly improve the image quality at the edge or texture portion of the restored image.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Overall Structure In FIG. 1, reference numeral 40 denotes an image coding apparatus to which the present invention is applied as a whole, and input image data D1 is input to a sub-sampling circuit 41, where predetermined pixels of the input image data D1 are input. Is thinned out to reduce the amount of information, and then sent to the subsequent compression encoder 42. The compression encoder 42 compression-encodes the remaining pixels after thinning to generate compression-encoded data D2, and outputs this as transmission image data.

The compressed coded data D2 is decoded by the local decoder 43, and the decoded data D20 is given to the coefficient selection circuit 44 and the class classification circuit 45.
The class classification circuit 45 regards the pixel to be interpolated (that is, the thinned pixel) as the pixel of interest, classifies each pixel of interest according to the distribution state of the transmission pixels spatially and / or temporally surrounding the pixel of interest, The classification result is the class information of the interpolation target pixel (hereinafter referred to as index data) D.
The value 21 is sent to the coefficient selection circuit 44.

The coefficient selection circuit 44, based on the input image data D1 and the decoded data D20 input through the delay circuit 46, for each class, the pixel value of the pixel of interest included in the input image data D1 and the decoded data around it. The correlation with the pixel value of D20 is obtained by learning for each frame, and the learning result is output as coefficient data D4. The delay circuit 46 includes the sub-sampling circuit 41 and the compression encoder 42.
The input image data D1 is delayed by the processing time required by the local decoder 43.

The transmission data sent from the image coding device 40 has the structure shown in FIG.
0 received and decoded. Image decoding device 60
The compressed coded data D2 is decoded by the compression decoder 61, and the decoded data D30 thus obtained is given to the class classification circuit 62. The class classification circuit 62 has a configuration similar to that of the class classification circuit 45 described above with reference to FIG. 1, and an interpolation target pixel that has not been actually transmitted is set as a pixel of interest, and spatially and / or temporally peripheral transmission of the pixel of interest is performed. By classifying the target pixel according to the state of the pixel, the index data D31 of each target pixel is generated,
This is sent to the interpolation data creation circuit 63.

The interpolation data creating circuit 63 obtains the interpolation data D32 based on the index data D31, the decoded data D30 and the coefficient data D4 of each interpolation target pixel.
In practice, the interpolation data creating circuit 63 selects the coefficient data D4 corresponding to the index data D31, and the selected coefficient data D4 and the decoded data D30 around the interpolation target pixel.
Interpolation data D32 is obtained by establishing a linear linear combination equation using and.

Interpolation data D32 obtained in this way
Is sent to the time series conversion circuit 64. Time series conversion circuit 6
4 arranges the decoded data D30 and the interpolation data D32 in the same arrangement as the original image (that is, the input image data D1) to generate the restored image data D33. Thus, the image decoding device 60 can obtain a restored image with almost no image quality deterioration by being able to obtain an interpolation pixel close to the true value.

(2) Class Classification Circuit The class classification circuits 45 and 62 are constructed as shown in FIG. Here, since the class classification circuit 62 of the image decoding device 60 has the same configuration as the class classification circuit 45 of the image coding device 40, the class classification circuit 45 will be described. The class classification circuit 45 arranges the decoded data D20 by the time series conversion circuit 50 in the line scan order,
Then, the average interpolation value generation circuit 51 sends the average interpolation value generation circuit 51.

The average interpolation value generation circuit 51 obtains the images thinned out by the sub-sampling circuit 41 by the average value interpolation in the horizontal line direction. That is, the average interpolation value generation circuit 51 supplies the input data to the adder circuit 51D via delay elements (D) 51A and 51B that delay the input data by one pixel in the horizontal direction, and supplies the input data as it is to the adder circuit 51D. As a result, in the addition circuit 51D, for example, the pixel value of the pixel thinned out from the sampling pixel indicated by the mark “◯” in FIG. 5 is calculated.

Incidentally, the addition circuit 51D has a multiplication function of multiplying the addition result by 1/2 in addition to the addition operation.
The pixel value of the pixels thinned out between the two sampling pixels in the horizontal direction is calculated by averaging. Adder circuit 51
The output of D is output to the selector 5 via the delay element (D) 51E.
Given to 1C. The selector 51C alternately outputs the sampling pixel and the average interpolation pixel. Time series conversion circuit 5
In No. 2, the sampling pixels and the average interpolation pixels are rearranged in an order suitable for the processing of the edge enhancement filter 53.

The edge emphasizing filter 53 emphasizes edges in the vicinity of the average interpolated pixel by performing a two-dimensional edge emphasizing process. As a result, the edge information and texture information lost by the average value interpolation are returned to values close to the original image. In the case of the embodiment, the filter coefficient of the edge emphasizing filter 53 is selected as shown in FIG. The output of the edge enhancement filter 53 is rearranged in a predetermined order by the time series conversion circuit 54 and then given to the ADRC encoder 55. As a result, assuming that the average interpolation pixel is the target pixel, the ADRC encoder 55 receives n peripheral pixels including this target pixel.

The ADRC encoder 55 applies k-bit adaptive quantization to each of the n input pixels and arranges the resulting codes in a predetermined order and stores them in the shift register 56. Thus index data D21 formed of the pattern of the total n × k bits from the shift register 56 (or D31) (L ₁ ~L _m) is output. Therefore, the pixel of interest is classified into any of the 2 ^{n * k} classes.

As described above, in the class classification circuits 45 and 62, the pixel values corresponding to the thinned pixels are first obtained by the average interpolation, then the edge enhancement processing is performed, and the pixels obtained as a result are classified into the classes. By performing the processing, it is possible to classify the restored image in consideration of the vicinity of the edge portion or the texture portion, which is particularly desired to be restored clearly.

(3) Detailed Configuration Next, the sub-sampling circuit 41, the compression encoder 42,
Detailed configurations of the coefficient selection circuit 44 and the interpolation data creation circuit 63 will be described.

In the case of this embodiment, the sub-sampling circuit 41, as shown in FIG. 5, has time points T0, T1, T2,
The thinning process is applied to the frame image in ... so that the number of transmitted pixels is halved. At this time, the sub-sampling circuit 41 selects a pixel position where the pixels at the same position as the position sampled at the previous time point are not sampled at the next time point between successive time points. By performing spatial offset sub-sampling, the amount of transmitted information is reduced by half while preserving image feature amounts as much as possible. Incidentally, the mark "○" in FIG. 5 represents the pixels (that is, the transmission pixels) remaining after the thinning,
The "+" mark represents an interpolation target pixel (thinned pixel).
Further, w ₁ to w _{38 in the} lower stage represent the coefficients of the linear linear combination model obtained by the coefficient selecting circuit 44 described later.

The compression encoder 42 is an ADRC (Adapti
ve Dynamic Range Coding) circuit, and, for example, pixel data having an information amount of 8 bits per pixel, dynamic range information representing the difference between the maximum pixel value and the minimum pixel value within a predetermined block, By expressing the minimum pixel information and the difference between each pixel value and the minimum pixel value by, for example, the quantization information when one-bit quantization is performed, the transmission information amount is effectively reduced.

The coefficient selection circuit 44 represents the interpolation target pixel by a linear linear combination model of the transmission pixel and the coefficient around the interpolation target pixel, and the coefficient used at this time is subjected to the least squares method for each class. Calculated by calculation. The principle of selecting the coefficient will be described. As shown in FIG. 5, when the frame in which the interpolation target pixel y _m exists is T1, the surrounding area is cut out from the frame T1 with the interpolation target pixel y _m as the center, and the frame T0 and the frame T0 at the immediately preceding time point are extracted. A region at the same position as the region cut out in the frame T1 from the frame T2 at the immediately subsequent time point is cut out. The pixel value of the interpolation target pixel y _m is extracted from the input image data D1.

Here, first, by multiplying each transmission pixel in the region cut out from the frames T0 to T2 by a coefficient w _i , the interpolation target pixel y _m is linearly linearly combined by spatially and temporally surrounding transmission pixels. Express As a result, the determinant Y of the interpolation target pixel y _{m and} the determinant X of the surrounding transmission pixel x _mi are calculated using the determinant W of the coefficient w _i as follows:

[Equation 1] Can be expressed in the form of an observation equation. However,
In the equation (1), n represents the number of spatially and temporally surrounding transmission pixels when one interpolation target pixel y _m is represented by a linear linear combination equation (in the case of the embodiment, one interpolation target pixel Since y _m is represented by the linear linear combination model of 38 taps, n = 38), and m represents the number of interpolation target pixels existing in one frame.

Here, if one coefficient set is to be calculated for one frame based on the equation (1), the following equation is obtained from the equation (1).
Simultaneous equations for the number of pixels to be interpolated in the frame are created. Basically, it suffices to solve this simultaneous equation and obtain the coefficient. In the embodiment, the simultaneous equations are solved by using the least square method. That is, first, using the residual matrix E, the equation (1) is transformed into the following equation,

[Equation 2] It is re-expressed in the form of residual equation like.

Here, in order to obtain the most probable value of each coefficient value w _i from equation (2), e ₁ ² + e ₂ ² + ... + e _i-1 ² + e _i ²
The condition that minimizes

(Equation 3) W ₁ , w ₂ , which satisfy the following condition by entering n conditions
......, just find w _n . Here, from the equation (2), the following equation,

[Equation 4] And the condition of equation (3) is established for i = 1, 2, ..., N,

(Equation 5) Is obtained. Here, the following normal equation is obtained from the equations (2) and (5).

(Equation 6)

Since the normal equation represented by the equation (6) is a simultaneous equation in which the number of unknowns is n, the most probable coefficient w _i can be obtained. To be precise, in equation (6), w _i is multiplied (Σx _jn x _jn ) (where j =
If the matrix of 1 ... m) is regular, it can be solved. In reality, the simultaneous equations are solved using the Gauss-Jordan elimination method (sweep method).

In the case of the embodiment, the coefficient value w _i is obtained for each class obtained by the class classification circuit 45 by using the above-mentioned least squares method. As a result, it is sufficient to transmit one set of coefficients for each frame for each class, which is far more significant than the case where coefficient values w _i for all interpolation target pixels are obtained and transmitted. The amount of calculation can be reduced. In fact, the information amount of the coefficient is of a negligible order as compared with the pixel information amount per frame.

Specifically, the coefficient selection circuit 44 is constructed as shown in FIG. That is, the coefficient selection circuit 44
Is the time series conversion memory 9 for the interpolation target pixel data y _m contained in the decoded data D20 (x _i ) and the input image data D1.
Enter 0. Time series conversion memory 90, and outputs the synchronizing pixel _{(x 1 ~x n, y m} ) to form a linear combination model as described above for FIG. The data output from the time series conversion memory 90 is given to the normalization equation generation circuit 91, and the normalization equation generation circuit 9 concerned.
1, the normalization equation as expressed by the equation (6) is generated for each class, and the subsequent CPU arithmetic circuit 92 obtains the coefficient set for each class by the sweeping method.

The normalization equation generation circuit 91 first multiplies each pixel by the multiplier array 93. The multiplier array 93 is configured as shown in FIG. 7, performs pixel-by-pixel multiplication for each cell represented by a square, and gives each multiplication result obtained thereby to the subsequent adder memory 94.

The adder memory 94, as shown in FIG.
An adder array 95 and a memory (or register) array 96, which are cells arranged in the same manner as the multiplier array 93.
It is composed of A ₁ , 96A ₂ , .... The memory arrays 96A ₁ , 96A ₂ , ... Are provided for the number of classes (in the case of the embodiment, for 2 ^{n * k} classes classified by the class classification circuit 45), and the index decoder for decoding the index data D21. In response to the output (class) of 97, one memory array 96A ₁ , 96A ₁
A ₂ , ... Is selected, and the selected memory array 96A
The stored values of ₁ , 96A ₂ , ... Are fed back to the adder array 95. At this time, the addition result obtained by the adder array 95 is again the corresponding memory arrays 96A ₁ and 96A.
₂ , stored in ....

In this manner, the product array operation is performed by the multiplier array 93, the adder array 95 and the memory array 96A ₁ , and the memory arrays 96A ₁ , 96A ₂ , ... For each class determined by the index. Is selected, and the memory array 96A is selected according to the result of the product-sum operation.
Contents of ₁ , 96A ₂ , ... are updated.

The position of each array depends on w _{i of the} normalization equation represented by the equation (6) (Σx _jn x _jn )
(However, it corresponds to the position of j = 1 ... m). As is clear from the normalization equation (6), if the upper right term is inverted, the result will be the same as the lower left term, so that each array has a triangular shape.

In this way, the product-sum operation is performed for each class during one frame, and the normalization equation for each class is generated. The result of each term of the normalization equation for each class is the memory array 96A ₁ , 96A corresponding to each class.
CPUs that are stored in A ₂ , ... And then each term of the normalization equation for each class realizes the sweeping method operation
It is calculated by the arithmetic circuit 92. As a result, a set of coefficients w _i for each class is obtained, and this is sent as coefficient data D4.

As shown in FIG. 9, the interpolation data generating circuit 63 has a coefficient memory 100 for storing the coefficient set of each class for each frame, and the coefficient memory 100 has a coefficient set w ₁ to w for each class. In addition to storing w _n , coefficient sets w _{1 to} w _n corresponding to the class are output in response to the output (class) of the index decoder 101. This coefficient set w _{1 to} w
_n is applied to the multiplier 103A ₁ ~103A _n via respective registers 102A ₁ ~102A _n. Further, the multipliers 103A _{1 to} 103A _n are provided with the time series conversion circuit 104.
The decoded data x _{1 to} x _n selected by are given.
Accordingly, the outputs of the multipliers 103A _{1 to} 103A _n are added by the adder circuit 105 to obtain the pixel value y (= x ₁ w ₁ + x ₂ w ₂ + ... + x _n w _n ) of the interpolation target pixel. .

(4) Operation of the Embodiment With the above-mentioned configuration, the image coding apparatus 40 reduces the data amount by performing the thinning process on the input image data D1 and then generates the compressed coded data D2. Further, the image coding apparatus 40 sets the pixels thinned out in the class classification circuit 45 as the target pixel, classifies the thinned pixels according to the state of the pixels around the target pixel, and then selects the coefficient. In the circuit 44, the coefficient set w _{1 for} each class
w _n is obtained and this is output as coefficient data D4.

Here, the class classification circuit 45 first obtains the pixel value of the thinned pixel by averaging the pixels left undecimated on both sides, and the edge of the image containing the average interpolated value obtained thereby. Emphasize processing. As a result, the edge information and texture information lost by the average interpolation are restored. Then, the class classification circuit 45 determines pixels of the surrounding pixels including the thinned pixels (that is, the interpolation target pixels subjected to the edge enhancement processing after the average interpolation) based on the image in which the edge information and the texture information are restored in this way. Each thinned pixel is classified into classes according to the state. As a result, the thinned pixels can be accurately classified, particularly in the edge portion and the texture portion.

In the image decoding device 60, the decoded data D30 obtained by decoding the compression coded data D2 and the coefficient data D4.
The restored image data D33 is generated based on At this time, the class classification circuit 62 classifies the thinned pixels by performing the same class classification processing as that performed by the class classification circuit 45 of the image encoding device 40. Next, in the interpolation data creation circuit 63, the classification result D3
Coefficient sets w _{1 to} w selected from the coefficient data D4 according to ₁ .
Interpolation data D32 is created by linearly combining _n and the decoded data D30.

Thus, the interpolation target pixel (thinned pixel)
By accurately classifying the data according to the characteristics of the surrounding images, it is possible to obtain the interpolation data D32 that is much closer to the true value, and it is possible to obtain a restored image with little image deterioration due to thinning.

(5) Effects of the Embodiments According to the above-mentioned configuration, the remaining pixels which are not thinned are used to generate the interpolated pixel values corresponding to the thinned pixels, and the interpolated pixels and the thinned pixels are not thinned. By performing edge enhancement processing on an image composed of the remaining pixels and then classifying thinned pixels into classes, accurate classification can be performed particularly in the edges and textures. Thus, the image quality at the edge or texture portion of the restored image can be significantly improved.

(6) Other Embodiments In the class classification circuits 45 and 62 of the above-described embodiments, the pixel values of the pixels thinned out by the average interpolation value generation circuit 51 are interpolated with the average value in the horizontal line direction. However, the present invention is not limited to this.
The interpolation pixel value of the thinned pixel may be obtained by dimensional adaptive interpolation or the like, and various interpolation methods of the thinned pixel before edge enhancement can be applied.

Further, in the class classification circuits 45 and 62 of the above-described embodiment, the ADRC encoder 55 is provided, the image after the edge intensity is ADRC encoded, and the bit number obtained by this is based on the code pattern compressed. The case of classifying thinned pixels has been described.
Instead of the DRC encoder 55, for example, DCT (Discre
te Cosine Transform) Transform coding, DPCM (differential quantization), vector quantization, sub-band coding, and a circuit having other compression functions such as wavelet transform are provided, and thinning is performed based on the output code pattern. The removed pixels may be classified into classes. Further, it is not limited to the case of classifying based on the compressed code pattern, but the thinned pixels may be classified according to the direction in which the degree of correlation of the image after edge emphasis is strong, that is, the class after edge emphasis. Various methods can be applied for classification.

Further, in the above-described embodiment, the case where the transmission pixels remaining after the thinning-out are compression-encoded by the compression encoder 42 and transmitted is described, but the present invention is not limited to this, and the thinning-out is performed after the thinning-out without compression-encoding. It can also be applied when transmitting the remaining thinned-out data as it is. In this case, the compression encoder 4 on the image encoding device 40 side
2 and the local decoder 43 may be omitted, and the compression decoder 61 may be omitted on the image decoding device 60 side.

In the above embodiment, the case where the ADRC circuit is used as the compression encoder 42 has been described, but the present invention is not limited to this, and the compression encoder 42 may be, for example, DCT transform coding, DPCM, vector quantization, Even when a compression method such as subband coding or wavelet transform is used, the same effect as that of the above-described embodiment can be obtained.

Further, in the above-described embodiment, the sub-sampling circuit 41 outputs 1 to the input image data D1.
However, the present invention is not limited to this, and the same effect as that of the embodiment can be obtained even when other thinning processing such as 1/4 thinning processing is performed. .

Further, in the above-mentioned embodiment, the linear linear combination model is obtained by solving the linear first-order coupling model as a parameter representing the correlation between the pixel of interest and the transmission pixels spatially and / or temporally surrounding the pixel of interest. Although the case of transmitting the coefficient data D4 described above has been described, the present invention is not limited to this.
As the parameter, for example, an average value of transmission pixel values around the pixel of interest obtained for each class may be transmitted as a representative value. In this case, if the center of gravity method is used for the average calculation, a representative value with less error can be easily obtained for each class. Not limited to this, it is possible to use various parameters that represent the correlation between the pixel of interest that is not transmitted and the transmitted pixel.

Further, in the above-described embodiment, the present invention is applied to the class classification circuit 45 of the image coding device 40 and the class classification circuit 62 of the image decoding device 60 for transmitting the coefficient data D4 together with the compression coded data D2. However, the present invention is not limited to this, and as shown in FIG. 11, for example, the class classification means of the image decoding device (receiver) 30 having the memory 36 for generating the prediction coefficient D13 for each class. The present invention can be applied to the (clustering circuit) 35.

In this case, the memory 36 may store the prediction coefficient acquired for each class in advance by learning. As an example, there is a method of obtaining a prediction coefficient by performing a process on various images when the coefficients of the linear linear combination model set for each class are obtained by the least square method.

Further, a coefficient selecting circuit 44 (FIG. 1) for generating coefficient data as in the embodiment is also provided on the side of the image encoding device (transmitter) 20 and the memory 3 acquired in advance by learning.
The present invention can also be applied to the class classification means of the image signal transmission device in which the contents of 6 are updated by the coefficient data newly transmitted from the image encoding device.

Further, in the memory 36 shown in FIG. 11, the class classification means (clustering circuit) 35 of the image decoding apparatus (receiver) 30 in which the prediction value for each class is stored in advance in place of the prediction coefficient is also provided. The invention can be applied.

In this case, the first method for obtaining the predicted value to be stored in the memory 36 is a learning method using weighted average. More specifically, using the transmission pixels around the interpolation target pixel, the interpolation target pixel is classified into classes, and the pixel value of the interpolation target pixel integrated for each class is incremented according to the number of interpolation target pixels. The predicted value can be obtained by performing the process of dividing by the frequency on various images.

As a second method for obtaining the predicted value stored in the memory 36, there is a learning method by normalization. In detail, a block consisting of a plurality of pixels including the interpolation target pixel is formed, and the value obtained by subtracting the reference value of the block from the pixel value of the interpolation target pixel is normalized by the dynamic range in this block. The predicted value can be obtained by performing a process for a predicted value that is a value obtained by dividing the cumulative value of the normalized values by the cumulative frequency.

As described above, the class classification method according to the present invention can be widely applied to the case where the interpolated pixel value is generated by using the prediction coefficient or the prediction value obtained by class classification.

[0073]

As described above, according to the present invention, the thinning means for generating the transmission pixel data in which the number of pixels is reduced by thinning the predetermined pixels of the input image data, and the remaining pixels which are not thinned out are used. Interpolation value generation means for generating interpolation pixel values corresponding to thinned pixels, edge enhancement means for performing edge enhancement processing on an image composed of interpolated pixel values and pixel values remaining without being thinned, and edge enhancement. For the subsequent image, a class classification unit that classifies the thinned pixels according to the states of the interpolation pixels and the pixels around the interpolation pixels, and the thinned pixels for each class classified by the class classification unit. And parameter generation means for generating a parameter representing a correlation between a plurality of transmission pixels around the thinned pixel and transmission image data and parameter information are transmitted. More, in the case of classification classes the pixels decimated, considering edge information and texturized information can more accurate classification. As a result, the quality of the restored image can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing the overall configuration of an image coding apparatus according to an embodiment.

FIG. 2 is a block diagram showing the overall configuration of an image decoding apparatus according to an embodiment.

FIG. 3 is a block diagram showing a configuration of a class classification circuit according to an embodiment.

FIG. 4 is a schematic diagram showing a filter coefficient of an edge enhancement filter.

FIG. 5 is a schematic diagram for explaining thinning-out processing by a sub-sampling circuit and for explaining pixels and coefficients used in a linear linear combination model.

FIG. 6 is a block diagram showing the configuration of a coefficient selection circuit.

FIG. 7 is a schematic diagram showing a configuration of a multiplier array.

FIG. 8 is a schematic diagram showing a configuration of an adder memory.

FIG. 9 is a block diagram showing the configuration of an interpolation data creation circuit.

FIG. 10 is a block diagram showing a configuration of a conventional digital data converter.

FIG. 11 is a block diagram showing the configuration of a digital data conversion device having a memory that stores prediction coefficients or prediction values on the receiver side.

[Explanation of symbols]

1, 20 ... Transmitter, 10, 30 ... Receiver, 35 ...
Clustering circuit, 36 ... Memory, 37 ... Interpolation data creation circuit, 40 ... Image coding device, 41 ... Sub-sampling circuit, 42 ... Compression encoder, 43 ...
Local decoder, 45, 62 ... Class classification circuit, 4
4 ... Coefficient selection circuit, 51 ... Average interpolated value generation circuit, 5
3 ... Edge enhancement filter, 55 ... ADRC encoder, 63 ... Interpolation data estimation circuit, 60 ... Image decoding device, D1 ... Input image data, D2 ... Compressed coded data, D4 ... Coefficient data, D21, D31 ...... index data, D32 ...... interpolated data, D33 ...... restored image data, W ₁ ~W ₃₈ ...... coefficients.

Claims (13)

[Claims] 1. An image signal processing method for classifying thinned pixels according to the state of pixels around the thinned pixels, and performing different processing for each class to obtain an interpolation value of the thinned pixels. In the above, an interpolation step of generating an interpolation pixel value corresponding to the thinned pixel using the remaining pixel not thinned, and an image including the interpolation pixel value and the pixel value remaining not thinned On the other hand, an edge enhancement step for performing edge enhancement processing, and an edge-enhanced image are provided with a class classification step for classifying each thinned pixel according to the states of the interpolation pixel and the pixels around the interpolation pixel, Image signal processing characterized in that the pixel value of the decimated pixel is obtained by performing different processing depending on the class of the decimated pixel obtained by the class classification step. Reasoning method. 2. In the class classification step, the interpolated pixel after edge enhancement and the pixels around the interpolated pixel are ADRC-encoded, and the number of bits obtained thereby is thinned out based on a compressed code pattern. The image signal processing method according to claim 1, wherein the pixels are classified into classes. 3. A decimating unit for decimating a predetermined number of pixels of input image data to generate transmission pixel data having a reduced number of pixels, and an interpolation pixel corresponding to a decimated pixel using the remaining pixels not decimated. Interpolation value generating means for generating a value, edge enhancement means for performing edge enhancement processing on the image composed of the interpolation pixel value and the pixel value remaining without being thinned, and the interpolation pixel for the image after edge enhancement. And a class classification unit that classifies each thinned pixel according to the state of pixels around the interpolated pixel, and the thinned pixel and the thinned pixel for each class classified by the class classification unit. A parameter generating means for generating a parameter representing a correlation with a plurality of the transmission pixels around the pixel, and transmitting the transmission pixel data and the parameter information. Image signal transmission device. 4. The class classifying means classifies the interpolated pixel and pixels around the interpolated pixel by ADRC coding, and classifies each thinned pixel based on a code pattern in which the bit number is compressed. The image signal transmission device according to claim 3, wherein 5. The parameter generating means, for each class classified by the class classifying means, forms a linear linear combination model from the thinned pixels and a plurality of the transmission pixels around the thinned pixels. 4. The image signal transmitting apparatus according to claim 3, wherein the coefficient of the linear linear combination model solved by the operation of the least square method is set as the parameter. 6. The transmission pixel data in which the number of transmission pixels is reduced by thinning, and each thinned pixel class classified according to the distribution state of the plurality of transmission pixels around the thinned pixel. In an image signal transmission device that receives the obtained parameters and generates a restored image, an interpolation value generation unit that generates an interpolation pixel value corresponding to a pixel thinned using the transmission pixel data, and the interpolation pixel Values and the edge enhancement means for performing edge enhancement processing on the image composed of the transmission pixel values, and for the image after edge enhancement, the thinned pixels are selected according to the states of the interpolation pixels and the pixels around the interpolation pixels. By using the class classification means for classifying, the transmission pixel data, and the parameter corresponding to the class of each thinned pixel, the interpolation data corresponding to each thinned pixel is used. An image signal transmission device, comprising: an interpolation data generating means for generating a data. 7. The class classification means performs ADRC encoding on the interpolation pixel and pixels around the interpolation pixel, and classifies each thinned pixel on the basis of a code pattern in which the bit number is compressed. The image signal transmission device according to claim 6, wherein 8. The received parameters are, for each class, a linear linear combination model established from the thinned pixels and a plurality of the transmission pixels around the thinned pixels. Coefficients are coefficient data obtained by the calculation of the least squares method, and the interpolation data generating means performs linear linear combination of the coefficient data corresponding to each thinned pixel class and the transmission pixel data. ,
The image signal transmission device according to claim 6, wherein the interpolation data corresponding to each thinned pixel is generated. 9. An image signal transmission device for receiving transmission pixel data in which the number of transmission pixels has been reduced by thinning, and generating a restored image by interpolating the thinned pixels, wherein thinning is performed using transmission pixels. The interpolated value generating means for generating the interpolated pixel value corresponding to the pixel, the edge emphasizing means for performing the edge emphasizing process on the image composed of the interpolated pixel value and the transmission pixel value, and the image after the edge emphasizing. Class classification means for classifying each thinned pixel according to the state of the interpolated pixel and the pixel around the interpolated pixel, and the thinned pixel and the thinned pixel using the class classified by the class classification means as an address. Memory means for outputting coefficient data representing a correlation with a plurality of said transmission pixels in the vicinity of the generated pixel, coefficient data output from said memory means, and said transmission image By using the data, the image signal transmission apparatus characterized by comprising an interpolation data generating means for generating interpolation data corresponding to each decimated pixel. 10. The linear data of the coefficient data output from the memory means is preliminarily set for each class from a pixel to be thinned out and a plurality of the transmission pixels around the pixel to be thinned out to obtain a linear linear combination model. 10. The image signal transmission device according to claim 9, wherein the coefficient of the combined model is coefficient data obtained by a calculation of a least square method. 11. An image signal transmission apparatus for receiving transmission pixel data in which the number of transmission pixels has been reduced by thinning, and generating a restored image by interpolating the thinned pixels, wherein thinning is performed using transmission pixels. The interpolated value generating means for generating the interpolated pixel value corresponding to the pixel, the edge emphasizing means for performing the edge emphasizing process on the image composed of the interpolated pixel value and the transmission pixel value, and the image after the edge emphasizing. A class classification unit that classifies the thinned pixels according to the states of the interpolation pixel and the pixels around the interpolation pixel, and the class classified by the class classification unit as an address, which corresponds to the thinned pixels. An image signal transmission device, comprising: a memory unit that outputs a predicted value. 12. The predicted value output from the memory means is obtained in advance for each class by learning by a weighted average calculation using the transmission pixel data and the pixel data to be thinned out. 11. The method according to claim 11, wherein
The image signal transmission device according to. 13. The predicted value output from the memory means is obtained in advance for each class by learning by normalization using the transmission pixel data and the pixel data to be thinned out. The image signal transmission device according to claim 11, wherein the image signal transmission device is provided.