JP2006141046A - Image processing apparatus, image processing method, learning apparatus and learning method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accomplish an image processing apparatus of simple configuration determining an interpolation pixel value close to a true value. <P>SOLUTION: In a block formed around a position to be defined as an interpolation target using a plurality of pixels in the vicinity thereof as blocks, flatness in the vicinity of the center is detected, peripheral pixels are selected so that the number of peripheral pixels to be selected at the center of a selection target increases as the flatness is increased, pixels to be generated are classified in accordance with a pattern of level distribution of the peripheral pixels, a predictive coefficient corresponding to the class is generated, and pixels at the position of the interpolation target are calculated by operating the product-sum based on the predictive coefficient and the peripheral pixels. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and an image processing method, and is suitable for application to, for example, improving the resolution of an image by interpolating pixels thinned out by subsampling.

  Conventionally, a method of thinning out pixels of an original image at predetermined intervals by sub-sampling is widely used as a method for band compression or information reduction when recording and transmitting an image signal. As an example, there is a multiple sub Nyquist sampling encoding method in the MUSE (MUltiple Sub-nyquist Sampling Encoding) method.

  As an example of subsampling, offset subsampling is widely used. In this offset sub-sampling, in the two-dimensional case, as shown in FIG. 6, the sampling interval (Tx, Ty) in the horizontal direction (x direction) and the vertical direction (y direction) is set to the pixel interval (Hx in the original signal). , Hy), sub-sample (×) every other pixel. In offset subsampling, sampling points (（) adjacent in the vertical direction are offset by half of the sampling interval (Tx / 2). As a result, the transmission band of the image signal after offset sub-sampling can widen the horizontal or vertical spatial frequency component with respect to the diagonal spatial frequency as shown in FIG. It is possible to perform a thinning process in which deterioration is not noticeable.

  Here, when the offset subsampled image signal is displayed on a monitor or printed out, it is necessary to interpolate pixels between sampling points using adjacent pixels as shown in FIG. Such an interpolation process functions as a spatial filter that passes the frequency components in the hatched area shown in FIG. 7 and prevents the frequency components in the area including the turning point A from passing. Above it is positioned as a post filter.

By the way, offset subsampling is a very effective method when the prefilter before subsampling is applied correctly, but the prefilter cannot be applied sufficiently due to hardware restrictions, for example. In some cases, or when the pre-filter is not sufficiently applied in order to increase the transmission band, there is a problem that image degradation based on aliasing distortion occurs.

  As one method for reducing the occurrence of aliasing distortion, an adaptive interpolation method has been proposed. In this method, when interpolation processing is performed on a subsampled image signal, a direction having a strong correlation is detected around the interpolation pixel, and a plurality of different interpolation means are selectively used according to the detection result. Interpolation processing is performed.

  By the way, in the adaptive interpolation method, the interpolation accuracy largely depends on the detection accuracy when detecting a direction having a strong correlation and the ability of each interpolation means. Therefore, when the ability of each interpolation means is not sufficient and proper interpolation is not possible, or when a direction with strong correlation is misjudged, not only the original signal component is reduced but also the aliasing distortion is increased. There was a problem.

  The present invention has been made in consideration of the above points, and intends to propose an image processing apparatus, an image processing method, a learning apparatus, and a learning method having a simple configuration capable of obtaining an interpolation pixel value close to a true value. It is.

  In order to solve such a problem, the present invention detects a flatness in the vicinity of the center among the blocks formed by using a plurality of pixels in the periphery around the position to be interpolated, and the flatness. Select a peripheral pixel and classify the class of the pixel to be generated according to the level distribution pattern of the peripheral pixel so that the larger the is, the greater the number of peripheral pixels selected at the center to be selected. The prediction coefficient corresponding to the class is generated, and the pixel at the position to be interpolated is calculated by the product-sum operation based on the prediction coefficient and the surrounding pixels.

  Further, the present invention detects the flatness in the vicinity of the center among the blocks formed by using a plurality of pixels in the periphery with the position to be interpolated as the center, and the higher the flatness, the more the selection is made. Select peripheral pixels so that the number of peripheral pixels at the center to be targeted increases, classify the class of pixels to be generated according to the level distribution pattern of the peripheral pixels, and correspond to the classes A representative value is generated, and the representative value is used as data corresponding to a pixel at a position to be interpolated.

  Furthermore, the present invention detects the flatness in the vicinity of the center around the position to be interpolated among the plurality of pixels constituting the learning data corresponding to the known image, and the larger the flatness, Select the peripheral pixels so that the number of peripheral pixels distributed around the center increases, classify the class of pixels to be generated according to the level distribution pattern of the peripheral pixels, and select this class The prediction coefficient corresponding to is generated, and the prediction coefficient is stored in the memory.

  According to the present invention, an interpolation pixel value close to a true value can be obtained by obtaining a pixel at a position to be interpolated using a prediction coefficient or a representative value corresponding to a class. In addition, by detecting the flatness in the vicinity of the position to be interpolated and selecting peripheral pixels to be used for class classification according to the detection result, the number of pixels at the position to be interpolated is reduced. It becomes possible to classify accurately by the number of classes. Thus, an image processing device, an image processing method, a learning device, and a learning method with a simple configuration capable of obtaining an interpolation pixel value close to a true value can be realized.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) Overall Configuration In FIG. 1, reference numeral 1 denotes an image signal interpolation apparatus as a whole, and by applying adaptive interpolation processing by class classification to input image data D1 whose information amount has been reduced by offset subsampling. Interpolation data D2 close to the true value is generated.

Specifically, the image signal interpolating apparatus 1 performs interpolation target pixels (that is, thinned out) in accordance with the level distribution pattern of sampling pixels (that is, pixels included in the input image data D1) around the interpolation target pixels. Pixels) are classified into classes, prediction coefficients obtained by learning for each class in advance are read out, and interpolation pixel values are obtained using the prediction coefficients.
As a result, in the image signal interpolation device 1, it is possible to obtain an interpolated pixel value that is extremely close to the true value as compared with the case where the interpolation target pixel is obtained simply by average interpolation using the surrounding sampling pixels.

  The image signal interpolating apparatus 1 inputs the input image data D1 to the block forming circuit 2, and the block forming circuit 2 divides the input image data D1 into blocks of a predetermined size. Actually, as shown in FIG. 2, the blocking circuit 2 is configured to form each block by twelve peripheral pixels a to l around the pixel to be interpolated (x mark).

  The flatness detection circuit 3 receives the blocked image data D3 and detects the local flatness in the area including the interpolation target pixel in the block. Specifically, the flatness detection circuit 3 detects the flatness using the four peripheral pixels a to d adjacent to the interpolation target pixel (x mark).

  Then, when the flatness of the peripheral pixels a to d is small, it is determined that the class of the interpolation target pixel (x mark) can be sufficiently estimated only by the four pixels a to d, and the subsequent pixel selection circuit 4 is informed. A flatness detection signal D4 indicating this is sent out. On the other hand, when the flatness of the peripheral pixels a to d is large, it is determined that the class of the interpolation target pixel cannot be estimated only by the four pixels a to d, and the subsequent pixel selection circuit 4 is informed of this. The flatness detection signal D4 indicating this is sent out. Incidentally, in the flatness detection circuit 3 of the embodiment, when the difference between the maximum value and the minimum value of the pixel values of the four pixels a to d exceeds 20, the flatness is small, and when the difference is 20 or less. The flatness is large.

  Based on the flatness detection signal D4, the pixel selection circuit 4 adapts only the pixel data of the four pixels a to d out of the pixels a to l in the block as the selected pixel data D5 when the flatness is low. The data is sent to an encoding (ADRC (Adaptive Dynamic Range Coding)) circuit 5. On the other hand, when the flatness is large, the pixel selection circuit 4 obtains a total of nine pieces of pixel data of the peripheral pixels e to l together with the average value M of the four pixels a to d as the selection pixel data D5. To send.

  The ADRC circuit 4 receives the selected pixel data D5 from the pixel selection circuit 4, and performs adaptive dynamic range encoding processing on the selected pixel data D5 to perform bit compression on each pixel value, thereby obtaining pattern compressed data D6. Form.

  As described above, in the image signal interpolating apparatus 1, the classification is not always performed using all the peripheral pixels a to l cut out by the blocking circuit 2, but according to the flatness of the local peripheral pixels a to d. By adaptively selecting the number of pixels used for class classification, the number of classes can be effectively reduced.

As a result, when configuring the ADRC circuit 5 followed by performs adaptive quantization of example 1 bit, when the flatness is small, resulting representing the state of the periphery of the interpolation target pixel with two ^four patterns, when a large flatness 2 It can be expressed by ⁹ patterns. Therefore, considering that there are 2 ¹² patterns when all the surrounding 12 pixels are used, it can be seen that the interpolation target pixels can be expressed with very few patterns.

  In addition, the ADRC circuit 5 switches the bit compression rate based on the flatness detection signal D4 from the flatness detection circuit 3. Specifically, when a flatness detection result signal D4 indicating that the flatness is small is received, each pixel value of 8 bits per pixel is compressed to 2 bits for 4 pixels a to d. On the other hand, when the flatness detection signal D4 indicating that the flatness is large is received, a total of nine pixels of the average pixel value M and the peripheral eight pixels e to l are compressed to 1 bit.

State of the slave connexion peripheral pixels, when the flatness is small is represented by 4 ^four patterns, when a large flatness is expressed by 2 ^nine patterns. As a result, the ADRC circuit 5 can perform efficient compression processing while leaving the feature values of the peripheral pixels.

  The pattern compression data D6 is given to the class code generation circuit 6. The class code generation circuit 6 generates a class code D6 based on the pattern compression data D6 and supplies it to the prediction coefficient memory 7. The prediction coefficient memory 7 outputs a prediction coefficient D8 that is obtained and stored in advance by learning to be described later for each class by using the class code D6 as a read address.

  The prediction calculation circuit 8 calculates the pixel value of the pixel to be interpolated by performing a calculation based on the linear linear combination formula using the pixel values a to l and the prediction coefficient D8 included in the blocked image data D3. Is output as interpolation data D2.

  Thus, in the image signal interpolation device 1, an interpolation pixel value close to a true value can be obtained by forming an interpolation pixel using a prediction coefficient obtained by learning in advance. In addition, by selecting the pixels used for class classification according to the flatness and changing the compression rate at the time of class classification, the efficiency of class classification can be remarkably improved. As a result, the configuration of the prediction coefficient memory 7 can be simplified by reducing the number of classes.

(2) Class Classification Processing Next, class classification processing by the ADRC circuit 5 and the class code generation circuit 6 will be described. The ADRC circuit 5 defines an intra-block dynamic range as a local feature of the image, and adaptively removes redundancy mainly in the level direction. For example, as shown in FIG. 3, in the dynamic range of 0 to 255 of pixel data of 8 bits per pixel, the in-block dynamic range A and B necessary for requantization for each block are as follows. It can be seen that it is significantly smaller. Therefore, if requantization is performed within the small dynamic ranges A and B, the number of necessary bits can be greatly reduced.

  Specifically, the ADRC circuit 5 first sets the dynamic range in the block as DR, bit allocation as p, each pixel level in the block as x, and the requantization code as Q.

Thus, as shown in FIG. 4A, requantization is performed by equally dividing the maximum value MAX and the minimum value MIN in the block by the designated bit length. Incidentally, FIG. 4 shows a case where requantization is performed with 3 bits (that is, p = 3).

Next, an average value of the pixels in the block existing in the data level corresponding to the highest gradation level (2 ^p −1) in FIG. 4A is taken, and this is obtained as shown in FIG. The maximum value is MAX ′. Also, the average value of the pixels in the block existing in the data level corresponding to the lowest gradation level 0 in FIG. 4A is taken, and this is set as the minimum value MIN ′ as shown in FIG. 4B.

  Next, the dynamic range DR ′ in the block is newly redefined with the newly determined maximum value MAX ′ and minimum value MIN ′, and the requantized code is defined as q, and the maximum value MAX ′ and minimum value in the newly determined block. Based on the value MIN ′,

Thus, requantization as shown in FIG.

  In this way, the ADRC circuit 5 performs double requantization, thereby reducing the adverse effects of noise and performing efficient information compression to form pattern compressed data D6. However, the ADRC circuit 5 of the embodiment does not re-quantize with 3 bits, but re-quantizes with 1 bit or 2 bits according to the flatness detection signal D4 as described above.

The class classification circuit 6 uses q ₁ to q _n as re-quantized data obtained as a result of data compression to p bits by the ADRC circuit 5 in this way, and

Is executed to determine the class code c (D7) representing the class to which the block belongs, that is, the class of the pixel to be interpolated.

(3) Creation of Prediction Coefficient Memory Next, how to obtain the prediction coefficient stored in the prediction coefficient memory 7 will be described with reference to FIG. First, in order to learn the prediction coefficient in step SP1, learning data corresponding to an already known image is formed. Specifically, the pixel values before sub-sampling of the sampling pixels a to l and the interpolation target pixel (x mark) in FIG.

  In addition, when forming learning data in this way, a more accurate prediction coefficient can be obtained if a large number of learning data is formed by using a plurality of images instead of using only one image. .

  In step SP2, it is determined whether necessary and sufficient learning data has been formed. If it is determined that more learning data is required, the process proceeds to step SP3. If it is determined that sufficient learning data is obtained, the process proceeds to step SP4.

  In step SP3, the learning data is classified. At this time, the same class classification as in the image signal interpolation apparatus 1 described above is performed. Specifically, first, the local flatness of the learning sampling data is detected, a pixel used for class classification is selected according to the detection result, and the class code is formed after the selected pixel is compressed by ADRC encoding. This classifies each learning data.

Next, in step SP5, a normalization equation is formed for each class based on the learning data classified into classes. The processing at step SP5 will be specifically described. Here, for generalization, a case where n sampling pixels exist as learning data will be described. First pixel level x ₁ of each sampling _pixel, ..., a sub-sample previous relationship pixel level y of the target interpolation pixel and x _n, prediction coefficients w ₁ for each class _(c), ..., n according to w _n taps By expressing the linear primary prediction formula of

Form.

Prediction coefficients _w 1 in the equation (4), ..., may be obtained a _{w n.} Therefore the prediction coefficients w _1, such as errors in the actual interpolation target pixel and the interpolation processing result is _minimized, ..., seek w _n. It is performed for a plurality of learning data learned for each class, where the prediction coefficients w _1, if the number of learning data is a general m> n When _m, ......, w _n can not be uniquely determined . Therefore, the element of the error vector e is set to e _k as a prediction error in each learning data x _k1 ,..., X _kn , y _k (k = 1, 2,..., M),

Is defined as

Prediction coefficients _w 1 to minimize, ..., seek _{w n.} This is a so-called least square method.

Here, when the partial differential coefficient by w _{i in} equation (6) is obtained,

It becomes. (7) may be determined each w _i such expression becomes zero. Therefore, the following formula

as well as

If X _ij and Y _i are defined as follows, the above equation (7) uses a matrix and the following equation:

Can be rewritten as

Since this case (10) normal equation of formula is unknowns are n simultaneous equations, thereby the unknown coefficients are the most probable value w _1, ......, it can be obtained w _n. That is, in this prediction coefficient calculation processing procedure, steps SP1-SP2-SP3-SP5-SP1 are performed until a normalization equation that can determine undetermined coefficients w ₁ ,..., W _n for each class in step SP5 can be formed. Repeat the loop.

Eventually, in step SP5, a normalization equation expressed by equation (10) is formed for each class. If a positive result is obtained in step SP2, the process proceeds to step SP4, where the normalization equation of equation (10) is solved. The prediction coefficients w ₁ ,..., W _n for each class are determined. Specifically, since the matrix on the left side of equation (10) is generally positive definite, it can be solved by the Cholesky method.

Next, in step SP6, the prediction coefficients w ₁ ,..., W _n determined for each class are stored in the address of the corresponding class in the prediction coefficient memory 7, and the prediction coefficient calculation processing procedure is terminated in the subsequent step SP7. To do.

(4) Operation of Embodiment In the above configuration, the image signal interpolation device 1 classifies the thinned pixels as interpolation target pixels and classifies the interpolation target pixels according to the state of the surrounding block data D3. At this time, as preprocessing for class classification, the image signal interpolation apparatus 1 detects the flatness of the pixels a to d in the vicinity of the interpolation target pixel, and selects peripheral pixels used for class classification based on the magnitude of the flatness. As a result, it is possible to perform class classification using the minimum necessary pixels while leaving as much image feature quantity as possible, and it is possible to suppress an increase in the number of unnecessary classes and to perform efficient class classification.

  Next, the image signal interpolating apparatus 1 forms pattern compressed data D6 by compressing the selected pixel, and forms a class code D7 based on the pattern compressed data D6. At this time, the image signal interpolating apparatus 1 changes the compression rate depending on the degree of flatness. As a result, it is possible to perform sufficient compression processing while leaving the feature amount of the image, and it is possible to suppress an unnecessary increase in the number of classes.

Next, the image signal interpolation apparatus 1 reads the prediction coefficient D8 stored in the prediction coefficient memory 7 using the class code D7 as a read address. Then, the prediction calculation circuit 8 obtains the prediction coefficients w ₁ (c) to w ₁₂ (c) corresponding to the class c and the pixel data x _{1 to} x _{12 of the} sampling pixels a to l included in the blocked data D3. ,

  In this way, the image interpolation apparatus 1 can form interpolation data D2 that is almost the same as the pixel data included in the original image. The interpolation data D2 is combined with the input image data D1 by a combining circuit (not shown), and then supplied to, for example, a television receiver or a video tape recorder.

(5) Effects of the embodiment According to the above configuration, the local flatness of the peripheral pixels including the interpolation target pixel is detected, and the pixel used for class classification is selected and the class is selected according to the flatness detection result. By changing the compression rate at the time of classification, the interpolation target pixel can be accurately classified with a small number of classes. As a result, the image signal interpolation device 1 having a simple configuration capable of forming interpolation data close to the true value can be realized.

(6) Other Embodiments In the above-described embodiment, the flatness is detected from the four neighboring pixels a to d of the interpolation target pixel, and when the flatness is small, only these four pixels a to d are class. The case where the classification is performed using the average value M of the four pixels a to d and the peripheral pixels e to l distributed in a wider range when the flatness is large has been described. The number of pixels to be used and the number of pixels to be selected according to the flatness detection result are not limited to this. In short, when the flatness is detected using pixels near the interpolation target pixel and the flatness is large, the flatness The class classification may be performed using more pixels than the pixels used when.

  In the above-described embodiment, the case where the number of pixels used for class classification is changed and the compression rate of the ADRC circuit 5 is changed according to the flatness detection result is described. However, only the number of pixels used for class classification is changed. Anyway. Further, when changing the compression ratio, it is not limited to switching between 1 bit and 2 bits as described above.

  Further, in the above-described embodiment, the case where the selected pixel data D5 selected by the image selection circuit 4 is classified by bit compression using the ADRC circuit 5 and the class code generation circuit 6 has been described. The classification means is not limited to the ADRC circuit 5 and the class code generation circuit 6, and various compression means such as discrete cosine transform (DCT), differential quantization (DPCM), subband coding, and wavelet transform can be used. . Further, the class classification method is not limited to the bit compression method. For example, a direction having a strong correlation may be detected in the peripheral image of the interpolation target pixel, and the class classification may be performed based on the detection result.

  In the above-described embodiment, the case where the prediction coefficient for each class is obtained by learning by the least square method has been described. However, the method for obtaining the prediction coefficient is not limited to this, and various learning methods can be used.

  In the above embodiment, the prediction coefficient memory 7 and the prediction arithmetic circuit 8 are provided, the prediction coefficient D8 stored in the prediction coefficient memory 7 by learning in advance is read according to the class code D7, and the read prediction coefficient D8 and block are read. In the above description, the interpolation data is obtained by linearly combining the digitized data D3. However, the present invention is not limited to this, and instead of the prediction coefficient memory 7 and the prediction arithmetic circuit 8, a class obtained by learning in advance is used. It is also possible to provide a memory for storing each representative value, read the representative value corresponding to the class, and use the read representative value as interpolation data.

  In this case, as a first method for obtaining data in the memory, there is learning by weighted average. More specifically, the true pixel values corresponding to the interpolation target pixels are accumulated for each class, and the result of dividing the accumulation result by the number of accumulated pixel values is performed on various images to class. When you get a representative value for each, it is learning.

  As a second method for obtaining the representative value, there is learning by normalization. More specifically, a block formed of a plurality of pixels including an interpolation target pixel is formed, and a block reference value is subtracted from a true pixel value corresponding to the interpolation target pixel by a dynamic range in the block. Is obtained by dividing the cumulative value of the normalized values by the cumulative frequency, and performing processing on various images to obtain representative values for each class.

  The present invention can be used, for example, in the field of image processing for improving the resolution of an image.

It is a block diagram which shows the structure of one Example of the image signal interpolation apparatus by this invention. It is a basic diagram with which it uses for description of the surrounding pixel used for the class classification | category process of an Example. It is a basic diagram with which it uses for description of the bit compression by an ADRC circuit. It is a basic diagram with which it uses for description of the bit compression by an ADRC circuit. It is a flowchart which shows the prediction coefficient calculation processing procedure by learning. It is a basic diagram with which it uses for description of offset subsampling. It is a basic diagram which shows the spatial frequency spectrum of the zone | band which can be transmitted by two-dimensional offset subsampling. It is an approximate line figure used for explanation of interpolation processing.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image signal interpolation apparatus, 3 ... Flatness detection circuit, 4 ... Pixel selection circuit, 5 ... ADRC circuit, 6 ... Class code generation circuit, 7 ... Prediction coefficient memory, 8 ... Prediction arithmetic circuit , D1... Input image data, D2... Interpolation data, D3... Blocking data, D4... Flatness detection signal, D5... Selected pixel data, D6. …… Prediction coefficient, a to l …… Peripheral pixels.

Claims (22)

Flatness detection means for detecting the flatness in the vicinity of the center around the position to be interpolated among the plurality of pixels constituting the input image data;
Peripheral pixel selection means for selecting the peripheral pixels such that the greater the flatness detected by the flatness detection means, the greater the number of peripheral pixels distributed around the center;
Class classification means for classifying a pixel class at the position to be interpolated according to the level distribution pattern of the peripheral pixels selected by the peripheral pixel selection means;
Prediction coefficient generation means for generating a prediction coefficient corresponding to the class classified by the class classification means;
Image processing comprising: pixel calculation means for calculating a pixel at the position to be interpolated by a product-sum operation based on the prediction coefficient generated by the prediction coefficient generation means and the surrounding pixels. apparatus. The flatness detection means includes
When the difference between the maximum and minimum pixel values in the vicinity of the center exceeds a predetermined value, it is detected that the flatness is large, and when the difference is equal to or less than the above value, the flatness is The image processing apparatus according to claim 1, wherein the image processing apparatus is detected as a small one. The pixel selection means includes
When the level of flatness detected by the flatness detection means is below a predetermined reference, peripheral pixels distributed within the first range from the center were selected and exceeded the reference In this case, the image processing apparatus according to claim 1, wherein peripheral pixels distributed in a second range wider than the first range from the center are selected. The image processing apparatus according to claim 3, wherein a part of the pixels distributed in the second range is averaged. The classification means is
Reducing the amount of information on the peripheral pixels selected by the pixel selection means, and classifying the pixel class at the position to be interpolated according to the reduced level distribution pattern of the peripheral pixels. The image processing apparatus according to claim 1. The classification means is
The image processing apparatus according to claim 5, wherein the information amount of peripheral pixels selected by the pixel selection unit is reduced by adaptive dynamic range coding (ADRC). The classification means is
If the level of flatness detected by the flatness detection means is less than or equal to a predetermined reference, the peripheral pixels selected by the pixel selection means are reduced to a first information amount, and 7. If the reference is exceeded, the peripheral pixels selected by the pixel selection means are reduced so that the second information amount is smaller than the first information amount. An image processing apparatus according to 1. The pixel selection means includes
When the magnitude of the flatness detected by the flatness detection means is less than or equal to a predetermined reference, peripheral pixels distributed within the first range from the center are selected and the reference is exceeded. In the case, the peripheral pixels distributed in the second range wider than the first range from the center are selected,
The classification means is
When peripheral pixels distributed within the first range are selected by the pixel selection unit, the peripheral pixels are reduced to the first information amount, and the second range is selected by the pixel selection unit. When the peripheral pixels distributed within are selected, the peripheral pixels are reduced so as to have an information amount smaller than the first information amount, and according to the level distribution pattern of the reduced peripheral pixels, The image processing apparatus according to claim 1, wherein a class of pixels at a position to be interpolated is classified. The prediction coefficient generating means is:
The image processing apparatus according to claim 1, further comprising: a memory that stores a prediction coefficient for each class obtained by learning in advance, and generating a prediction coefficient corresponding to the class classified by the class classification unit. . Flatness detection means for detecting the flatness in the vicinity of the center around the position to be interpolated among the plurality of pixels constituting the input image data;
Peripheral pixel selection means for selecting the peripheral pixels such that the greater the flatness detected by the flatness detection means, the greater the number of peripheral pixels distributed around the center;
Class classification means for classifying a pixel class at the position to be interpolated according to the level distribution pattern of the peripheral pixels selected by the peripheral pixel selection means;
Representative value generating means for generating a representative value corresponding to the class classified by the class classification means,
An image processing apparatus, wherein the representative value generated by the representative value generating means is data corresponding to a pixel at the position to be interpolated. The flatness detection means includes
When the difference between the maximum and minimum pixel values in the vicinity of the center exceeds a predetermined value, it is detected that the flatness is large, and when the difference is equal to or less than the above value, the flatness is The image processing apparatus according to claim 10, wherein the image processing apparatus is detected as a small one. The pixel selection means includes
When the level of flatness detected by the flatness detection means is below a predetermined reference, peripheral pixels distributed within the first range from the center were selected and exceeded the reference In this case, the image processing apparatus according to claim 10, wherein peripheral pixels distributed in a second range wider than the first range from the center are selected. The image processing apparatus according to claim 12, wherein some of the pixels distributed in the second range are averaged. The classification means is
Reducing the amount of information on the peripheral pixels selected by the pixel selection means, and classifying the pixel class at the position to be interpolated according to the reduced level distribution pattern of the peripheral pixels. The image processing apparatus according to claim 10. The classification means is
The image processing apparatus according to claim 14, wherein the information amount of peripheral pixels selected by the pixel selection unit is reduced by adaptive dynamic range coding (ADRC). The classification means is
If the level of flatness detected by the flatness detection means is less than or equal to a predetermined reference, the peripheral pixels selected by the pixel selection means are reduced to a first information amount, and 16. If the reference is exceeded, the neighboring pixels selected by the pixel selection means are reduced so that the second information amount is smaller than the first information amount. An image processing apparatus according to 1. The pixel selection means includes
When the magnitude of the flatness detected by the flatness detection means is less than or equal to a predetermined reference, peripheral pixels distributed within the first range from the center are selected and the reference is exceeded. In the case, the peripheral pixels distributed in the second range wider than the first range from the center are selected,
The classification means is
When peripheral pixels distributed within the first range are selected by the pixel selection unit, the peripheral pixels are reduced to the first information amount, and the second range is selected by the pixel selection unit. When the peripheral pixels distributed within are selected, the peripheral pixels are reduced so as to have an information amount smaller than the first information amount, and according to the level distribution pattern of the reduced peripheral pixels, The image processing apparatus according to claim 10, wherein a pixel class at a position to be interpolated is classified. The representative value generating means is:
The image processing apparatus according to claim 10, further comprising a memory that stores a prediction coefficient for each class obtained in advance by learning, and that generates a prediction coefficient corresponding to the class classified by the class classification unit. . A first step of detecting flatness in the vicinity of the center of a position to be interpolated among a plurality of pixels constituting the input image data;
A second step of selecting the peripheral pixels such that the greater the flatness detected in the first step, the greater the number of peripheral pixels distributed around the center;
A third step for classifying a pixel class at the position to be interpolated according to a level distribution pattern of peripheral pixels selected in the second step;
A fourth step for generating a prediction coefficient corresponding to the class classified in the third step;
An image comprising: a prediction coefficient generated in the fourth step; and a fifth step for calculating a pixel at the position to be interpolated by a product-sum operation based on the surrounding pixels. Processing method. A first step of detecting flatness in the vicinity of the center of a position to be interpolated among a plurality of pixels constituting the input image data;
A second step of selecting the peripheral pixels such that the greater the flatness detected in the first step, the greater the number of peripheral pixels distributed around the center;
A third step for classifying a pixel class at the position to be interpolated according to a level distribution pattern of peripheral pixels selected in the second step;
A fourth step for generating a representative value corresponding to the class classified in the third step,
An image processing method characterized in that the representative value generated in the fourth step is used as data corresponding to a pixel at a position to be interpolated. Flatness detection means for detecting the flatness in the vicinity of the center around a position to be interpolated among a plurality of pixels constituting learning data corresponding to a known image;
Peripheral pixel selection means for selecting the peripheral pixels such that the greater the flatness detected by the flatness detection means, the greater the number of peripheral pixels distributed around the center;
Class classification means for classifying a pixel class at the position to be interpolated according to the level distribution pattern of the peripheral pixels selected by the peripheral pixel selection means;
A learning apparatus comprising: prediction coefficient storage means for generating a prediction coefficient corresponding to the class classified by the class classification means and storing the prediction coefficient in a memory. A first step of detecting flatness in the vicinity of the center around a position to be interpolated among a plurality of pixels constituting learning data corresponding to a known image;
A second step of selecting the peripheral pixels such that the greater the flatness detected in the first step, the greater the number of peripheral pixels distributed around the center;
A third step for classifying a pixel class at the position to be interpolated according to a level distribution pattern of peripheral pixels selected in the second step;
A learning method comprising: generating a prediction coefficient corresponding to the class classified in the third step, and storing the prediction coefficient in a memory.

Images