JP2000341706A - Picture signal processor, its processing method, learning device and method, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To highly accurately obtain a picture signal of high resolution by providing a picture signal processor with a pixel generation means for generating a pixel having a color component different from at least the color component included in a remarked pixel on the basis of a class determined by a class determination step or the like. SOLUTION: A gamma correction part 24 corrects a signal level of a picture signal whose white balance is adjusted by a white balance adjustment part 23 in accordance with a gamma curve. The picture signal gamma corrected by the gamma correction part 24 is supplied to a prediction processing part 25. The prediction processing part 25 extracts a plurality of pixels existing near a remarked pixel in each remarked pixel of an input picture signal from the input picture signal and determines a class on the basis of a color component of the pixel having the highest density out of plural extracted pixels. Then class sorting adaptive processing for generating the pixel having the color component different from the color component included in the remarked pixel on the position of the remarked pixel on the basis of the determined class is applied to generate a picture signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing device, an image signal processing method, a learning device, a learning method, and a recording medium, and more particularly to an image signal processing device capable of obtaining a higher definition image. The present invention relates to a device, an image signal processing method, a learning device, a learning method, and a recording medium.

[0002]

2. Description of the Related Art An imaging apparatus using a solid-state image sensor such as a CCD (Charge Coupled Device) image sensor is mainly a single-panel type using one CCD image sensor (hereinafter, referred to as a single-panel camera). And three C
Three-plate system using a CD image sensor (hereinafter referred to as 3
Board camera).

In a three-panel camera, for example, three CCD image sensors for R signal, G signal and B signal are used, and three primary color signals are obtained by the three CCD image sensors. Then, a color image signal generated from the three primary color signals is recorded on a recording medium.

[0004] In a single-panel camera, a color coding filter composed of a color filter array assigned to each pixel is used by using a single CCD image sensor provided on the front surface of a single color CCD. A signal is obtained for each pixel. Examples of the color filter array constituting the color coding filter include primary color filter arrays of R (Red), G (Green), and B (Blue), Ye (Yellow), Cy (Cyanogen), M
g (Magenta) complementary color filter array is used. In the single-panel camera, one color component signal is obtained for each pixel by the CCD image sensor, and color signals other than the color component signals of each pixel are generated by linear interpolation processing. An image close to the image obtained by the plate camera was obtained. 2. Description of the Related Art In a video camera or the like, a single-panel type is used for miniaturization and weight reduction.

In a single-panel camera, for example, a CCD image sensor provided with a color coding filter composed of a color filter array having a color arrangement as shown in FIG. 21A is used for three primary colors of R, G and B. From each pixel on which a filter of one of the colors is arranged, only an image signal corresponding to the color of the filter is output. That is, from the pixels in which the R color filters are arranged, the R component image signal is output, but the G component and B component image signals are not output. Similarly, only the G component image signal is output from the G pixel, no R component and B component image signals are output, and only the B component image signal is output from the B pixel. And the G component image signal are not output.

The color arrangement of the color filter array shown in FIG. 21A is called a Bayer arrangement. In this case, the G color filters are arranged in a checkered pattern, and R and B are alternately arranged in the remaining portion for each row.

However, when the signal of each pixel is processed in the subsequent stage, image signals of R component, G component and B component are required for each pixel. Therefore, as shown in FIGS. 21B to 21D, the output of a CCD image sensor composed of n × m (n and m are positive integers) pixels as shown in FIGS. An image signal of R pixels, an image signal of n × m G pixels, and an image signal of n × m B pixels, that is,
Image signals corresponding to the CCD output of the three-chip camera are obtained by linear interpolation calculations, and these image signals are output to the subsequent stage.

[0008]

However, in the single-chip camera described above, since the color signals are interpolated by performing the linear processing, the resolution of the image based on the image signal obtained as the imaging output is three-chip type. Lower than the image by the image signal obtained as the imaging output of the camera,
There is a problem that an image becomes blurred as a whole due to the influence of the linear processing.

In addition, by the classification adaptive processing, the output of the CCD of the single-chip camera is converted to the CC of the three-chip camera.
To obtain an output equivalent to the D output, as a class tap,
When predicting the R pixel by independently selecting each pixel of RGB, only the R pixel is used as a class tap,
When predicting G pixels, only G pixels are used as prediction taps, and when predicting B pixels, only B pixels are used as prediction taps. For example, as shown in FIG. In a case where a Bayer array is used as a color filter array for a CCD image sensor of a single-chip camera, R pixels and B pixels are only one in four pixels out of n × m pixels. Since it does not exist, accurate classification adaptive processing cannot be performed.

The present invention has been made in view of such a situation, and an object of the present invention is to make it possible to execute the classification adaptive processing with high accuracy.

[0011]

According to the present invention, there is provided an image signal processing apparatus for processing an input image signal having a color component representing one of a plurality of colors for each pixel position. Extracting means for extracting a plurality of pixels having a color component having the highest density among the plurality of color components for each pixel of interest and located in the vicinity of the pixel of interest; and a plurality of pixels extracted by the extracting means And a pixel generation unit that generates a pixel having a color component different from at least the color component of the pixel of interest based on the class determined in the class determination step. It is characterized by the following.

According to another aspect of the present invention, there is provided an image signal processing method for processing an input image signal having a color component representing any one of a plurality of colors at each pixel position. An extraction step of extracting a plurality of pixels having a color component having the highest density among the plurality of color components and located in the vicinity of the pixel of interest, based on the plurality of pixels extracted in the extraction step, A class determining step of determining a class, and a pixel generating step of generating a pixel having a color component different from at least the color component of the target pixel based on the class determined in the class determining step. I do.

According to the present invention, there is provided a recording apparatus in which a computer-controllable program for performing image signal processing for processing an input image signal having a color component representing one of a plurality of colors for each pixel position is recorded. In the medium, the program includes, for each pixel of interest of the input image signal, a color component having the highest density among the plurality of color components, and extracting a plurality of pixels located near the pixel of interest. A class determining step of determining a class based on the plurality of pixels extracted in the extracting step; and a color component different from at least the color component of the pixel of interest based on the class determined in the class determining step And a pixel generation step of generating a pixel having the following.

Further, the learning apparatus according to the present invention is located near a target pixel of a student image signal having a color component representing any one of a plurality of colors for each pixel position, and First pixel extracting means for extracting a plurality of pixels having a color component having the highest density; class determining means for determining a class based on the plurality of pixels extracted by the first pixel extracting means; Extracting a plurality of pixels near a position corresponding to a position of a target pixel of the student image signal from a teacher image signal which is an image signal corresponding to the student image signal and has a plurality of color components for each pixel position. Two pixel extracting means, and the teacher image signal from the image signal corresponding to the student image signal for each class based on the pixel values of the plurality of pixels extracted by the first and second pixel extracting means. Produces an image signal equivalent to Characterized in that it comprises a prediction coefficient generating means for generating a prediction coefficient set to be used for prediction calculation for.

Further, the learning method according to the present invention is arranged such that each of the plurality of color components is located near a target pixel of a student image signal having a color component representing any one of a plurality of colors at each pixel position. A first pixel extraction step of extracting a plurality of pixels having a color component having the highest density, a class determination step of determining a class based on the plurality of pixels extracted in the first pixel extraction step, Extracting a plurality of pixels near a position corresponding to a position of a target pixel of the student image signal from a teacher image signal which is an image signal corresponding to the student image signal and has a plurality of color components for each pixel position. (2) pixel extraction step, and, based on the pixel values of the plurality of pixels extracted by the first and second pixel extraction means, the image signal corresponding to the student image signal and the teacher image signal for each of the classes. Equivalent to It is characterized and a prediction coefficient generation step of generating a prediction coefficient set to be used for prediction calculation for generating the image signal that.

Further, the present invention provides a recording medium on which a computer-controllable program for performing a learning process for generating a prediction coefficient set corresponding to a class is recorded.
The above program is located near a pixel of interest of a student image signal having a color component representing any one of a plurality of colors for each pixel position, and has a color component having the highest density among the plurality of color components. A first pixel extracting step of extracting a plurality of pixels, a class determining step of determining a class based on the plurality of pixels extracted in the first pixel extracting step, and an image signal corresponding to the student image signal A second pixel extraction step of extracting, from a teacher image signal having a plurality of color components for each pixel position, a plurality of pixels near a position corresponding to the position of the pixel of interest of the student image signal, An image signal corresponding to the teacher image signal is generated from an image signal corresponding to the student image signal for each of the classes based on the pixel values of the plurality of pixels extracted by the first and second pixel extraction means. Characterized in that it comprises a prediction coefficient generation step of generating a prediction coefficient set to be used for the prediction calculation.

[0017]

Embodiments of the present invention will be described below in detail with reference to the drawings.

The present invention is applied to, for example, a digital still camera 1 having a configuration as shown in FIG. The digital still camera 1 is a single-panel camera that performs color imaging using a single CCD image sensor 5 provided with a color coding filter 4 composed of color filters assigned to each pixel on the front side. Is condensed by the lens 2 and is incident on the CCD image sensor 5 via the iris 3 and the color coding filter 4. A subject image is formed on the imaging surface of the CCD image sensor 5 by the incident light having a predetermined level of light quantity by the iris 3. In the digital still camera 1, the color coding filter 4 and the CCD image sensor 5 are provided separately, but may be integrated.

The CCD image sensor 5 performs exposure for a predetermined time in accordance with an electronic shutter controlled by a timing signal from a timing generator 9, and outputs a signal charge (corresponding to the amount of incident light transmitted through the color coding filter 4). By generating (an analog amount) for each pixel, a subject image formed by the incident light is captured, and an image signal obtained as an image output is supplied to the signal adjustment unit 6.

The signal adjusting section 6 adjusts the gain so that the signal level of the image signal becomes constant.
ain Control) circuit and a CDS (Correiated Doubl) for removing 1 / f noise generated by the CCD image sensor 5.
le Sampling) circuit.

The image signal output from the signal adjuster 6 is converted from an analog signal to a digital signal by an A / D converter 7 and supplied to an image signal processor 8. The A / D converter 7 generates a digital imaging signal of, for example, 10 bits per sample according to the timing signal from the timing generator 9.

In the digital still camera 1, the timing generator 9 includes the CCD image sensor 5,
Signal adjustment unit 6, A / D conversion unit 7, and CPU (Central Pro
cessing Unit) 10 to supply various timing signals.
The CPU 10 controls the iris 3 by driving the motor 11. Further, the CPU 10 drives the motor 12 to move the lens 2 and the like, and controls zoom, autofocus, and the like. Furthermore, CPU
Numeral 10 controls the flash 13 to emit a flash as needed.

The image signal processing unit 8 performs prediction on the image signal supplied from the A / D conversion unit 7 using defect correction processing, digital clamping processing, white balance adjustment processing, gamma correction processing, and class classification adaptive processing. Perform processing such as processing.

The memory 15 connected to the image signal processing unit 8 is composed of, for example, a RAM (Random Access Memory), and stores signals required when the image signal processing unit 8 performs image processing. The image signal processed by the image signal processing unit 8 is stored in the memory 16 via the interface 14. The image signal stored in the memory 16 is recorded on a recording medium 17 detachable from the digital still camera 1 via the interface 14.

The motor 11 drives the iris 3 based on control information from the CPU 10 to control the amount of light incident through the lens 2. Also, the motor 12
CCD of lens 2 based on control information from CPU 10
The focus state for the image sensor 2 is controlled.
Thereby, an automatic aperture control operation and an automatic focus control operation are realized. The flash 13 irradiates a predetermined flash to the subject under the control of the CPU 10.

The interface 14 stores the image signal from the image signal processing unit 8 in the memory 16 as necessary, and after executing a predetermined interface process, supplies the image signal to the recording medium 17 for storage. As the recording medium 17, a recording medium detachable from the main body of the digital still camera 1, for example, a disk recording medium such as a floppy disk or a hard disk, or a flash memory such as a memory card can be used.

Under the control of the CPU 10, the controller 18 supplies control information to the image signal processing section 8 and the interface 14 to control them. Operation information by the user is input to the CPU 10 from an operation unit 20 including operation buttons such as a shutter button and a zoom button. The CPU 10 determines, based on the input operation information,
The above-described units are controlled. The power supply unit 19 includes a battery 19
A and a DC / DC converter 19B. DC /
The DC converter 19B converts the power from the battery 19A into a DC voltage having a predetermined value, and supplies the DC voltage to each component in the device. The rechargeable battery 19A is detachable from the main body of the digital still camera 1.

Next, the operation of the digital still camera 1 shown in FIG. 1 will be described with reference to the flowchart of FIG. In step S1, the digital still camera 1 starts imaging a subject when the power is turned on. That is, the CPU 10 drives the motor 11 and the motor 12 to form a subject image on the CCD image sensor 5 via the lens 2 by focusing and adjusting the iris 3.

In step S2, the formed image is converted to a CCD image.
The image signal picked up by the image sensor 5 is gain-adjusted in the signal adjuster 6 so that the signal level becomes constant, noise is removed, and the signal is digitized by the A / D converter 7.

In step S3, the image signal processing unit 8 performs image signal processing including class classification adaptive processing on the image signal digitized by the A / D conversion unit 7.

Here, the user can confirm the object image by displaying an image signal obtained as an image pickup output of the CCD image sensor 5 on an electronic viewfinder. In addition, the subject image can be confirmed by the user using an optical viewfinder.

When the user wants to record the image of the subject image confirmed by the viewfinder on the recording medium 17, the user operates the shutter button of the operation unit 20. In step S4, the CPU 10 of the digital still camera 1 determines whether the shutter button has been operated. The digital still camera 1 repeats the processing of steps S2 to S3 until it determines that the shutter button has been operated, and proceeds to step S5 when it determines that the shutter button has been operated.

In step S5, the image signal subjected to the image signal processing by the image signal processing section 8 is recorded on the recording medium 17 via the interface 14.

Next, the image signal processing section 8 will be described with reference to FIG.

The image signal processing section 8 includes a defect correction section 21 to which the image signal digitized by the A / D conversion section 7 is supplied. Among the pixels of the CCD image sensor 5, pixels that do not respond to incident light for some reason,
Pixels in which electric charges are always stored, that is, defective pixels are detected without depending on incident light, and the image signal is corrected according to the detection result so that the influence of the defective pixels is not exposed. Perform processing.

In the A / D converter 7, in order to prevent the negative value from being cut, the A / D conversion is generally performed with the signal value slightly shifted in the positive direction. Clamp part 2
Reference numeral 2 clamps the image signal corrected for defects by the defect correction unit 21 so that the above-described shift amount is eliminated.

The image signal clamped by the clamp unit 22 is supplied to a white balance adjustment unit 23. The white balance adjustment unit 23 adjusts the white balance by correcting the gain of the image signal supplied from the clamp unit 22. This white balance adjustment unit 2
The image signal whose white balance has been adjusted by 3 is supplied to the gamma correction unit 24. The gamma correction unit 24 corrects the signal level of the image signal whose white balance has been adjusted by the white balance adjustment unit 23 according to a gamma curve. The image signal gamma-corrected by the gamma correction unit 24 is supplied to the prediction processing unit 25.

The prediction processing unit 25 converts the output of the gamma correction unit 24 into an image signal equivalent to, for example, a CCD output of a three-panel camera by performing the classification adaptive processing, and supplies the image signal to the correction unit 26. The prediction processing unit 25 includes a blocking unit 28, an ADRC (Adaptive Dynamic Range Coding) processing unit 29, a class classification unit 30, an adaptive processing unit 31, a coefficient memory 32, and the like.

The blocking section 28 supplies an image signal of a class tap described later to the ADRC processing section 29 and supplies an image signal of a prediction tap to the adaptive processing section 31. The ADRC processing unit 29 performs an ADRC process on the input image signal of the class tap to generate a requantized code, and supplies the requantized code to the class classification unit 30 as feature information. ADRC
The image signal patterns are classified based on the feature information supplied from the processing unit 29, and a class number (class code) indicating the classification result is generated. The coefficient memory 32 supplies a coefficient set corresponding to the class number classified by the class classification unit 30 to the adaptive processing unit 31. Adaptive processing unit 31
Performs a process of obtaining a predicted pixel value from the image signal of the prediction tap supplied from the blocking unit 28 using the coefficient set supplied from the coefficient set memory 32.

The correcting section 26 performs so-called image forming processing required for visually enhancing an image such as edge enhancement on the image signal processed by the prediction processing section 25.

Then, the color space conversion section 27
Signal (RG) that has been subjected to processing such as edge enhancement
B signal) is converted into an image signal of a predetermined signal format such as YUV (a signal composed of luminance Y and color differences U and V) by matrix conversion. However, the RGB signals may be directly output from the color space conversion unit 27 without performing the matrix conversion processing. In one embodiment of the present invention, it is possible to switch which of a YUV signal and an RGB signal to output, for example, by a user operation. The image signal converted by the color space conversion unit 27 is supplied to the interface 14 described above.

Here, the image signal processing performed by the image signal processing unit 8 in step S3 of the flowchart shown in FIG. 2 will be described with reference to the flowchart in FIG.

That is, in the image signal processing section 8, the A / D
When the image signal processing for the image signal digitized by the conversion unit 7 is started, first, in step S11, the defect correction unit 21 performs defect correction of the image signal so that the defect of the CCD image sensor 5 is not affected. .
Then, in the next step S12, the clamp unit 2 performs a clamp process for returning the amount shifted in the positive direction to the original value for the image signal subjected to the defect correction by the defect correction unit 21.
Perform by 2.

In the next step S13, the clamping unit 22
The white balance adjustment unit 23 adjusts the white balance of the image signal clamped by the above to adjust the gain between the respective color signals. Further, step S14
Then, the gamma correction unit 24 performs correction according to the gamma curve on the image signal whose white balance has been adjusted.

In step S15, a prediction process using the classification adaptive process is performed. This processing is performed in step S1
It consists of steps from 51 to S155. In step S151, the image signal gamma-corrected by the gamma correction unit 24 is blocked by the blocking unit 28, that is, class taps and prediction taps are cut out. Here, the class tap includes pixels corresponding to a plurality of types of color signals. In step S152, AD
ADRC processing is performed by the RC processing unit 29.

In step S153, the classifying unit 30
Performs a class classification process for classifying the classes based on the ADRC process. Then, a class number corresponding to the classified class is given to the adaptive processing unit 31.

In step S154, the adaptive processing unit 3
1 reads a prediction coefficient set corresponding to the class number given from the class classification unit 30 from the coefficient memory 32,
The prediction pixel value is calculated by multiplying the image signal of the corresponding prediction tap by the prediction coefficient set and calculating the sum thereof.

In step S155, it is determined whether or not processing has been performed on all the areas. If it is determined that the processing has been performed on all the areas, step S
The process proceeds to step S151, otherwise the process proceeds to step S151 to perform processing for the next area.

In step S16, the image corresponding to the CCD output of the three-panel camera obtained in step S15 is subjected to a correction process (so-called image creation) to make the image look good. In step S17, the image obtained in step S16 is subjected to a color space conversion process such as converting RGB signals into YUV signals. This allows
For example, an output image having a signal format suitable as a recording signal is generated.

Here, the classification adaptive processing will be described. FIG. 5 shows a general configuration example for performing a prediction operation using the classification adaptive processing. The input image signal is supplied to the region cutout units 101 and 102. The area cutout unit 101 extracts a predetermined image area (referred to as a class tap) from an input image signal, and converts a signal of the class tap into an AD signal.
It is supplied to the RC processing unit 103. ADRC processing unit 103
Performs ADRC processing on the supplied signal,
Generate a requantization code. Note that a method other than ADRC may be used as a method for generating the requantized code.

The ADRC is originally designed for VTR (Video Tape Re
This is an adaptive requantization method developed for high-efficiency coding for corders, but has the feature that local patterns of signal levels can be efficiently expressed in short tones. Therefore, it can be used to detect a pattern in the spatiotemporal of an image signal, that is, a spatial activity, for generating a code for class classification. The ADRC processing unit 103 equally divides the area between the maximum value MAX and the minimum value MIN in the area cut out as the class tap by the designated number of bits and requantizes according to the following equation (1).

DR = MAX−MIN + 1 Q = [(L−MIN + 0.5) × 2n / DR] (1) Here, DR is a dynamic range in the area. Further, n is the number of allocated bits, and can be, for example, n = 2. L is the signal level of the pixel in the area, and Q is the requantization code. However, square brackets ([...]) mean a process of truncating decimal places.

As a result, an image signal of a class tap consisting of, for example, 8 bits per pixel is converted into, for example, a 2-bit requantized code value. With the requantized code value generated in this way, the pattern of the level distribution in the signal of the class tap is represented by a smaller amount of information. For example, when a class tap structure including seven pixels is used, seven requantization codes q1 to q7 corresponding to each pixel are generated by the above-described processing. The class code class is as shown in the following expression (2).

[0054]

(Equation 1)

Here, n is the number of pixels cut out as class taps. As the value of p, for example, p
= 2.

The class code class represents a class obtained by classifying the pattern of the level distribution of the image signal in the space and time, that is, the spatial activity as feature information. The class code class is supplied to the prediction coefficient memory 104. The prediction coefficient memory 104 stores a prediction coefficient set for each class determined in advance as described later, and outputs a prediction coefficient set of a class represented by the supplied requantization code. On the other hand, the region cutout unit 102 extracts a predetermined image region (referred to as a prediction tap) from the input image and supplies an image signal of the prediction tap to the prediction calculation unit 105. The prediction calculation unit 105 generates an output image signal y by performing a calculation such as the following Expression (3) based on the output of the region cutout unit 102 and the prediction coefficient set supplied from the prediction coefficient memory 104. I do.

Y = w ₁ × x ₁ + w ₂ × x ₂ + ... + w _n × x _n (3) where x ₁ ,..., X _n are pixel values of each prediction tap, and w _1,..., it is w _n are each prediction coefficient.

The processing for determining the prediction coefficient set will be described with reference to FIG. An HD (High Definition) image signal having the same image signal format as the output image signal is supplied to the HD-SD conversion unit 201 and the pixel extraction unit 208. The HD-SD conversion unit 201 converts the HD image signal into an image signal having the same resolution (number of pixels) as the input image signal (hereinafter referred to as SD (Standard De-
finition) image signal). This SD image signal is supplied to the region cutout units 202 and 203. The region cutout unit 202 cuts out a class tap from the SD image signal and supplies the image signal of the class tap to the ADRC processing unit 204, similarly to the region cutout unit 101.

The ADRC processing unit 204 is the ADR processing unit shown in FIG.
An ADRC process similar to that of the C processing unit 103 is performed to generate a requantized code based on the supplied signal. The requantized code is supplied to the class code generation unit 205. The class code generation unit 205 generates a class code indicating a class corresponding to the supplied requantization code, and supplies the class code to the normal equation addition unit 206. On the other hand, the region cutout unit 203 cuts out prediction taps from the supplied SD image signal and supplies the cutout prediction tap image signal to the normal equation addition unit 206, similarly to the region cutout unit 102 in FIG.

The normal equation adding unit 206 converts the image signal supplied from the area extracting unit 203 and the image signal supplied from the pixel extracting unit 208 into a class code generating unit 2.
The addition is performed for each class code supplied from step 05, and the added signal for each class is supplied to the prediction coefficient determination unit 207.
The prediction coefficient determination unit 207 determines a prediction coefficient set for each class based on the supplied signals for each class.

Further, the operation for determining the prediction coefficient set will be described. The prediction coefficient w _i is calculated as follows by supplying a plurality of types of image signals as HD image signals to the configuration shown in FIG. When the number of types of these image signals is expressed as m, the following equation (4) is set from the equation (3).

Y _k = w ₁ × x _k1 + w ₂ × x _k2 + ... + w _n × x _kn (4) (k = 1, 2,..., M) If m> n, w _{Since 1} ,..., W _n are not uniquely determined, the element ek of the error vector e is calculated by the following equation (5).
The prediction coefficient set is determined so as to minimize the square of the error vector e defined by the equation (6). That is, the prediction coefficient set is uniquely determined by the so-called least square method.

[0063] _{e k = y k - {w} 1 × x k1 + w 2 × k 2 + ··· + w n × k n} (5) (k = 1,2, ··· m)

[0064]

(Equation 2)

[0065] A practical calculation method for obtaining the prediction coefficient set that minimizes the e ² of formula (6), predicts e ² coefficients _{w i (i = 1,2, ···} ) partially differentiated by Then, each prediction coefficient w _i may be determined so that the partial differential value of each value of i becomes 0.

[0066]

(Equation 3)

A specific procedure for determining each prediction coefficient w _i from equation (7) will be described. When X _ji and Y _i are defined as in equations (8) and (9), equation (7) can be written in the form of a determinant of equation (10).

[0068]

(Equation 4)

[0069]

(Equation 5)

[0070]

(Equation 6)

Equation (10) is generally called a normal equation. The normal equation adding unit 205 performs an operation as shown in Expressions (8) and (9) based on the supplied signal, thereby _obtaining X _ji , Y _i (i = 1, 2,..., N). Is calculated respectively. The prediction coefficient determination unit 207 solves the normal equation (10) according to a general matrix solution such as a sweeping-out method, so that the prediction coefficient w _i (i = 1, 2,...)
Is calculated.

Here, using the prediction coefficient set and the prediction tap corresponding to the feature information of the pixel of interest, the above equation (3) is used.
By performing the operation of the linear linear combination model in,
Perform adaptive processing. The prediction coefficient set corresponding to the feature information of the target pixel used in the adaptive processing is obtained by learning, but the adaptive processing can be performed by using a pixel value corresponding to a class or by calculating a nonlinear model.

Various kinds of image signal conversion for generating, for example, an image signal from which noise has been removed, an image signal obtained by converting a scanning line structure, and the like as an output image signal from an input image signal by the above-described classification adaptive processing. Processing is realized.

In the digital still camera 1, for example, an input image signal having a color component representing one of a plurality of color components for each pixel position generated by a CCD image sensor of a single-chip camera is converted from an input image signal. For each pixel of interest, a plurality of pixels in the vicinity of the pixel of interest are extracted, a class is determined based on the color component of the highest density pixel among the extracted pixels, and a class is determined based on the determined class. Then, the prediction processing unit 25 performs a class classification adaptive process for generating a pixel having a color component different from the color component of the target pixel at the position of the target pixel, thereby obtaining an image equivalent to the CCD output of the three-chip camera. Get the signal.

That is, there are various types of color filter arrays constituting the color coding filter 4 that can be used in the CCD image sensor 5 of the digital still camera 1 in the CCD image sensor 5, but among the color filter arrays, If there is a difference in the information density of each signal value, if the classification adaptive processing is performed independently for each color signal, the difference in the information density causes a difference in prediction accuracy. For example, when a Bayer array color filter array is used, if the classification adaptive processing is performed independently for each of the R, G, and B color signals, m × n (m and n are positive integers) Regarding the R pixel and the B pixel among the pixels, the same processing as that of G (existing at a ratio of two per four pixels) is performed although there is only one per four pixels.
For this reason, it is not possible to perform a more accurate prediction process for R and B than for G.

Therefore, in the present invention, when there is a difference in information density, only one of a plurality of colors is used for each pixel position by using only the color components arranged at the highest density. Is extracted for each pixel of interest of the input image signal having a color component that represents a pixel having a color component having the highest density among the plurality of color components and located near the pixel of interest. Prediction accuracy can be improved by generating a pixel having a color component different from the color component of the target pixel based on a class determined based on a plurality of pixels.

A specific example of class taps and prediction taps for an image signal color-coded by a Bayer array color filter array in this case will be described. For example, as shown in FIG. 7, when a B pixel is set as a predicted pixel position,
As shown in FIG. 8, four G pixels adjacent to the B pixel located at the predicted pixel position in the vertical and horizontal directions, two G pixels adjacent to the upper left and lower left of the G pixel adjacent to the left side, and a predicted pixel A total of eight G pixels including two G pixels adjacent to the upper right and lower right of the G pixel adjacent to the right side of the B pixel at the position are set as class taps. As the prediction tap in this case, as shown in FIG. 9, pixels including 5 × 5 RGB components centered on the central B pixel are used.

When the R pixel is located at the predicted pixel position as shown in FIG. 10, as shown in FIG. 11, four G pixels adjacent to the R pixel at the predicted pixel position in the upper, lower, left and right directions.
Pixel, G pixel adjacent to the left and 2 adjacent to the upper left and lower left
A total of eight G pixels, that is, a total of eight G pixels, that is, two G pixels adjacent to the upper right and lower right of the G pixel adjacent to the right side, are set as class taps. As the prediction tap in this case,
As shown in FIG. 12, 25 pixels including 5 × 5 RGB color components centering on the R pixel in the prediction pixel are used.

Further, as shown in FIG. 13, when the G pixel is located at the predicted pixel position, the pixel as shown in FIG. Used as a class tap. That is, FIG.
4A shows four G taps adjacent to the upper left, lower left, upper right, and lower right of the G pixel at the predicted pixel position.
Pixel, two G pixels adjacent to each other via a R pixel in the vertical direction from the G pixel at the predicted pixel position, two G pixels adjacent to each other via a B pixel in the horizontal direction, and a total including itself It is composed of nine G pixels. FIG.
4B shows four G taps adjacent to the upper left, lower left, upper right, and lower right of the G pixel at the predicted pixel position.
Pixel, two G pixels vertically adjacent to each other via a B pixel from the G pixel at the predicted pixel position, two G pixels horizontally adjacent to each other via an R pixel, and a total including itself It is composed of nine G pixels. Further, as the prediction tap in this case, (A) in FIG.
(B) includes a pixel at a predicted pixel position,
The surrounding pixels including 5 × 5 RGB color components are used.

From each of the RGB signal values, the luminance signal Y is calculated according to the following equation.

Y = 0.59G + 0.30R + 0.11B As described above, the influence on the luminance signal is greatest for the G component, and therefore, as shown in FIG. It is arranged at high density. The luminance signal Y includes a large amount of information that affects human visual characteristics and resolution.

Therefore, if the class tap is composed of only G pixels for the G component image signal, the class can be determined with higher accuracy, and the class classification adaptive processing can be performed with higher accuracy.

The prediction coefficient set is obtained in advance by learning, and is stored in the coefficient memory 32.

Here, this learning will be described. FIG.
6 is a block diagram illustrating a configuration of a learning device 40 that obtains a prediction coefficient set by learning.

In the learning apparatus 40, an output image signal to be generated as a result of the classification adaptive processing, that is, an image signal having the same signal format as an image signal equivalent to a CCD output of a three-chip camera is thinned out as a teacher image signal. Part 4
1 and the teacher image blocking unit 45. The thinning unit 41 thins out pixels from the teacher image signal according to the arrangement of each color of the color filter array. The thinning process is performed by applying a filter assuming an optical low-pass filter attached to the CCD image sensor 5. That is, the thinning process is performed assuming an actual optical system. The output of the thinning section 41 is supplied to the student image blocking section 42 as a student image signal.

The student image blocking section 42 includes the thinning section 4
1 is used to extract the class tap and the prediction tap based on the target pixel from the student signal generated in step 1 while associating the prediction pixel with the prediction pixel of the teacher image signal for each block. 43
Is supplied to the arithmetic unit 46. The ADRC processor 43 converts the student image signal supplied from the student image
DRC processing is performed to generate feature information, and the classifying unit 44
To supply. The class classification unit 44 generates a class code from the input feature information and outputs the generated class code to the calculation unit 46.

Here, the teacher image signal is an image signal having a resolution equivalent to the CCD output of the three-chip camera, and the student image signal is an image signal having a resolution equivalent to the CCD output of the single-chip camera, in other words, This is an image signal having a lower resolution than the three-panel camera. In other words, the teacher image signal is an image signal in which one pixel has an R component, a G component, and a B component, that is, three primary color components, and the student image signal has one pixel in the R component, the G component, or the B component. Is an image signal having only one color component.

On the other hand, the teacher image blocking section 45 makes correspondence with the class tap in the student image signal,
The image signal of the prediction pixel is cut out from the teacher image signal, and the cut out prediction image signal is supplied to the arithmetic unit 46. Arithmetic unit 4
Reference numeral 6 denotes a class supplied from the classifying unit 44 while associating the image signal of the prediction tap supplied from the student image blocking unit 42 with the predicted image signal supplied from the teacher image blocking unit 45. According to the numbers, an operation for generating data of a normal equation which is an equation in which the prediction coefficient set is a solution is performed. The normal equation data generated by the arithmetic unit 46 is sequentially read into the learning data memory 47 and stored.

The arithmetic unit 48 executes processing for solving the normal equation using the data of the normal equation stored in the learning data memory 47. Thus, a prediction coefficient set for each class is calculated. The calculated prediction coefficient set is stored in the coefficient memory 49 in association with the class. The content stored in the coefficient memory 49 is loaded into the above-described coefficient memory 32 and used when performing the classification adaptive processing.

Next, the operation of the learning device 40 will be described with reference to the flowchart of FIG.

The digital image signal input to the learning device 40 is an image signal capable of obtaining image quality equivalent to an image captured by a three-panel camera. The image signal (teacher image signal) obtained by the three-plate camera includes three primary color signals of R, G, and B as an image signal of one pixel.
The image signal (student image signal) obtained by the single-panel camera is
The image signal of one pixel includes only one of the three primary color signals of R, G, and B. For example, FIG.
As shown in FIG. 18B, the HD image signal picked up by the three-plate camera is filtered, and as shown in FIG.
The teacher image signal converted into the SD image signal of the size is input to the learning device 40.

In step S31, the input teacher image signal is divided into blocks by the teacher image blocking unit 45, and the input teacher image signal is set to a position corresponding to the pixel set as the target pixel by the student image blocking unit 42 from the input teacher image signal. The pixel value of the located prediction pixel is extracted and output to the calculation unit 46.

In step S32, the thinning section 41 applies a color coding filter 4 used in the CCD image sensor 5 of the single-chip camera to the teacher image signal that provides an image quality equivalent to the image picked up by the three-chip camera. By executing a thinning process for applying a corresponding filter, a CCD of a single-chip camera is used as shown in FIG.
A student image signal corresponding to the image signal output from the image sensor 5 is generated from the teacher image signal, and the generated student image signal is output to the student image blocking unit.

In step S33, the input student image signal is blocked in the student image blocker 42, and class taps and prediction taps are generated for each block based on the target pixel.

In step S34, the ADRC processing section 43
, The ADRC processing is performed on the class tap signal cut out from the student image signal for each color signal.

In step S35, the class classifying section 44 classifies the class based on the result of the ADRC processing in step S33, and outputs a signal indicating a class number corresponding to the class to be classified.

In step S 36, in the calculating section 46, the prediction tap supplied from the student image blocking section 42 and the predicted image supplied from the teacher image blocking section 45 for each class supplied from the class classification section 44. , A process of generating the normal equation of the above equation (10) is executed. The normal equation is stored in the learning data memory 47.

In step S37, it is determined whether or not the processing for all blocks has been completed by the arithmetic section 46. If there is a block that has not been processed yet, the process returns to step S36, and the subsequent processing is repeatedly executed. If it is determined in step S37 that the processing for all blocks has been completed,
Proceed to step S38.

In step S38, the arithmetic unit 48 executes a process of solving the normal equation stored in the learning data memory 47 using, for example, a sweeping-out method (Gauss-Jordan elimination method) or a Cholesky decomposition method. Calculate a prediction coefficient set. The prediction coefficient set thus calculated is associated with the class code output by the class classification unit 44 and stored in the coefficient memory 49.

In step S39, it is determined whether or not the processing for solving the normal equations for all the classes has been executed in the operation unit 48. If there is any class that has not been executed yet, the flow returns to step S38. The subsequent processing is repeatedly executed.

If it is determined in step S39 that the process of solving the normal equations for all the classes has been completed, the process ends.

The learning device 40 is located near a target pixel of a student image signal having a color component representing one of a plurality of colors for each pixel position, and has the highest density among the plurality of color components. By extracting a plurality of pixels having a certain color component and determining a class based on the extracted plurality of pixels, a prediction coefficient set for performing the above-described adaptive processing can be obtained.

The prediction coefficient set stored in the coefficient memory 49 in association with the class code in this manner is stored in the coefficient memory 32 of the image signal processing section 8 shown in FIG. Then, as described above, the adaptive processing unit 31 of the image signal processing unit 8 uses the prediction coefficient set stored in the coefficient memory 32 and performs the linear pixel combination model shown in Expression (3) to obtain the target pixel. Perform adaptive processing on. As described above, based on the target pixel in the input image, a class tap and a prediction tap are extracted, a class code is generated from the extracted class tap, and a prediction coefficient set corresponding to the generated class code is generated. Since all the color signals of RGB are generated at the position of the pixel of interest using the prediction tap, it is possible to obtain an image with high resolution. The prediction accuracy can be improved by using only the pixels of the color components arranged at the highest density as the class taps.

In order to evaluate the effects of the above embodiment,
Assuming that a color filter array having a Bayer array is used, the ITE (Institute of Television Enginee
A simulation was performed using nine high-definition standard images of rs) and calculating the prediction coefficient set. The image signal equivalent to the CCD output of the single-chip camera is generated from the image signal equivalent to the CCD output of the three-chip camera by a thinning operation in consideration of the magnification of the class classification adaptive processing and the positional relationship of the pixels. Generates a set of prediction coefficients using an algorithm that performs processing,
The output of the CCD image sensor of the single-panel camera is subjected to prediction processing by the above-described class classification adaptive processing and converted into an image signal having twice the number of pixels in each of the vertical and horizontal directions. The sharpness of edges and details has been improved and a higher resolution image has been obtained than when the class tap for predicting the pixel in ()) was a pixel mixed with RGB. Instead of the classification adaptive processing,
I also simulated the linear interpolation process,
The better the resolution and the S / N ratio were obtained by classifying.

As an evaluation result of the S / N ratio by the above simulation, the S / N ratio of each color signal generated by the class adaptive processing for the standard images A, B, and C is a class for independently extracting each pixel of RGB as a class tap. In the adaptive processing, standard image A R: 35.22 db G: 35.48 db B: 34.93 db standard image B R: 32.45 db G: 32.40 db B: 29.29 db In standard image C, R: 24.75 db G: 25.31 db B: 23.23 db In the class adaptation processing using G pixels as class taps, the standard image AR: 35.38 db G: 35.48 db B: 35.13 db Standard image B R: 32.60 db G: 32.40 db B: 29.46 db Standard image C R: 24.99 db G: 25.31 db B: 23 .79 db was obtained.

As described above, when there is a difference in the information density, the prediction accuracy can be improved by using only the pixels of the color components arranged at the highest density as the class taps. .

In the above description, the case where the Bayer array is used as the color coding filter 4 has been described. However, a color coding filter having a configuration having a difference in information density even with another configuration is used. In this case, the present invention can be applied.

Here, C of this digital still camera 1
FIG. 19 shows a configuration example of a color filter array constituting the color coding filter 4 that can be used for the CD image sensor 5.

(A) to (E) of FIG. 19 show the primary colors (R,
5 shows an example of a color arrangement of green (G), red (R), and blue (B) in a color coding filter 4 configured by a primary color filter array that passes a (G, B) component. 19A shows a Bayer array, FIG. 19B shows an interline array, and FIG. 19C shows a G stripe RB.
FIG. 19D shows a G stripe RB complete checkerboard arrangement, and FIG. 19E shows a primary color difference arrangement.

As described above, in the digital still camera 1 to which the present invention is applied, each color component is calculated based on the class determined based on the color component of the pixel with the highest density. An image signal can be generated with high accuracy. In addition, by the class classification adaptive processing, each color signal R, G,
Since B is obtained, the sharpness of edges and details is increased, and S /
The evaluation value of the N ratio is also improved. That is, the image signal equivalent to the CCD output of the three-chip camera obtained from the image signal equivalent to the CCD output of the single-chip camera by the class classification adaptive processing is:
A clear image is obtained as compared with the case of the conventional linear interpolation processing.

In the above description, ADRC is used as a class reduction method of an image signal.
(Discrete Cosine Transform), VQ (Vector Quantization), DPCM (Differential Pulse Code Modulatio)
n), BTC (Block Trancation Coding), nonlinear quantization, or the like may be used.

Further, compared with the total number of pixels of the CCD image sensor, the number of RGB pixels is, for example, twice each in the vertical and horizontal directions, that is, a total of 4 pixels.
The present invention can also be applied to the case where image signals having different resolutions such as double resolution are generated.
That is, an image signal to be generated is used as a teacher image signal, and a prediction coefficient set generated by learning using an output image signal from the CCD image sensor 5 mounted on the digital still camera 1 as a student image signal is used. In this case, the classification adaptive processing may be performed.

The present invention can be applied not only to a digital still camera but also to video equipment such as a camera-integrated VTR, an image processing apparatus used for broadcasting, for example, and further to a printer or scanner. be able to.

Further, as shown in FIG. 20, for example, the CPU (Central) connected to the bus 311 performs the class classification adaptive processing in the prediction processing unit 25 and the learning processing for obtaining the prediction coefficient set in the learning device 40. Process
ing Unit) 312, a memory 313, an input interface 314, a user interface 315, an output interface 316, and the like. The computer program for performing the above processing is a computer-controllable program that performs image signal processing for processing an input image signal having a color component representing any one of a plurality of pixel positions for each pixel position, which is recorded on a recording medium. Is provided to the user as a recording medium on which is recorded a computer-controllable program for performing a learning process for generating a prediction coefficient set corresponding to a class. The recording medium includes an information recording medium such as a magnetic disk and a CD-ROM, and a transmission medium via a network such as the Internet and a digital satellite.

[0115]

As described above, according to the present invention, for each pixel of interest of an input image signal having a color component representing one of a plurality of colors at each pixel position, A plurality of pixels having the highest-density color component and located near the target pixel are extracted, a class is determined based on the extracted pixels, and a prediction coefficient set and a prediction tap corresponding to the class are determined. Is used to generate all the RGB color signals at the position of the pixel of interest.
A high-resolution image signal can be obtained.

Further, according to the present invention, each pixel position is located in the vicinity of a target pixel of a student image signal having a color component representing one of a plurality of colors, and is the highest among a plurality of color components. A plurality of pixels having a color component having a density are extracted, a class is determined based on the plurality of extracted pixels, and an image signal corresponding to the student image signal has a plurality of color components for each pixel position. From the teacher image signal, a plurality of pixels in the vicinity of a position corresponding to the position of the pixel of interest of the student image signal are extracted. Based on the pixel values of the extracted pixels, the student image signal Since the prediction coefficient set used for the prediction operation for generating the image signal corresponding to the teacher image signal is generated from the image signal corresponding to the image signal, the image signal processing apparatus which performs the processing for obtaining the high-resolution image signal is used. prediction It is possible to calculate the number set.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a digital still camera to which the present invention has been applied.

FIG. 2 is a flowchart for explaining the operation of the digital still camera.

FIG. 3 is a block diagram showing a configuration of an image signal processing unit in the digital still camera.

FIG. 4 is a flowchart for explaining image signal processing performed by the image signal processing unit.

FIG. 5 is a block diagram illustrating a configuration example for performing a prediction operation using a class classification adaptive process.

FIG. 6 is a block diagram illustrating a configuration example for determining a prediction coefficient set.

FIG. 7 is a diagram illustrating a prediction pixel.

FIG. 8 is a diagram illustrating a class tap.

FIG. 9 is a diagram illustrating a prediction tap.

FIG. 10 is a diagram illustrating a prediction pixel.

FIG. 11 is a diagram illustrating a class tap.

FIG. 12 is a diagram illustrating a prediction tap.

FIG. 13 is a diagram illustrating a prediction pixel.

FIG. 14 is a diagram illustrating a class tap.

FIG. 15 is a diagram illustrating a prediction tap.

FIG. 16 is a block diagram illustrating a configuration example of a learning device that obtains a prediction coefficient set by learning.

FIG. 17 is a flowchart for explaining the operation of the learning device.

FIG. 18 is a diagram schematically illustrating an example of a learning process by the learning device.

FIG. 19 is a diagram schematically illustrating a configuration example of a color filter array of a color coding filter that can be used for a CCD image sensor of the digital still camera.

FIG. 20 is a block diagram showing a general configuration of a computer system that performs the above-described classification adaptive processing and learning processing for obtaining a prediction coefficient set.

FIG. 21 is a diagram schematically showing conventional image signal processing by linear interpolation.

[Explanation of symbols]

Reference Signs List 1 digital still camera, 5 CCD image sensor, 8 image signal processing unit, 25 prediction processing unit, 28 blocking unit, 29 ADRC processing unit, 30 class classification unit, 31 adaptive processing unit, 32 coefficient set memory, 40
Learning device, 41 thinning unit, 42 student image blocking unit, 43 ADRC processing unit, 44 class classification unit, 45
Teacher image blocking unit, 46 operation unit, 47 learning data memory, 48 operation unit, 49 coefficient memory

Claims (15)

[Claims] 1. An image signal processing apparatus for processing an input image signal having a color component representing any one of a plurality of colors for each pixel position, comprising: Extracting means for extracting a plurality of pixels having a color component having the highest density among the color components and located in the vicinity of the pixel of interest; and determining a class based on the plurality of pixels extracted by the extracting means. Image signal processing comprising: a class determination unit; and a pixel generation unit configured to generate, based on the class determined in the class determination step, at least a pixel having a color component different from a color component of the pixel of interest. apparatus. 2. The method according to claim 1, wherein the pixel generation unit stores a prediction coefficient set for each class, a prediction coefficient set corresponding to the class determined by the class determination unit, and the target extracted by the extraction unit. 2. The image signal processing apparatus according to claim 1, further comprising: an arithmetic unit configured to generate a pixel having the different color component by performing an operation based on a plurality of pixels near the pixel. 3. The image signal processing apparatus according to claim 1, wherein said pixel generation means generates a pixel having all color components at the position of said target pixel. 4. An image signal processing method for processing an input image signal having a color component representing any one of a plurality of colors for each pixel position, wherein the plurality of pixels include: An extraction step of extracting a plurality of pixels having a color component having the highest density among the color components and located near the pixel of interest, based on the plurality of pixels extracted in the extraction step,
A class determining step of determining a class; and a pixel generating step of generating a pixel having a color component different from at least the color component of the pixel of interest based on the class determined in the class determining step. Image signal processing method. 5. In the pixel generation step, a calculation is performed based on a prediction coefficient set corresponding to the class determined in the class determination step and a plurality of pixels near the target pixel extracted in the extraction step. 5. The image signal processing method according to claim 4, wherein pixels having different color components are generated. 6. The image signal processing method according to claim 4, wherein in the pixel generating step, a pixel having all color components is generated at the position of the target pixel. 7. A recording medium storing a computer-controllable program for performing image signal processing for processing an input image signal having a color component representing one of a plurality of colors for each pixel position, An extracting step of extracting, for each pixel of interest of the input image signal, a plurality of pixels having a color component having the highest density among the plurality of color components and located near the pixel of interest; Based on the multiple pixels extracted in the step,
A class determining step of determining a class; and a pixel generating step of generating, based on the class determined in the class determining step, a pixel having a color component different from at least the color component of the target pixel. Recording medium. 8. The pixel generation step includes performing a calculation based on a prediction coefficient set corresponding to the class determined in the class determination step and a plurality of pixels near the target pixel extracted in the extraction step. 8. The recording medium according to claim 7, wherein pixels having the different color components are generated. 9. The recording medium according to claim 7, wherein in the pixel generation step, pixels having all color components are generated at the position of the target pixel. 10. A color component which is located near a target pixel of a student image signal having a color component representing any one of a plurality of colors for each pixel position and has the highest density among the plurality of color components. First pixel extracting means for extracting a plurality of pixels having the same, class determining means for determining a class based on the plurality of pixels extracted by the first pixel extracting means, and an image corresponding to the student image signal A second pixel extracting means for extracting a plurality of pixels near a position corresponding to a position of a target pixel of the student image signal from a teacher image signal which is a signal and has a plurality of color components for each pixel position; An image signal corresponding to the teacher image signal is generated from an image signal corresponding to the student image signal for each of the classes based on the pixel values of the plurality of pixels extracted by the first and second pixel extraction means. Used for prediction calculation A prediction coefficient generating means for generating a prediction coefficient set. 11. The method according to claim 11, wherein the class determining unit performs ADRC on the plurality of pixels extracted by the first pixel extracting unit.
The learning device according to claim 10, wherein characteristic information is generated by (Adaptive Dynamic Range Coding) processing, and a class is determined based on the characteristic information. 12. A color component which is located in the vicinity of a pixel of interest of a student image signal having a color component representing one of a plurality of colors for each pixel position and has the highest density among the plurality of color components. A first pixel extracting step of extracting a plurality of pixels included in the image, a class determining step of determining a class based on the plurality of pixels extracted in the first pixel extracting step, and an image corresponding to the student image signal. A second pixel extraction step of extracting a plurality of pixels near a position corresponding to a position of a target pixel of the student image signal from a teacher image signal which is a signal and has a plurality of color components for each pixel position; An image signal corresponding to the teacher image signal is generated from an image signal corresponding to the student image signal for each of the classes based on pixel values of a plurality of pixels extracted by the first and second pixel extraction means. For A prediction coefficient generation step of generating a prediction coefficient set used for the prediction calculation of (i). 13. In the class determination step, characteristic information is generated by ADRC (Adaptive Dynamic Range Coding) processing for a plurality of pixels extracted in the first pixel extraction step, and the class is determined based on the characteristic information. The learning method according to claim 12, wherein is determined. 14. A recording medium on which a computer-controllable program for performing a learning process for generating a prediction coefficient set according to a class is recorded, wherein the program comprises one of a plurality of colors for each pixel position. A first pixel extracting step of extracting a plurality of pixels located near the target pixel of the student image signal having one color component and having a color component having the highest density among the plurality of color components; A class determination step of determining a class based on a plurality of pixels extracted in one pixel extraction step; a teacher image signal having a plurality of color components for each pixel position, the image signal corresponding to the student image signal A second pixel extraction step of extracting a plurality of pixels near a position corresponding to the position of the pixel of interest in the student image signal, and extracting the pixels by the first and second pixel extraction means. Based on the pixel values of the plurality of pixels, a prediction coefficient set used for a prediction operation for generating an image signal corresponding to the teacher image signal from an image signal corresponding to the student image signal is generated for each class. A prediction coefficient generating step of performing the following. 15. In the class determining step, characteristic information is generated by ADRC (Adaptive Dynamic Range Coding) processing for a plurality of pixels extracted in the first pixel extracting step, and the class is determined based on the characteristic information. 15. The recording medium according to claim 14, wherein is determined.