JP4581199B2 - Image signal processing apparatus, image signal processing method, learning apparatus, learning method, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image signal processing device, an image signal processing method, a learning device, a learning method, and a recording medium, and in particular, for an image obtained by one solid-state image sensor, one pixel of the image signal is red (R: Red). An image signal processing device, an image signal processing method, a learning device, a learning method, and a color component interpolation method using a class classification adaptive process so as to have a component, a green (G) component, and a blue (B) component The present invention relates to a recording medium.
[0002]
[Prior art]
An imaging apparatus using a solid-state image sensor such as a CCD (Charge Coupled Device) image sensor mainly includes a single plate type using a single CCD image sensor (hereinafter referred to as a single plate camera) and three CCDs. There is a three-plate type using an image sensor (hereinafter referred to as a three-plate camera).
[0003]
In the three-plate camera, for example, three CCD image sensors for R signal, G signal, and B signal are used, and three primary color signals are obtained by the three CCD image sensors. A color image signal generated from the three primary color signals is recorded on a recording medium.
[0004]
In a single-plate camera, a single CCD image sensor having a color coding filter composed of a color filter array assigned to each pixel is installed on the front surface, and 1 color component signal color-coded by the color coding filter is received. Get for each pixel. Examples of color filter arrays constituting the color coding filter include primary color filter arrays of R (Red), G (Green), and B (Blue), Ye (Yellow), Cy (Cyanogen), and Mg (Magenta). A complementary color filter array is used. In a single-plate camera, a CCD image sensor obtains one color component signal for each pixel, and generates a color signal other than the color component signal possessed by each pixel by linear interpolation processing. An image close to that obtained by a plate camera was obtained. In a video camera or the like, a single plate type is adopted to reduce the size and weight.
[0005]
In a single-plate camera, for example, a CCD image sensor provided with a color coding filter constituted by a color filter array having a color arrangement as shown in FIG. 22A is one of the three primary colors R, G, and B. From each pixel in which filters of one color are arranged, only an image signal corresponding to the color of the filter is output. That is, the R component image signal is output from the pixel in which the R color filter is arranged, but the G component and B component image signals are not output. Similarly, only the G component image signal is output from the G pixel, the R component and B component image signals are not output, and only the B component image signal is output from the B pixel. And the image signal of G component is not output.
[0006]
Here, the color arrangement of the color filter array shown in FIG. 22A is called a Bayer arrangement. In this case, G color filters are arranged in a checkered pattern, and R and B are alternately arranged in each row in the remaining portion.
[0007]
However, when the signal of each pixel is processed in the subsequent stage, R component, G component, and B component image signals are required for each pixel. Therefore, conventionally, as shown in FIG. 22B, an image of n × m R pixels is obtained from the output of a CCD image sensor composed of n × m pixels (n and m are positive integers). Signals, n × m G pixel image signals, and n × m B pixel image signals, that is, image signals corresponding to the CCD output of a three-plate camera, are obtained by interpolation, respectively. Output to the subsequent stage.
[0008]
Further, for example, when generating a quadruple density image signal, as shown in FIG. 22C, 2n × 2m R pixel image signals are interpolated from n × m R pixel image signals. 2n × 2m G pixel image signals are obtained by interpolation from the n × m G pixel image signals, and 2n × 2m from the n × m B pixel image signals. Image signals of B pixels are obtained by interpolation calculation.
[0009]
[Problems to be solved by the invention]
In a single-plate camera that obtains one color component signal for each pixel by a CCD image sensor and generates color signals other than the color component signals possessed by each pixel by linear interpolation processing, There is a problem that the linear interpolation process is likely to fail at the thin line portion.
[0010]
Moreover, since the color signal is interpolated by performing linear processing, the waveform of the image becomes dull and the entire image becomes unclear, so processing such as edge enhancement processing is performed to reduce the apparent resolution. The process to raise was necessary.
[0011]
Furthermore, as a processing method different from linear interpolation, it is equivalent to the CCD output of a three-plate camera by performing the class classification adaptive processing independently for each image signal of the three primary colors of R, G, B from the CCD output of the single-plate camera. It has been proposed to generate an image signal (International Publication Number: WO96 / 07275). However, the class classification adaptive processing also basically performs class classification based on the waveform, so that there is a tendency that the continuity of the waveform is broken at an oblique line or a thin line part of the image. This is a problem caused by the arrangement of color filters such as a Bayer arrangement, and the current situation is that it cannot be dealt with by class classification based on normal waveforms.
[0012]
The present invention has been made in view of such a situation, and an image signal that can reproduce a clear image with high resolution and fine details is obtained by class classification adaptive processing corresponding to a portion of an oblique line or a thin line of an image. The object is to obtain it from the CCD output of a plate camera.
[0013]
[Means for Solving the Problems]
The present invention relates to an image signal processing apparatus that processes an input image signal having, as pixel data, a luminance or a color component representing one of a plurality of colors for each pixel position. A solid-state imaging device for obtaining the input image signal; the above Obtained with a solid-state image sensor For each target pixel of the input image signal, a first extraction unit that extracts a plurality of pixels near the target pixel as a first pixel, and each pixel data of the plurality of pixels extracted by the first extraction unit First feature information generation means for generating first feature information based on the second extraction means for extracting a plurality of pixels near the target pixel as second pixels for each target pixel of the input image signal And the vicinity of the target pixel based on the pixel data of the second pixel. The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel Second feature information generating means for generating the second feature information, class determining means for determining one class based on the first feature information and the second feature information, Storage means storing a prediction coefficient set for each class, and stored in the storage means Class determined by the class determination means To generate a pixel having a color component different from the color component of the target pixel from a linear sum obtained by multiplying a prediction coefficient set corresponding to the pixel value and a plurality of pixel values in the vicinity of the target pixel extracted by the pixel extraction unit With means Pixel generating means Ru .
[0014]
The present invention also relates to an image signal processing method for processing an input image signal having, as pixel data, a color component representing any one of luminance or a plurality of colors for each pixel position. An acquisition step of acquiring the input image signal; the above Acquired in the acquisition step A first extraction step of extracting, as a first pixel, a plurality of pixels in the vicinity of the target pixel for each target pixel of the input image signal; A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step; the above Acquired in the acquisition step For each target pixel of the input image signal, a second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a second pixel and each pixel data of the second pixel The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel A second feature information generating step for generating the second feature information, a class determining step for determining one class based on the first feature information and the second feature information, Prediction coefficient sets for each class are stored in the storage means in which they are stored Class determined in the above class determination step A pixel having a color component different from the color component of the target pixel by performing a calculation of the prediction coefficient set according to the pixel value and a plurality of pixel values in the vicinity of the target pixel extracted by the pixel extraction unit And a pixel generation step for generating Ru .
[0015]
Further, the present invention records a computer-controllable program for performing image signal processing for processing an input image signal having, as pixel data, luminance or one of a plurality of colors for each pixel position. In the recording medium, the program is An acquisition step of acquiring the input image signal; the above Acquired in the acquisition step A first extraction step of extracting, as a first pixel, a plurality of pixels in the vicinity of the target pixel for each target pixel of the input image signal; A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step; the above Acquired in the acquisition step For each target pixel of the input image signal, a second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a second pixel and each pixel data of the second pixel The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel A second feature information generating step for generating the second feature information, a class determining step for determining one class based on the first feature information and the second feature information, Prediction coefficient sets for each class are stored in the storage means in which they are stored Class determined in the above class determination step A pixel having a color component different from the color component of the target pixel by performing a calculation of the prediction coefficient set according to the pixel value and a plurality of pixel values in the vicinity of the target pixel extracted by the pixel extraction unit And a pixel generation step for generating Ru .
[0016]
Further, the learning device according to the present invention provides a plurality of pixels in the vicinity of a target pixel of a student image signal having, as image data, any one of luminance or a plurality of colors for each pixel position as a first pixel. First extracting means for extracting the first feature information based on the pixel data of the plurality of pixels extracted by the first extracting means, and the student image For each target pixel of the signal, a second extraction unit that extracts a plurality of pixels near the target pixel as a second pixel, and the vicinity of the target pixel based on each pixel data of the second pixel The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel Corresponding to the student image signal, second feature information generating means for generating the first feature information, class determining means for determining one class based on the first feature information and the second feature information, and the student image signal From a teacher image signal having luminance or one color component of a plurality of colors as image data for each pixel position, in the vicinity of a position corresponding to the position of the target pixel of the student image signal. The student image for each class based on third pixel means for extracting a plurality of pixels as a third pixel and each pixel data of the plurality of pixels extracted by the first and third extraction means. Prediction coefficient generating means for generating a prediction coefficient set for use in a prediction calculation for generating an image signal corresponding to the teacher image signal from an image signal corresponding to the signal. Ru .
[0017]
In the learning method according to the present invention, the plurality of pixels in the vicinity of the target pixel of the student image signal having either one of luminance or a plurality of colors as image data for each pixel position is the first pixel. A first extraction step for extracting the first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step, and the student image For each target pixel of the signal, a second extraction step of extracting a plurality of pixels near the target pixel as a second pixel, and each pixel data of the second pixel, The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel Corresponding to the student image signal, a second feature information generating step for generating the first feature information, a class determining step for determining one class based on the first feature information and the second feature information, and the student image signal From a teacher image signal having luminance or one color component of a plurality of colors as image data for each pixel position, in the vicinity of a position corresponding to the position of the target pixel of the student image signal. The student image for each class based on a third extraction step of extracting a plurality of pixels as a third pixel and the pixel data of the plurality of pixels extracted in the first and third extraction steps. A prediction coefficient generation step of generating a prediction coefficient set used for a prediction calculation for generating an image signal corresponding to the teacher image signal from an image signal corresponding to the signal. Ru .
[0018]
Furthermore, the present invention relates to a recording medium on which a computer-controllable program for performing a learning process for generating a prediction coefficient set according to a class is recorded, wherein the program is a luminance or a plurality of colors for each pixel position. A first extraction step of extracting a plurality of pixels in the vicinity of a target pixel of a student image signal having any one of the color components as image data as a first pixel, and a plurality of pixels extracted in the first extraction step A first feature information generating step for generating first feature information based on each pixel data of the pixel, and a plurality of pixels in the vicinity of the target pixel as second pixels for each target pixel of the student image signal Based on the second extraction step to extract and each pixel data of the second pixel, The presence / absence of a diagonal line pixel and a thin line pixel is determined, and when there is at least one of the diagonal line pixel and the thin line pixel, the type of the pixel Corresponding to the student image signal, a second feature information generating step for generating the first feature information, a class determining step for determining one class based on the first feature information and the second feature information, and the student image signal From a teacher image signal having luminance or one color component of a plurality of colors as image data for each pixel position, in the vicinity of a position corresponding to the position of the target pixel of the student image signal. The student image for each class based on a third extraction step of extracting a plurality of pixels as a third pixel and the pixel data of the plurality of pixels extracted in the first and third extraction steps. A prediction coefficient generation step of generating a prediction coefficient set used for a prediction calculation for generating an image signal corresponding to the teacher image signal from an image signal corresponding to the signal. Ru .
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0020]
The present invention is applied to, for example, a digital still camera 1 configured as shown in FIG. The digital still camera 1 is a single-plate camera that performs color imaging using a single CCD image sensor 5 in which a color coding filter 4 composed of a color filter such as a Bayer array assigned to each pixel is installed on the front surface. The incident light from the subject is collected by the lens 2 and is incident on the CCD image sensor 5 through the iris 3 and the color coding filter 4. On the imaging surface of the CCD image sensor 5, a subject image is formed by incident light having a predetermined level of light quantity by the iris 3. In the digital still camera 1, the color coding filter 4 and the CCD image sensor 5 are separated from each other, but may be integrated.
[0021]
The CCD image sensor 5 performs exposure for a predetermined time in accordance with an electronic shutter controlled by a timing signal from the timing generator 9, and a signal charge (analog amount) corresponding to the amount of incident light transmitted through the color coding filter 4. Is generated for each pixel, the subject image formed by the incident light is imaged, and an image signal obtained as the imaging output is supplied to the signal adjustment unit 6.
[0022]
The signal adjustment unit 6 includes an AGC (Automatic Gain Control) circuit that adjusts the gain so that the signal level of the image signal is constant, and a CDS (Correiated Double Sampling) that removes 1 / f noise generated by the CCD image sensor 5. ) Circuit.
[0023]
The image signal output from the signal adjustment unit 6 is converted from an analog signal to a digital signal by the A / D conversion unit 7 and supplied to the image signal processing unit 8. In the A / D converter 7, for example, a digital imaging signal of 10 bits per sample is generated according to the timing signal from the timing generator 9.
[0024]
In the digital still camera 1, a timing generator 9 supplies various timing signals to a CCD image sensor 5, a signal adjustment unit 6, an A / D conversion unit 7, and a CPU (Central Processing Unit) 10. The CPU 10 controls the iris 3 by driving the motor 11. Further, the CPU 10 drives the motor 12 to move the lens 2 and the like, and controls zoom and autofocus. Further, the CPU 10 performs control to emit a flash by the flash 13 as necessary.
[0025]
The image signal processing unit 8 performs a defect correction process, a digital clamp process, a white balance adjustment process, a gamma correction process, a prediction process using a class classification adaptive process, and the like on the image signal supplied from the A / D conversion unit 7. Process.
[0026]
The memory 15 connected to the image signal processing unit 8 is composed of, for example, a RAM (Random Access Memory), and stores signals necessary when the image signal processing unit 8 performs image processing. The image signal processed by the image signal processing unit 8 is stored in the memory 16 via the interface 14. The image signal stored in the memory 16 is recorded on a recording medium 17 that can be attached to and detached from the digital still camera 1 via the interface 14.
[0027]
The motor 11 drives the iris 3 based on the control information from the CPU 10 and controls the amount of light incident through the lens 2. The motor 12 controls the focus state of the lens 2 with respect to the CCD image sensor 2 based on control information from the CPU 10. Thereby, an automatic aperture control operation and an automatic focus control operation are realized. The flash 13 irradiates the subject with a predetermined flash under the control of the CPU 10.
[0028]
Further, the interface 14 stores the image signal from the image signal processing unit 8 in the memory 16 as necessary, executes a predetermined interface process, and then supplies and stores the image signal in the recording medium 17. As the recording medium 17, a recording medium detachable from the main body of the digital still camera 1, for example, a disk recording medium such as a floppy disk or a hard disk, a flash memory such as a memory card, or the like can be used.
[0029]
Under the control of the CPU 10, the controller 18 supplies control information to the image signal processing unit 8 and the interface 14 to control them. Operation information by the user is input to the CPU 10 from an operation unit 20 including operation buttons such as a shutter button and a zoom button. The CPU 10 controls each unit described above based on the input operation information. The power supply unit 19 includes a battery 19A and a DC / DC converter 19B. The DC / DC converter 19B converts the electric power from the battery 19A into a DC voltage having a predetermined value and supplies it to each component in the apparatus. The rechargeable battery 19 </ b> A can be attached to and detached from the main body of the digital still camera 1.
[0030]
Next, the operation of the digital still camera 1 shown in FIG. 1 will be described with reference to the flowchart of FIG. In step S1, the digital still camera 1 starts imaging of a subject when the power is turned on. That is, the CPU 10 drives the motor 11 and the motor 12 to focus and adjust the iris 3 to form a subject image on the CCD image sensor 5 through the lens 2.
[0031]
In step S2, an image signal obtained by imaging the formed image with the CCD image sensor 5 is gain-adjusted in the signal adjustment unit 6 so that the signal level becomes constant, noise is further removed, and further, A / D Digitized by the converter 7.
[0032]
In step S3, image signal processing including class classification adaptation processing is performed by the image signal processing unit 8 on the image signal digitized by the A / D conversion unit 7.
[0033]
Here, the subject image can be confirmed by the user by displaying an image signal obtained as an imaging output of the CCD image sensor 5 on the electronic viewfinder. It should be noted that the subject image can be confirmed by the user using an optical viewfinder.
[0034]
Then, when the user wants to record the image of the subject image confirmed by the viewfinder on the recording medium 17, the user operates the shutter button of the operation unit 20. In step S4, the CPU 10 of the digital still camera 1 determines whether or not the shutter button has been operated. The digital still camera 1 repeats the processes in steps S2 to S3 until it determines that the shutter button has been operated, and proceeds to step S5 if it determines that the shutter button has been operated.
[0035]
In step S5, the image signal subjected to the image signal processing by the image signal processing unit 8 is recorded on the recording medium 17 via the interface 14.
[0036]
Next, the image signal processing unit 8 will be described with reference to FIG.
[0037]
The image signal processing unit 8 includes a defect correction unit 21 to which the image signal digitized by the A / D conversion unit 7 is supplied. Among the pixels of the CCD image sensor 5, a pixel that does not react to incident light for some reason, a pixel that does not depend on incident light, and in which charge is always stored, in other words, a defective pixel is detected. According to the detection result, a process of correcting the image signal is performed so that the influence of the defective pixels is not exposed.
[0038]
In the A / D conversion unit 7, in order to prevent the negative value from being cut, the A / D conversion is generally performed with the signal value slightly shifted in the positive direction. The clamp unit 22 clamps the image signal that has been defect-corrected by the defect correction unit 21 so that the shift amount described above is eliminated.
[0039]
The image signal clamped by the clamp unit 22 is supplied to the white balance adjustment unit 23. The white balance adjustment unit 23 adjusts the white balance by correcting the gain of the image signal supplied from the clamp unit 22. The image signal whose white balance has been adjusted by the white balance adjustment unit 23 is supplied to the class classification adaptation process 24.
[0040]
In the class classification adaptive processing 24, local image feature amounts are extracted by ADRC (Adaptive Dynamic Range Coding) processing or the like from the image signal whose white balance has been adjusted by the white balance adjustment unit 23, and the class based on each feature. And process for each class. As specific processing, various processing such as conversion from a single-plate image to an image corresponding to three plates, conversion to an arbitrary number of pixels, or simultaneous processing thereof is possible. Here, a special class is prepared for diagonal lines and thin lines to deal with the failure of processing in the diagonal lines and thin lines. The image signal subjected to the class classification adaptation process by the class classification adaptation process 24 is supplied to the gamma correction unit 25.
[0041]
The gamma correction unit 25 corrects the signal level of the image signal subjected to the class classification adaptation process according to the gamma curve. The image signal subjected to gamma correction by the gamma correction unit 25 is supplied to the correction unit 26.
[0042]
The correction unit 26 performs processing for so-called image creation necessary for visually displaying an image such as edge enhancement on the image signal that has been gamma corrected by the gamma correction unit 25.
[0043]
The color space conversion unit 27 performs matrix conversion on the image signal (RGB signal) that has been subjected to processing such as edge enhancement by the correction unit 26 and performs predetermined conversion such as YUV (a signal composed of luminance Y and color differences U and V). To an image signal of the signal format. However, the RGB signal may be directly output from the color space conversion unit 27 without performing the matrix conversion process. In one embodiment of the present invention, for example, a YUV signal or an RGB signal can be switched by a user operation.
The image signal converted by the color space conversion unit 27 is supplied to the interface 14 described above.
[0044]
Here, the image signal processing performed by the image signal processing unit 8 in step S3 of the flowchart shown in FIG. 2 will be described with reference to the flowchart of FIG.
[0045]
That is, when the image signal processing unit 8 starts the image signal processing on the image signal digitized by the A / D conversion unit 7, first, in step S11, the influence of the defect of the CCD image sensor 5 does not occur. The defect correction unit 21 performs defect correction of the image signal. In the next step S <b> 12, the clamping unit 22 performs a clamping process for returning the image signal corrected by the defect correcting unit 21 to the original amount shifted in the positive direction.
[0046]
In the next step S13, the white balance adjustment unit 23 adjusts the white balance of the image signal clamped by the clamp unit 22 to adjust the gain between the color signals. Furthermore, in step S14, class classification adaptation processing including conversion of a single-plate image to a three-plate equivalent image is performed by the class classification adaptation processing unit 24 on the image signal whose white balance has been adjusted.
[0047]
In step S15, the gamma correction unit 25 performs correction according to the gamma curve on the image signal corresponding to the CCD output of the three-plate camera obtained in step S15 subjected to the class classification adaptation process.
[0048]
In step S16, a correction process (so-called image creation) is performed on the image signal that has been gamma corrected in step S15 so that the image signal looks better. In step S17, the image obtained in step S16 is subjected to color space conversion processing such as converting RGB signals into YUV signals. Thereby, for example, an output image having a signal format suitable as a recording signal is generated.
[0049]
As shown in FIG. 5, the class classification adaptive processing unit 24 in the image signal processing unit 8 includes a diagonal line / thin line extraction circuit 31, a class tap extraction circuit 32, an ADRC processing circuit 33, a class classification circuit 34, a coefficient memory 35, and a prediction. It consists of a tap extraction circuit 36 and an adaptive processing circuit 37.
[0050]
The oblique line / thin line extraction circuit 31 functions as second extraction means for extracting the signal value of the pixel at the steep edge portion of the input image signal as second feature information, and filters the region of interest. Is used to determine the presence or absence of a waveform that is difficult to interpolate, such as an oblique line or a thin line, and if such a waveform exists, the type is passed to the class classification circuit 34 as second feature information.
[0051]
The class tap extraction circuit 32 extracts the pixel value at the specified tap position from the region of interest and passes it to the ADRC processing circuit 33. The ADRC processing circuit 33 performs ADRC processing on the pixel value of the class tap passed from the class tap extraction circuit 32, and classifies the result of ADRC processing in which a code of several bits is given to each pixel as first feature information. The data is passed to the classification circuit 34.
[0052]
The class classification circuit 34 includes the ADRC processing result of the class tap received from the ADRC processing circuit 33, that is, the first feature information, and the pixels of the steep edge portions such as diagonal lines and thin lines received from the diagonal line / thin line extraction circuit 31. The class is determined from the second feature information shown, and the class number is output to the coefficient memory 35. The coefficient memory 35 reads the prediction coefficient set corresponding to the class number received from the class classification circuit 34 and passes it to the adaptive processing circuit 37. Further, the prediction tap extraction circuit 36 extracts the pixel value at the tap position designated from the region of interest and passes it to the adaptive processing circuit 37.
[0053]
Then, the adaptive processing circuit 37 multiplies the pixel value of the prediction tap passed from the prediction tap extraction circuit 36 by a prediction coefficient set corresponding to the class number read from the coefficient memory 35, and outputs a linear sum as a prediction pixel value. To do.
[0054]
That is, the class classification adaptation processing unit 24 performs the class classification adaptation process according to the procedure shown in the flowchart of FIG.
[0055]
In step S <b> 21, for the image signal whose white balance has been adjusted by the white balance adjustment unit 23, a block process for extracting the pixel values of the class tap and the prediction tap is performed by the class tap extraction circuit 32 and the prediction tap extraction circuit 36.
[0056]
In the next step S22, the diagonal line / thin line extraction circuit 31 determines whether or not there is a diagonal line or thin line by processing such as a filter for the target pixel, and classifies the result as second feature information. It is passed to the classification circuit 34 and reflected in the class classification.
[0057]
In the next step S23, ADRC processing is performed on the pixel value of the class tap passed from the class tap extraction circuit 32, and a code of several bits is given to each tap.
[0058]
In the next step S 24, the class classification circuit 34 determines the class from the ADRC processing result by the ADRC processing circuit 33, that is, the first feature information and the second feature information from the oblique line / thin line extraction circuit 31. Print the number.
[0059]
In the next step S25, the pixel value of the prediction tap extracted by the prediction tap extraction circuit 36 is multiplied by the prediction coefficient set corresponding to the class number by the adaptive processing circuit 37, and their linear sum is used as the prediction pixel value.
[0060]
In the next step S26, it is determined whether or not the processing for all the blocks has been completed. If there is a block that has not been processed yet, the process returns to step S21, and the subsequent processing is repeatedly executed. If it is determined in step S26 that the processing for all blocks has been completed, the class classification adaptive processing is terminated, and the process proceeds to step S16 described above.
[0061]
Here, the prediction coefficient set used for the class classification adaptation process is obtained in advance by learning and is stored in the coefficient memory 35.
[0062]
Next, this learning will be described. FIG. 7 is a block diagram illustrating a configuration of a learning device 40 that obtains a prediction coefficient set by learning.
[0063]
In this learning device 40, an output image signal to be generated as a result of the class classification adaptive processing, that is, a high-resolution teacher image signal having the same signal format as an image signal equivalent to a CCD output of a three-plate camera, This is supplied to the prediction target pixel extraction circuit 47. The thinning circuit 41 thins out pixels from the teacher image signal according to the arrangement of each color in the color filter array. The thinning process is performed by applying a filter assuming an optical low-pass filter attached to the CCD image sensor 5. That is, thinning processing is performed assuming an actual optical system. The output of the thinning circuit 41 is supplied as a student image signal to the diagonal line / thin line extraction circuit 42, the class tap extraction circuit 43, and the prediction tap extraction circuit 46. Note that the thinning circuit 41 can be omitted by preparing the teacher image signal and the student image signal separately.
[0064]
The diagonal line / thin line extraction circuit 42 determines the presence / absence of diagonal lines, thin lines, etc., using a filter or the like with respect to the local attention area in the student image signal, and uses the determination result as second feature information to class classification circuit. Pass to 45.
[0065]
The class tap extraction circuit 43 extracts class taps used for class classification from the student image signals generated by the thinning circuit 41 and passes them to the ADRC processing circuit 44.
[0066]
The ADRC processing circuit 44 performs ADRC processing on the pixel value of the class tap passed from the class tap extraction circuit 43, and classifies the ADRC processing result that reflects the undulation of the waveform at the tap position in the class as first feature information. It passes to the classification circuit 45.
[0067]
The class classification circuit 45 includes the ADRC processing result of the class tap received from the ADRC processing circuit 44, that is, the first feature information, and the pixel of the steep edge portion such as the diagonal line or the thin line received from the diagonal line / thin line extraction circuit 42. The class is determined from the second feature information shown, and the class number is output to the first arithmetic circuit 48.
[0068]
The prediction tap extraction circuit 46 extracts the prediction tap from the student image signal generated by the thinning circuit 41 while taking correspondence with the class tap in the student image, and outputs it to the first arithmetic circuit 48. Here, it is assumed that prediction taps are extracted from all color signals.
[0069]
The prediction target pixel extraction circuit 47 extracts the pixel value to be predicted from the teacher image signal and outputs it to the first arithmetic circuit 48 while taking correspondence with the class tap and the prediction tap extracted from the student image. .
[0070]
For each class number output from the class classification circuit 45, the first arithmetic circuit 48 adds the pixel value of the prediction tap and the pixel value of the prediction target pixel to the normal equation for solving the least squares method, and the prediction coefficient The coefficient of the matrix of the normal equation that is an equation with the set as a solution is calculated. The coefficients of the normal equation matrix generated by the first arithmetic circuit 48 are sequentially read and accumulated in the learning data memory 49.
[0071]
The second arithmetic circuit 50 uses the coefficients of the normal equation matrix stored in the learning data memory 49 to execute processing for solving the normal equation by a technique such as Cholesky decomposition. Thereby, a prediction coefficient set for each class is calculated. The calculated prediction coefficient set is stored in the coefficient memory 51 in association with the class. The stored contents of the coefficient memory 51 are loaded into the coefficient memory 35 described above and used when performing the class classification adaptation process.
[0072]
Next, the operation of the learning device 40 will be described with reference to the flowchart of FIG.
[0073]
The digital image signal input to the learning device 40 is an image signal that provides an image quality equivalent to an image captured by a three-plate camera. The image signal (teacher image signal) obtained with the three-plate camera includes the three primary color signals of R, G, and B as one pixel image signal, whereas the image signal (student) obtained with the single-plate camera. The image signal) includes only one of the three primary color signals of R, G, and B as an image signal of one pixel. For example, a teacher image signal obtained by filtering an HD image signal captured by a three-plate camera as shown in FIG. 9A and converting it into a 1/4 size SD image signal as shown in FIG. Are input to the learning device 40.
[0074]
In step S31, a filter corresponding to the color coding filter 4 used for the CCD image sensor 5 of the single-plate camera is applied by the thinning circuit 41 to the teacher image signal from which the image quality corresponding to the image captured by the three-plate camera is obtained. By executing the thinning process, as shown in FIG. 9C, a student image signal corresponding to the image signal output from the CCD image sensor 5 of the single-plate camera is generated from the teacher image signal, and the generated student image signal Is supplied to the class tap extraction circuit 43 and the prediction tap extraction circuit 46.
[0075]
In step S32, the class tap extraction circuit 43 and the prediction tap extraction circuit 46 perform block processing for extracting the pixel values of the class tap and the prediction tap from the student image signal generated by the thinning circuit 41. Further, the prediction target pixel extraction circuit 47 extracts the pixel value of the prediction target pixel from the teacher image signal while taking correspondence with the class tap and the prediction tap extracted from the student image.
[0076]
In step S33, the diagonal line / thin line extraction circuit 42 determines the presence / absence of diagonal lines, fine lines, etc. by applying a filter to the local attention area of the student image signal, and uses the determination result as second feature information. The data is passed to the class classification circuit 45.
[0077]
In step S34, the ADRC processing circuit 44 performs ADRC processing on the pixel value of the class tap passed from the class tap extraction circuit 43.
[0078]
In step S35, the class is determined by the class classification circuit 45 based on the result of ADRC processing of the class tap by the ADRC processing circuit 44, and the class number is output.
[0079]
In step S36, the first arithmetic circuit 48 adds the pixel value of the prediction tap and the pixel value of the prediction target pixel to the normal equation according to the class number output from the class classification circuit 45.
[0080]
In step S37, it is determined whether or not the process of adding to the normal equation by the first arithmetic circuit 48 has been performed on all the learning target pixels. If there is a target pixel that has not been processed yet, the process returns to step S32, and if all the target pixels have been processed, the process proceeds to step S38.
[0081]
In step S <b> 38, the second arithmetic circuit 50 uses the coefficients of the normal equation matrix stored in the learning data memory 49 to execute processing for solving the normal equation by a technique such as Cholesky decomposition.
[0082]
In step S39, it is determined whether or not the process of solving the normal equation by the second arithmetic circuit 50 has been performed for the normal equations of all class numbers. If there is a normal equation that has not been processed yet, the process returns to step S38. If all the processes have been performed, the learning process is terminated.
[0083]
The prediction coefficient set associated with the class code and stored in the coefficient memory 51 in this way is stored in the coefficient memory 35 of the class classification adaptive processing unit 24 shown in FIG. Then, as described above, the class classification adaptive processing unit 24 of the image signal processing unit 8 uses the prediction coefficient set stored in the coefficient memory 35 and performs an adaptive process on the pixel of interest using a linear linear combination model. I do.
[0084]
For example, from an output image signal color-coded by a Bayer color filter array obtained by a CCD image sensor composed of n × m pixels (n and m are positive integers) shown in FIG. FIG. 10B shows an adaptive process for directly generating an image signal of 2n × 2m R pixels, an image signal of 2n × 2m G pixels, and an image signal of 2n × 2m B pixels, respectively. By doing so, a quadruple density image is generated. In this case, in the image signal processing unit 8, the class tap extraction unit 30 divides the input image signal into p × q (p and q are positive integers) blocks, and class taps are extracted for each block.
[0085]
In this case, for example, a class tap having a structure as shown in FIGS. 11 to 14 is used. 11A, FIG. 12A, FIG. 13A, and FIG. 14A, the R signal of the prediction pixel indicated by a cross in the position adjacent to the target pixel in the diagonal direction. When generating a G signal or a B signal, and calculating a prediction coefficient set therefor, a structure of an example of a class tap used to determine a class is shown in FIGS. It is shown in (B), (B) in FIG. 13 and (B) in FIG.
11 (B), 12 (B), 13 (B), and 14 (B), Δ indicates a class tap of the B signal, ○ indicates a class tap of the G signal, and □ indicates the class tap of the R signal.
[0086]
The class tap having such a structure is extracted by the class tap extraction unit 30, and the class classification circuit 45 uses the first feature information indicating the result of 1-bit ADRC of the class tap (9 taps) by the ADRC processing circuit 44. The 512 classes are classified into 2 classes according to the second feature information indicating the result of determination of the presence / absence of a steep edge pattern by the oblique line / thin line extraction circuit 42, and are classified into 1024 classes.
[0087]
Here, as a filter used for the diagonal line / thin line extraction circuit 42, for example, a filter having a tap structure as shown in FIGS. 15A to 15F can be used.
[0088]
FIG. 15A shows a tap structure of a filter for detecting a horizontal edge.
In the horizontal edge detection filter, as shown by □, with respect to the pixel of interest indicated by ◎ in (A) of FIG. 15, two G pixels adjacent vertically and 4 on the left and right of these G pixels. A total of 6 G pixels of the G pixels are used as filter taps.
[0089]
FIG. 15B shows a tap structure of a filter for detecting a vertical edge.
In the vertical edge detection filter, with respect to the target pixel indicated by （in FIG. 15B, as shown by □, two G pixels adjacent to the left and right and 4 above and below these G pixels. A total of 6 G pixels of the G pixels are used as filter taps.
[0090]
(C) and (D) of FIG. 15 each show a filter tap structure for detecting an oblique edge. In the diagonal edge detection filter, as shown by □, four G pixels adjacent in the vertical and horizontal directions are used as filter taps for the pixel of interest indicated by ◎ in FIGS. 15C and 15D. .
[0091]
FIG. 15E shows a tap structure of a filter for detecting horizontal fine lines. In the horizontal thin line detection filter, as shown by □, for the target pixel indicated by ◎ in (E) of FIG. 15, two G pixels adjacent to the left and right are six in the horizontal edge detection filter. A total of 8 G pixels added to the G pixels are used as filter taps.
[0092]
FIG. 15F shows a tap structure of a filter for detecting a vertical fine line. In the vertical thin line detection filter, for the target pixel indicated by Ｇ in FIG. 15 (F), as shown by □, two vertical G pixels adjacent to each other in the vertical edge detection filter are used. A total of 8 G pixels added to the G pixels are used as filter taps.
[0093]
Discrimination of steep edges in the oblique line / thin line extraction circuit 42 is performed by adding a determination based on a threshold to the results of these filters.
[0094]
Such a filter can be designed according to an edge or waveform causing a problem, and can also be adjusted by the characteristics of the color filter of the CCD image sensor.
[0095]
Further, the prediction tap extraction unit 36 extracts prediction taps having a structure as shown in FIGS. 16 (A), FIG. 17 (A), FIG. 18 (A), and FIG. FIG. 16B, FIG. 17B, and FIG. 18 show the structure of an example of a prediction tap used when generating a G signal or a B signal and calculating a prediction coefficient set therefor. This is shown in (B) and (B) of FIG. That is, the prediction tap is a 5 × 5 pixel including the pixel of interest as indicated by a circle in FIG. 16B, FIG. 17B, FIG. 18B, and FIG. Consists of 25 pixels.
[0096]
In addition, the position of the class tap and the prediction tap is originally arranged in the most efficient manner. And the accuracy of processing is improved by using many class taps and prediction taps.
[0097]
In order to evaluate the effect of the above embodiment, a simulation was performed assuming that a color filter array having a Bayer array was used.
[0098]
A prediction coefficient set is generated by an algorithm that performs the same processing as that of the learning device 40, and a single-plate type is obtained from an image signal corresponding to a CCD output of a three-plate camera by a thinning operation that takes into account the magnification of the class classification adaptive processing and the positional relationship of pixels. An image signal equivalent to the CCD output of the camera is generated and interpolated by the above-described class classification adaptive processing. As a result of comparison with conventional linear interpolation and class classification combining the waveform classification of RGB ADRC. Thus, the superiority of the present invention could be confirmed.
[0099]
For the simulation, nine high-definition standard images of ITE (Institute of Television Engineers) were used, and the prediction coefficient set was also calculated using the nine images. As a result, in comparison with linear interpolation, the sharpness of oblique lines, edge portions, and details increased in RGB of all images, and SN could be improved.
[0100]
In the above description, the case where a Bayer array filter is used as the color coding filter 4 has been described. However, in the case of using a color coding filter having a different information density even in other configurations. The present invention can be applied.
[0101]
Here, examples of the configuration of the color filter array constituting the color coding filter 4 that can be used in the CCD image sensor 5 of the digital still camera 1 are shown in FIGS.
[0102]
20A to 20G show green (G), red (R), and blue (B) in the color coding filter 4 formed of the primary color filter array that passes the primary color (R, G, B) components. An example of the color arrangement is shown.
[0103]
20A shows a Bayer arrangement, FIG. 20B shows an interline arrangement, FIG. 20C shows a G stripe RB checkered arrangement, and FIG. 20D shows a G stripe RB. FIG. 20E shows a stripe arrangement, FIG. 20F shows an oblique stripe arrangement, and FIG. 20G shows a primary color difference arrangement.
[0104]
20 (H) to (N) are magenta (M) / yellow (Y) in the color coding filter 4 composed of a complementary color filter array that allows the complementary color (M, Y, C, W, G) components to pass through. ), Cyan (C), and white (W). 20H shows the field color difference sequential arrangement, FIG. 20I shows the frame color difference sequential arrangement, FIG. 20J shows the MOS type arrangement, and FIG. 20K shows the improved MOS. FIG. 20 (L) shows a frame interleave arrangement, FIG. 20 (M) shows a field interleave arrangement, and FIG. 20 (N) shows a stripe arrangement.
[0105]
The complementary color (M, Y, C, W, G) components are
Y = G + R
M = R + B
C = G + B
W = R + G + B
Given in Each color (YM, YG, CM, CG) component passing through the color coding filter 4 corresponding to the frame color difference order shown in (I) of FIG.
YM = Y + M = 2R + G + B
CG = C + G = 2G + B
YG = Y + G = R + 2G
CM = C + M = R + G + 2R
Given in
[0106]
In the digital still camera 1, a four-fold density image signal is generated from the image signal captured by the single-plate CCD image sensor 5 by the class classification adaptive processing. The conversion to any number of pixels can be performed by class classification adaptive processing without being limited to the conversion to, and various processes such as performing these processes simultaneously can be performed by the class classification adaptive processing. Can be.
[0107]
As a class classification method for performing the class classification adaptive processing, there is a method such as using the MSB number bits of the pixel value of the class tap in addition to expressing the waveform characteristics by ADRC for the pixel value of the class tap. However, the present invention can employ any basic classification.
[0108]
Furthermore, the class classification adaptation processing in the class classification adaptation processing unit 24 and the learning processing for obtaining a prediction coefficient set in the learning device 40 are performed by a CPU (Central Processing Unit) connected to a bus 311 as shown in FIG. (Unit) 312, memory 313, input interface 314, user interface 315, output interface 316, and the like, can be executed by a general computer system 310. A computer program that executes the above processing is a computer-controllable program that performs image signal processing for processing an input image signal recorded on a recording medium and having a color component representing any one of a plurality of pixel positions. Is provided to the user as a recording medium on which is recorded or a computer-controllable program that performs a learning process for generating a prediction coefficient set according to a class. The recording medium includes an information recording medium such as a magnetic disk and a CD-ROM, and a transmission medium via a network such as the Internet and a digital satellite.
[0109]
【The invention's effect】
As described above, according to the present invention, the present invention provides the target pixel for each target pixel of the input image signal having, as pixel data, a luminance or a color component representing any one of a plurality of colors for each pixel position. A plurality of neighboring pixels are extracted as first pixels, first feature information is generated from each signal value of the extracted plurality of pixels, and for each target pixel of the input image signal, A plurality of pixels are extracted as second pixels, and edge information in the vicinity of the target pixel is generated as second feature information based on each pixel data of the second pixel, and the first feature information and the first feature information One class is determined based on the feature information of 2, and the pixel data as the color component is generated at least at the position of the target pixel based on the class. For the corresponding class The adaptive processing, higher resolution, it is possible to obtain an image signal which can reproduce a clear image to details from the CCD output of a single-plate camera.
[0110]
Further, according to the present invention, a plurality of pixels in the vicinity of a target pixel of a student image signal having luminance or any one of a plurality of colors as image data for each pixel position are extracted as first pixels. Then, first feature information is generated based on the pixel data of the plurality of extracted pixels, and a plurality of pixels near the target pixel are extracted as second pixels for each target pixel of the student image signal. Then, based on each pixel data of the second pixel, edge information in the vicinity of the target pixel is generated as second feature information, and one class is generated based on the first feature information and the second feature information. To decide.
An image signal corresponding to the student image signal, and from a teacher image signal having luminance or any one of a plurality of colors as image data for each pixel position, the pixel of interest of the student image signal A plurality of pixels in the vicinity of the position corresponding to the position is extracted as a third pixel, and for each class based on each pixel data of the plurality of pixels extracted as the first pixel and the third pixel, Since the prediction coefficient set used for the prediction calculation for generating the image signal corresponding to the teacher image signal is generated from the image signal corresponding to the student image signal, the classification classification adaptation corresponding to the diagonal line and the thin line part of the image By the processing, it is possible to calculate a prediction coefficient set used by the image signal processing apparatus that performs processing for obtaining an image signal that can reproduce an image with high resolution and clear details.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a digital still camera to which the present invention is applied.
FIG. 2 is a flowchart for explaining the operation of the digital still camera.
FIG. 3 is a block diagram illustrating a configuration of an image signal processing unit in the digital still camera.
FIG. 4 is a flowchart for explaining image signal processing performed by the image signal processing unit;
FIG. 5 is a block diagram illustrating a configuration example of a class classification adaptation processing unit that performs class classification adaptation processing;
FIG. 6 is a flowchart for explaining class classification adaptation processing performed by the class classification adaptation processing unit.
FIG. 7 is a block diagram illustrating a configuration example of a learning device that obtains, by learning, a prediction coefficient set used for class classification adaptation processing in the class classification adaptation processing unit.
FIG. 8 is a flowchart for explaining the operation of the learning apparatus.
FIG. 9 is a diagram schematically showing an example of learning processing by the learning device.
FIG. 10 is a diagram schematically illustrating an example of image signal processing by class adaptation processing in the image signal processing unit.
FIG. 11 is a diagram schematically showing the structure of a class tap used for the class adaptation process.
FIG. 12 is a diagram schematically showing the structure of a class tap used for the class adaptation process.
FIG. 13 is a diagram schematically showing a structure of a class tap used for the class adaptation processing.
FIG. 14 is a diagram schematically showing the structure of a class tap used for the class adaptation processing.
FIG. 15 is a diagram schematically showing a tap structure of a filter used in the diagonal line / thin line extraction circuit in the class adaptation processing unit.
FIG. 16 is a diagram schematically illustrating the structure of a prediction tap used for the class adaptation process.
FIG. 17 is a diagram schematically illustrating a structure of a prediction tap used for the class adaptation process.
FIG. 18 is a diagram schematically illustrating the structure of a prediction tap used for the class adaptation process.
FIG. 19 is a diagram schematically illustrating the structure of a prediction tap used for the class adaptation process.
FIG. 20 is a diagram schematically illustrating a configuration example of a color filter array of a color coding filter that can be used in the CCD image sensor of the digital still camera.
FIG. 21 is a block diagram showing a general configuration of a computer system that performs the above-described class classification adaptation processing and learning processing for obtaining a prediction coefficient set.
FIG. 22 is a diagram schematically illustrating conventional image signal processing by linear interpolation.
[Explanation of symbols]
1 digital still camera, 5 CCD image sensor, 8 image signal processing unit, 24 class classification adaptive processing unit, 31 diagonal line / thin line extraction circuit, 32 class tap extraction circuit, 33 ADRC processing circuit, 34 class classification circuit, 35 coefficient set Memory, 36 Predictive tap extraction circuit, 37 Adaptive processing circuit, 40 Learning device, 41 Thinning out part, 42 Diagonal line / thin line extraction circuit, 43 Class tap extraction circuit, 44 ADRC processing circuit, 45 class classification circuit, 46 Prediction tap extraction circuit 47 Prediction target pixel extraction circuit, 48 First arithmetic circuit, 49 Learning data memory, 50 Second arithmetic circuit, 51 Coefficient memory

Claims (13)

In an image signal processing apparatus for processing an input image signal having, as pixel data, a luminance or a color component representing any one of a plurality of colors for each pixel position,
A solid-state imaging device for obtaining the input image signal;
First extraction means for extracting, as a first pixel, a plurality of pixels near the target pixel for each target pixel of the input image signal acquired by the solid-state imaging device ;
First feature information generating means for generating first feature information based on pixel data of a plurality of pixels extracted by the first extracting means;
Second extraction means for extracting, as a second pixel, a plurality of pixels near the target pixel for each target pixel of the input image signal;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists A second feature information generating means for generating the type of the pixel as second feature information;
Class determining means for determining one class based on the first feature information and the second feature information;
Storage means for storing a prediction coefficient set for each class, prediction coefficient set corresponding to the class determined by the class determination means stored in the storage means, and the pixel of interest extracted by the pixel extraction means from linear sum obtained by multiplying the plurality of pixel values in the neighborhood, images signal processing apparatus Ru and a pixel generating means and a computing means for generating a pixel having different color components from the color component of the pixel of interest. The first feature information generating means, ADRC (Adaptive Dynamic Range Coding) image signal processing apparatus 請 Motomeko 1 wherein that generates the first feature information by the processing. Color represented by the color components are red, blue, the image signal processing apparatus according to any der Ru請 Motomeko 1 wherein the green. The solid-state imaging device, Ru CCD image sensor der the Bayer array 請 Motomeko first image signal processing method according. In an image signal processing method for processing an input image signal having, as pixel data, a luminance or a color component representing any one of a plurality of colors for each pixel position,
An acquisition step of acquiring the input image signal;
A first extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a first pixel for each target pixel of the input image signal acquired in the acquisition step ;
A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step;
A second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as second pixels for each target pixel of the input image signal acquired in the acquisition step ;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists Includes a second feature information generation step of generating the type of the pixel as second feature information;
A class determining step for determining one class based on the first feature information and the second feature information;
A prediction coefficient set corresponding to the class determined in the class determination step stored in a storage unit storing a prediction coefficient set for each class, and a plurality of pixels in the vicinity of the target pixel extracted by the pixel extraction unit by performing the calculation of the value, images signal processing method Ru and a pixel generating step of generating a pixel having different color components from the color component of the pixel of interest. In the first feature information generating step, ADRC (Adaptive Dynamic Range Coding) image signal processing method 請 Motomeko 5 wherein that generates the first feature information by the processing. Color represented by the color components are red, blue, image signal processing method 請 Motomeko 5 wherein Ru der any green. In a recording medium recorded with a computer-controllable program for performing image signal processing for processing an input image signal having pixel data having luminance or a color component representing one of a plurality of colors for each pixel position,
The above program
An acquisition step of acquiring the input image signal;
A first extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a first pixel for each target pixel of the input image signal acquired in the acquisition step ;
A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step;
A second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as second pixels for each target pixel of the input image signal acquired in the acquisition step ;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists Includes a second feature information generation step of generating the type of the pixel as second feature information;
A class determining step for determining one class based on the first feature information and the second feature information;
A prediction coefficient set corresponding to the class determined in the class determination step stored in a storage unit storing a prediction coefficient set for each class, and a plurality of pixels in the vicinity of the target pixel extracted by the pixel extraction unit by performing the calculation of the value, record medium Ru and a pixel generating step of generating a pixel having different color components from the color component of the pixel of interest. First extraction means for extracting, as a first pixel, a plurality of pixels in the vicinity of a target pixel of a student image signal having either one of a luminance or a plurality of colors as image data for each pixel position;
First feature information generating means for generating first feature information based on pixel data of a plurality of pixels extracted by the first extracting means;
Second extraction means for extracting, as a second pixel, a plurality of pixels near the target pixel for each target pixel of the student image signal;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists A second feature information generating means for generating the type of the pixel as second feature information;
Class determining means for determining one class based on the first feature information and the second feature information;
An image signal corresponding to the student image signal, and from a teacher image signal having, as image data, any one of a luminance or a plurality of colors for each pixel position to the position of the target pixel of the student image signal A third extracting means for extracting a plurality of pixels in the vicinity of the corresponding position as a third pixel;
An image signal corresponding to the teacher image signal is generated from an image signal corresponding to the student image signal for each class based on the pixel data of the plurality of pixels extracted by the first and third extraction means. Ru learning apparatus and a prediction coefficient generating means for generating a prediction coefficient set to be used for prediction calculation for. In the first feature information generating unit, ADRC (Adaptive Dynamic Range Coding) learning apparatus 請 Motomeko 9 wherein that generates the first feature information by the processing. A first extraction step of extracting, as first pixels, a plurality of pixels in the vicinity of a target pixel of a student image signal having luminance or any one of a plurality of colors as image data for each pixel position;
A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step;
A second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a second pixel for each target pixel of the student image signal;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists Includes a second feature information generation step of generating the type of the pixel as second feature information;
A class determining step for determining one class based on the first feature information and the second feature information;
An image signal corresponding to the student image signal, and from a teacher image signal having, as image data, any one of a luminance or a plurality of colors for each pixel position to the position of the target pixel of the student image signal A third extraction step of extracting a plurality of pixels in the vicinity of the corresponding position as a third pixel;
An image signal corresponding to the teacher image signal is generated from the image signal corresponding to the student image signal for each class based on the pixel data of the plurality of pixels extracted in the first and third extraction steps. Manabu習方method Ru and a prediction coefficient generation step of generating a prediction coefficient set to be used for prediction calculation for. In the first feature information generating step, ADRC (Adaptive Dynamic Range Coding) 請 Motomeko 11 learning method according that generates the first feature information by the processing. In a recording medium recorded with a computer-controllable program for performing a learning process for generating a prediction coefficient set according to a class,
The above program
A first extraction step of extracting, as first pixels, a plurality of pixels in the vicinity of a target pixel of a student image signal having luminance or any one of a plurality of colors as image data for each pixel position;
A first feature information generation step for generating first feature information based on the pixel data of the plurality of pixels extracted in the first extraction step;
A second extraction step of extracting a plurality of pixels in the vicinity of the target pixel as a second pixel for each target pixel of the student image signal;
Based on each pixel data of the second pixel, the presence / absence of a diagonal line pixel and a thin line pixel in the vicinity of the target pixel is determined, and when at least one of the diagonal line pixel and the thin line pixel exists Includes a second feature information generation step of generating the type of the pixel as second feature information;
A class determining step for determining one class based on the first feature information and the second feature information;
An image signal corresponding to the student image signal, and from a teacher image signal having, as image data, any one of a luminance or a plurality of colors for each pixel position to the position of the target pixel of the student image signal A third extraction step of extracting a plurality of pixels in the vicinity of the corresponding position as a third pixel;
An image signal corresponding to the teacher image signal is generated from the image signal corresponding to the student image signal for each class based on the pixel data of the plurality of pixels extracted in the first and third extraction steps. record medium Ru and a prediction coefficient generation step of generating a prediction coefficient set to be used for prediction calculation for.

Images