JP2003052051A - Image signal processing method, image signal processor, imaging apparatus and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the S/N ratio of image signals outputted from a CCD image sensor improved to raise the density and the resolution by the fine processing technology. SOLUTION: A/D converter 7 converts input analog image signals to digital signals at a lower resolution than that of output image signals. An image signal processor 8 applies an adaptive classification process to the input image signals converted to the digital signals by the A/D converter 7, thereby conducting a gradation creating process to obtain multi-gradationed output image signals.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing method, an image signal processing device, an imaging device and a recording medium,
In particular, an image signal processing method, an image signal processing device, an imaging device, and a recording device which improve the S / N ratio of an imaging output obtained by an image sensor whose density and resolution have advanced by miniaturization technology Regarding the medium.

[0002]

2. Description of the Related Art Imaging devices using a solid-state image sensor such as a CCD (Charge Coupled Device) image sensor are mainly of a single plate type using one CCD image sensor (hereinafter referred to as a single plate camera). And three CCs
There is a three-plate system using a D image sensor (hereinafter referred to as a three-plate camera).

In a three-plate type camera, for example, three CCD image sensors for R signal, G signal and B signal are used, and three primary color signals are obtained by the three CCD image sensors. Then, the color image signal generated from the three primary color signals is recorded on the recording medium.

In a single-panel camera, a CCD image sensor having a color coding filter consisting of a color filter array assigned to each pixel is installed on the front surface of the CCD, and the color components coded by the color coding filter are used. A signal is obtained for each pixel. The color filter array forming the color coding filter includes, for example, R (Red), G (Green), B (Blue) primary color filter arrays, Ye (Yellow), Cy (Cyanogen), Mg (Ma).
genta) complementary color filter array is used. In the single-chip camera, a CCD image sensor obtains one color component signal for each pixel, and color signals other than the color component signals possessed by each pixel are generated by linear interpolation processing to generate 3 An image similar to that obtained by a plate camera was obtained. In a video camera or the like, a single plate type is used for downsizing and weight reduction.

Further, in an image pickup apparatus in which the image pickup signal is subjected to signal processing by digital processing, C before A / D conversion is performed.
The signal is amplified by applying AGC (Automatic Gain Control) to the CD output, and processed so that the gradation at the time of A / D conversion has a certain set value.

[0006]

By the way, in an image pickup apparatus using an image pickup element represented by a CCD image sensor, an image with much noise can be obtained when the amount of incident light is small at the time of photographing. The case where the amount of incident light is small corresponds to cases such as shooting at night, the subject being dark, and the sensitivity of the image sensor being low. In such a case, the problem of noise is unavoidable. This is because the AGC before A / D conversion tries to increase the narrow dynamic range to secure the gradation, and therefore the noise is amplified together with the original signal value.

Particularly, in the recent CCD image sensor,
The number of pixels has been remarkably improved by the miniaturization technology and the high-density technology, but the size of each pixel is extremely small, and the area that receives the incident light is small. As a result, the amount of light incident on each pixel is also reduced. However, there is a problem that the S / N of the output image is reduced.

Although AGC is processed as an analog signal before A / D conversion, generally, there is no signal processing for performing multi-gradation on a digital signal after A / D conversion of CCD output.

Therefore, an object of the present invention is to improve the S / N of the image pickup output obtained by a CCD image sensor whose density and resolution have been advanced by the miniaturization technique, by signal processing.

[0010]

According to the present invention, a gradation is created after A / D conversion by using a class classification adaptive process.
/ N is improved.

Further, according to the present invention, gradation is created adaptively to the photographing environment to improve S / N.

That is, the image signal processing method according to the present invention comprises an analog / digital conversion step of converting an input image signal from an analog signal into a digital signal at a resolution lower than that of the output image signal, and an input converted into a digital signal. A gradation creation processing step of performing gradation creation processing by adaptive class classification processing on the image signal, and obtaining an output image signal having multiple gradations.

The image signal processing apparatus according to the present invention is
An analog / analog that converts an input image signal from an analog signal to a digital signal with a resolution lower than that of the output image signal
Digital conversion means, and image signal processing means for performing gradation creation processing by adaptive class classification processing on the input image signal converted into a digital signal by the analog / digital conversion means to obtain an output image signal with multiple gradations. It is characterized by including.

Further, the image pickup device according to the present invention includes an image pickup means for picking up an image of a subject and an input image signal obtained as an image pickup signal by the image pickup means at a resolution lower than the resolution of the output image signal from an analog signal to a digital signal. An analog / digital converting means for converting into an analog / digital converting means, and an image for obtaining a multi-gradation output image signal by performing gradation creation processing by adaptive class classification processing on the input image signal converted into the analog / digital converting means digital signal And a signal processing means.

The recording medium according to the present invention further comprises an analog / digital conversion step for converting an input image signal into a digital signal from an analog signal at a resolution lower than that of the output image signal, and the input image signal converted into the digital signal. A gradation creation processing step of performing gradation creation processing by adaptive class classification processing for the above, and performing image signal processing for generating an output image signal having multiple gradations from the input image signal by the gradation creation processing. A computer controllable program characterized by is recorded.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

The present invention is effective for general image input devices such as CCD image sensors. There are several possible targets for processing, as shown in FIG.
It is an image sensor, or three CCDs for R, G and B
In a system having an image sensor, it is only necessary to perform processing for multi-gradation with the same number of pixels. However, when outputting an RGB image from a single-plate CCD image sensor as shown in FIG. 2, it is necessary to perform multi-gradation and perform single-plate to three-plate conversion. Further, as shown in FIG. 3, there may be a case where the resolution is increased, but this can be processed even if the input is a single color CCD image sensor or a single plate CCD image sensor.

In the embodiments described below, a digital still camera equipped with a single-plate CCD having a color filter such as a Bayer array is assumed, and the process of creating gradation together with resolution by the class classification adaptation process is collectively performed. Done in.

The present invention is applied to a digital still camera 1 having a structure as shown in FIG. 4, for example. This digital still camera 1 is a single-panel camera that performs color imaging using one CCD image sensor 5 having a color coding filter 4 composed of color filters assigned to each pixel on the front surface, The incident light of is condensed by the lens 2 and is incident on the CCD image sensor 5 via the iris 3 and the color coding filter 4. A subject image is formed on the image pickup surface of the CCD image sensor 5 by the incident light whose light amount has a predetermined level by the iris 3.

The CCD image sensor 5 performs exposure for a predetermined time according to an electronic shutter controlled by a timing signal from a timing generator 9, and a signal charge (corresponding to the amount of incident light passing through the color coding filter 4 ( By generating an (analog amount) for each pixel, a subject image formed by the incident light is imaged, and an image signal obtained as an imaging output is supplied to the signal adjusting unit 6.

The signal adjusting section 6 adjusts the gain so that the signal level of the image signal becomes constant.
ain control circuit and CDS (Correiated Doub) that removes 1 / f noise generated by the CCD image sensor 5.
le Sampling) circuit.

The image signal output from the signal adjusting section 6 is converted from an analog signal to a digital signal by the A / D converting section 7 and supplied to the image signal processing section 8. The A / D conversion unit 7 generates a digital image pickup signal having an appropriate number of bits smaller than the number of bits of the output image according to the timing signal from the timing generator 9.

In this digital still camera 1, the timing generator 9 includes a CCD image sensor 5,
The signal adjusting unit 6, the A / D converting unit 7, and the CPU (Central Pro
Various timing signals are supplied to the cessing unit) 10.
The CPU 10 controls the iris 3 by driving the motor 11. Further, the CPU 10 drives the motor 12 to move the lens 2 and the like, and controls zooming, autofocusing, and the like. Furthermore, CPU
Reference numeral 10 controls the flash 13 to emit a flash light as needed.

The image signal processing section 8 predicts the image signal supplied from the A / D conversion section 7 using defect correction processing, digital clamp processing, white balance adjustment processing, gamma correction processing, and class classification adaptation processing. Perform processing such as processing.

The memory 15 connected to the image signal processing unit 8 is composed of, for example, a RAM (Random Access Memory), and stores signals necessary for the image signal processing unit 8 to perform image processing. The image signal processed by the image signal processing unit 8 is stored in the memory 16 via the interface 14. The image signal stored in the memory 16 is recorded on the recording medium 17 which is detachable from the digital still camera 1 via the interface 14.

The motor 11 drives the iris 3 based on the control information from the CPU 10 and controls the amount of light incident through the lens 2. Further, the motor 12 is
CCD of lens 2 based on control information from CPU 10
The focus state for the image sensor 5 is controlled.
As a result, the automatic diaphragm control operation and the automatic focus control operation are realized. Further, the flash 13 emits a predetermined flash light to the subject under the control of the CPU 10.

Further, the interface 14 stores the image signal from the image signal processing unit 8 in the memory 16 as required, and after performing a predetermined interface process, supplies the image signal to the recording medium 17 to store it. As the recording medium 17, a recording medium detachable from the main body of the digital still camera 1, for example, a disk recording medium such as a flexible disk or a hard disk, a flash memory such as a memory card, or the like can be used.

Under the control of the CPU 10, the controller 18 supplies control information to the image signal processing section 8 and the interface 14 to control them. The operation information by the user is input to the CPU 10 from the operation unit 20 including operation buttons such as a shutter button and a zoom button. The CPU 10 controls each unit described above based on the input operation information. The power supply unit 19 is a battery 19A.
And a DC / DC converter 19B and the like. DC / D
The C converter 19B converts the electric power from the battery 19A into a DC voltage having a predetermined value and supplies the DC voltage to each component in the device. The rechargeable battery 19A is attachable to and detachable from the main body of the digital still camera 1.

Next, the operation of the digital still camera 1 shown in FIG. 4 will be described with reference to the flowchart of FIG. In step S1, the digital still camera 1 starts imaging a subject when the power is turned on. That is, the CPU 10 drives the motor 11 and the motor 12 to focus and adjust the iris 3 to form a subject image on the CCD image sensor 5 via the lens 2.

In step S2, the formed image is transferred to the CCD.
The image signal picked up by the image sensor 5 is gain-adjusted by the signal adjusting unit 6 so that the signal level is constant, noise is further removed, and further digitized by the A / D converting unit 7. The A / D converter 7 generates a digital image pickup signal having an appropriate number of bits smaller than the number of bits of the output image, thereby avoiding amplification of noise.

Here, the subject image can be confirmed by the user by displaying an image signal obtained as an image pickup output of the CCD image sensor 5 on the electronic viewfinder. The subject image may be made visible to the user by an optical viewfinder.

When the user wants to record the image of the subject image confirmed by the viewfinder on the recording medium 17, the user operates the shutter button of the operation unit 20. In step S3, the CPU 10 of the digital still camera 1 determines whether or not the shutter button has been operated.

The CPU 10 of the digital still camera 1 is
If the determination result in step S3 is NO, that is, if the shutter button is not operated, in step S4, the image signal processing unit 8 performs a view on the image signal digitized by the A / D conversion unit 7. The first image processing for displaying the image of the subject image is performed by the viewfinder, the process returns to step S2, and the steps S2 to S4 are repeated until the shutter button is operated, and the shutter button is operated. If so, the process proceeds to step S5.

In step S5, the image signal digitized by the A / D conversion unit 7 is subjected to the second image signal processing including the class classification adaptive processing by the image signal processing unit 8.

Then, in step S6, the image signal subjected to the second image signal processing by the image signal processing section 8 is recorded on the recording medium 17 via the interface 14.

Next, the image signal processing section 8 will be described with reference to FIG.

The image signal processing section 8 includes a defect correction circuit 21 to which the image signal digitized by the A / D conversion section 7 is supplied. Among the pixels of the CCD image sensor 5, a pixel that does not react to incident light for some reason, a pixel that does not depend on the incident light, and that has a constant charge, that is, a defective pixel is detected. According to the detection result, a process of correcting the image signal is performed so that the influence of these defective pixels is not exposed.

In the A / D converter 7, in order to prevent the negative value from being cut, generally, the A / D conversion is performed with the signal value being slightly shifted in the positive direction. The clamp circuit 22 secures a negative value by performing a clamp process on the image signal corrected for defects by the defect correction circuit 21 so that the above-described shift amount is eliminated.

The image signal clamped by the clamp circuit 22 is supplied to the white balance adjusting circuit 23. The white balance adjustment circuit 23 is the clamp circuit 2
The white balance is adjusted by correcting the gain of the image signal supplied from 2. The image signal whose white balance is adjusted by the white balance adjusting circuit 23 is supplied to the class classification adaptive processing unit 25 or the gamma correction circuit 26 via the switch 24.

Here, the switch 24 is switched in conjunction with the shutter operation, and when the shutter button is not operated, the image signal from the white balance adjusting circuit 23 is supplied to the gamma correction circuit 26, and the shutter button is pressed. When is operated, the above white balance adjustment circuit 2
The image signal from 3 is supplied to the class classification adaptive processing unit 25.

In the class classification adaptive processing section 25, the ADRC
The local image feature amount is extracted by (Adaptive Dynamic Range Coding) process, etc., a class is created based on each feature, and the local process is performed. As specific processing, all processing such as multi-gradation, conversion from a single plate image to a three-plate phase image, conversion to an arbitrary number of pixels, and the like are collectively performed. The image signal subjected to the class classification adaptation processing by the class classification adaptation processing unit 25 is supplied to the gamma correction circuit 26.

The gamma correction circuit 26 is a signal level of the image signal supplied from the white balance adjustment circuit 23 via the switch 24 or the image signal subjected to the image signal class classification adaptation processing in the class classification adaptation processing section 25. Is corrected according to the gamma curve. The image signal gamma-corrected by the gamma correction circuit 26 is corrected by the correction circuit 2.
7 is supplied. Since the gamma correction is a non-linear process, the class classification adaptation processing unit 25 that performs a linear signal process.
It is conceivable that the signal processing will be improved if there is the first one.

The correction circuit 27 is the gamma correction circuit 26.
The image signal processed by is subjected to processing for so-called image formation necessary for visually enhancing the image such as edge enhancement.

Then, the color space conversion circuit 28 performs a matrix conversion of the image signal (RGB signal) which has been subjected to the edge enhancement processing by the correction circuit 27 and YUV (a signal consisting of the luminance Y and the color differences U and V). And the like into an image signal of a predetermined signal format. However, the RGB signals may be directly output from the color space conversion circuit 28 without performing the matrix conversion process. In the embodiment of the present invention, which of the YUV signal and the RGB signal is to be output can be switched by, for example, a user operation.
The image signal converted by the color space conversion circuit 28 is supplied to the interface 14 described above.

Here, the first image signal processing performed by the image signal processing unit 8 in step S4 of the flowchart shown in FIG. 5 will be described with reference to the flowchart of FIG.

That is, in the image signal processing section 8, the A / D
First for the image signal digitized by the converter 7
When the image signal processing is started, first, in step S41, the defect correction circuit 21 corrects the defect of the image signal so that the defect of the CCD image sensor 5 is not affected. Then, in the next step S42, the clamp circuit 22 performs a clamp process for restoring the amount of the image signal corrected in the defect correction circuit 21 in the positive direction to the original amount.

In the next step S43, the clamp circuit 2
With respect to the image signal clamped by 2, the white balance adjustment circuit 23 adjusts the white balance to adjust the gain between the color signals.

In the next step S44, the image signal whose white balance has been adjusted by the white balance adjusting circuit 23 is converted from a single plate output image signal into a three plate phase image signal.

In the next step S45, the gamma correction circuit 26 corrects the converted three-plate phase image signal according to the gamma curve.

Further, in step S46, a correction process is performed on the gamma-corrected image signal so that the image signal looks good. In step S47, the color space conversion circuit 28 performs color space conversion processing on the image signal obtained in step S46, such as converting an RGB signal into a YUV signal. As a result, an image signal for monitoring the subject with the viewfinder is generated.

Next, the second image signal processing performed by the image signal processing unit 8 in step S5 of the flowchart shown in FIG. 5 will be described with reference to the flowchart of FIG.

That is, in the image signal processing unit 8, the A / D
Second for the image signal digitized by the converter 7
When the image signal processing is started, first, in step S51, the defect correction circuit 21 performs defect correction on the image signal so that the defect of the CCD image sensor 5 is not affected. Then, in the next step S52, the clamp circuit 22 performs a clamping process for returning the amount of the image signal corrected in the defect correction circuit 21 in the positive direction to the original amount.

In the next step S53, the clamp circuit 2
With respect to the image signal clamped by 2, the white balance adjustment circuit 23 adjusts the white balance to adjust the gain between the color signals.

In the next step S54, the class classification adaptation processing unit 25 performs processing such as multi-gradation, conversion of a single plate output image signal into a three-plate phase image signal, and high resolution processing by using the class classification adaptation processing. Do it all at once.

In the next step S55, the gamma correction circuit 26 is applied to the image signal subjected to the class classification adaptation processing.
The correction is performed according to the gamma curve.

Further, in step S56, for the image equivalent to the CCD output of the gamma-corrected three-plate type camera,
Correction processing is performed to make the image look better. Step S
In 57, the color space conversion circuit 28 performs color space conversion processing such as conversion of RGB signals into YUV signals on the image signals obtained in step S56. Thereby, for example, an output image signal having a signal format suitable as a recording signal is generated.

Here, in the operation of the digital still camera 1 shown in the flow chart of FIG. 5, the first image signal processing (step S4) for generating the image signal for monitoring the subject with the viewfinder is performed. Although the second image signal processing (step S5) for generating the image signal for recording is performed by the image signal processing unit 8, the operation as shown in the flowchart of FIG. 9 may be performed.

That is, in the operation shown in the flowchart of FIG. 9, the digital still camera 1 starts image pickup of a subject when the power is turned on in step S11.

In step S12, the formed image is CC
The image signal picked up by the D image sensor 5 is gain-adjusted by the signal adjusting unit 6 so that the signal level becomes constant, noise is further removed, and further digitized by the A / D converting unit 7. The A / D converter 7 generates a digital image pickup signal having an appropriate number of bits smaller than the number of bits of the output image, thereby avoiding amplification of noise. This is achieved by reducing the number of bits, that is, reducing the resolution and increasing the quantization step per bit.

At step S13, the A / D converter 7
The image signal processing unit 8 performs first image processing such as defect processing and white balance on the image signal digitized by.

In the first image processing in step S13, the processing of steps S131 to S133 shown in FIG. 10 is performed. That is, in step S131, CC
The defect correction circuit 21 performs defect correction of the image signal so that the defect of the D image sensor 5 is not affected. In the next step S132, the clamp circuit 22 performs a clamp process for returning the amount of the image signal corrected in the defect correction circuit 21 in the positive direction to the original amount. Further, in the next step S133, the white balance adjustment circuit 23 adjusts the white balance of the image signal clamped by the clamp circuit 22 to adjust the gain between the color signals.

Then, in step S14, the CPU 10 of the digital still camera 1 determines whether or not the shutter button of the operation unit 20 has been operated.

The CPU 10 of the digital still camera 1 is
If the determination result in step S14 is NO, that is, if the shutter button is not operated, in step S15, a viewfinder object image is displayed for the image signal subjected to the first image processing. Second
The image signal processing unit 8 performs the image processing of step S12, returns to step S12, and repeats steps S12 to S15 until it is determined that the shutter button is operated.
If it is determined that the shutter button has been operated, step S
Proceed to 16.

In the second image processing in step S15, the processing of steps S151 to S154 shown in FIG. 11 is performed. That is, in step S151, the image signal subjected to the white balance adjustment by the white balance adjustment circuit 23 is converted from the single plate output image signal into the three-plate phase equal image signal.

In the next step S152, the converted 3
The gamma correction circuit 26 corrects the plate phase image signal according to the gamma curve. In the next step S153, the gamma-corrected image signal is subjected to a correction process to make it look good visually. Further, step S15
In step 4, the color space conversion circuit 28 performs color space conversion processing on the image signal obtained in step S153, such as converting an RGB signal into a YUV signal. As a result, an image signal for monitoring the subject with the viewfinder is generated.

In step S16, the image signal processing unit 8 performs the third image signal processing including the class classification adaptive processing on the image signal subjected to the first image processing. Then, in step S17, the image signal subjected to the third image signal processing by the image signal processing unit 8 is recorded on the recording medium 17 via the interface 14.

Here, the third step in step S16 is performed.
In the image processing of step S161 to S1 shown in FIG.
64 processing is performed. That is, in step S161,
Processing such as multi-gradation, conversion from a single-plate output image signal to a three-plate phase image signal, and higher resolution is collectively performed by the class classification adaptation processing unit 25 using the class classification adaptation processing. In the next step S162, the gamma correction circuit 26 corrects the image signal subjected to the class classification adaptive processing according to the gamma curve. Further, in step S163, a correction process is performed on the image corresponding to the CCD output of the gamma-corrected three-plate camera so that the image looks good. In step S164, the color space conversion circuit 28 performs color space conversion processing such as conversion of RGB signals into YUV signals on the image signals obtained in step S163. Thereby, for example, an output image signal having a signal format suitable as a recording signal is generated.

In the flowcharts of FIGS. 5 and 9 described above,
The operation when the image signal for monitoring the subject is generated by the viewfinder on the camera body side has been described. However, when the image signal for monitoring is not generated when the optical viewfinder is used, the operation of FIG. The operation may be performed according to the flowchart shown in.

That is, in the operation shown in the flowchart of FIG. 13, the digital still camera 1 operates in step S21.
In, the imaging of the subject is started by turning on the power.

In the next step S22, the CPU 10 of the digital still camera 1 determines whether or not the shutter button of the operation unit 20 has been operated. The CPU 10 of the digital still camera 1 repeats the determination process of step S22 until it determines that the shutter button is operated,
If it is determined that the shutter button has been operated, step S
Proceed to 23.

In step S23, the formed image is CC
The image signal picked up by the D image sensor 5 is gain-adjusted by the signal adjusting unit 6 so that the signal level becomes constant, noise is further removed, and further digitized by the A / D converting unit 7. The A / D converter 7 generates a digital image pickup signal having an appropriate number of bits smaller than the number of bits of the output image, thereby avoiding amplification of noise.

At step S24, the A / D converter 7
The image signal processed by the image signal processing section 8 is subjected to the image processing (steps S51 to S57) shown in FIG. Then, in step S25, the image signal subjected to the image signal processing by the image signal processing unit 8 is recorded in the recording medium 17 via the interface 14.

Next, the class classification adaptation processing section 25 will be described with reference to FIG.

The class classification adaptation processing section 25 includes a class tap extraction circuit 31, a class classification circuit 32, and a coefficient memory 3.
3, a prediction tap extraction circuit 34, an adaptive processing circuit 35, and the like.

The class tap extraction circuit 31 extracts the pixel value from the tap position designated from the attention area and passes it to the class classification circuit 32. The class classification circuit 32 applies the ADRC to the pixel value extracted by the class tap extraction circuit 31.
The image signal pattern is classified based on the characteristic information by processing such as (Adaptive Dynamic Range Coding), and a class number (class code) indicating the classification result is output. The coefficient memory 33 stores the prediction coefficient set corresponding to the class number classified by the class classification circuit 32 in the adaptive processing circuit 3.
Supply to 5. The prediction tap extraction circuit 34 extracts a pixel value from the tap position designated from the attention area and passes it to the adaptive processing circuit 35. The adaptive processing circuit 35 applies a prediction coefficient corresponding to each pixel value of the prediction tap, performs a process to obtain a predicted pixel value by linear sum, and outputs the obtained predicted pixel value.

Here, with respect to the class classification adaptive processing performed by the class classification adaptive processing unit 25 in step S54 of the flowchart shown in FIG. 8 and step S161 of the flowchart shown in FIG. 12, refer to the flowchart of FIG. Explain.

That is, the class classification adaptive processing section 25 described above.
First, in step S81, the class tap extraction circuit 31 and the prediction tap extraction circuit 34 extract the pixel values of the class tap and the prediction tap.

In the next step S82, a class classification process for classifying the class by the ADRC process is performed on the pixel value extracted as the class tap by the class classification circuit 32. Then, the class number corresponding to the classified class is given to the adaptive processing circuit 35.

In the next step S83, the adaptive processing circuit 35 reads the prediction coefficient set corresponding to the class number given from the class classification circuit 32 from the coefficient memory 33, and the prediction coefficient set is the image signal of the corresponding prediction tap. To calculate the predicted pixel value.

Then, in step S84, it is determined whether or not processing has been performed on the entire screen, and if there is an unprocessed area, the process returns to step S2l. It is determined whether or not the processing has been performed on all the areas. If it is determined that the processing has been performed on all the regions, the process proceeds to step S15 described above, otherwise, the process returns to step S2l to perform the process on the next region.

The prediction coefficient set is obtained by learning in advance and is stored in the coefficient memory 33.

Here, this learning will be described. Figure 1
6 is a block diagram showing a configuration of a learning device 40 that obtains a prediction coefficient set by learning.

In the learning device 40, an output image signal to be generated as a result of the class classification adaptation process, that is, an image signal having the same signal format as the image signal corresponding to the CCD output of the three-plate type camera is thinned out as a teacher image signal. It is supplied to the circuit 41 and the prediction target pixel extraction circuit 45.

The thinning circuit 41 applies a low-pass filter to the teacher image signal in consideration of the processing magnification, and then thins pixels according to the arrangement of each color in the color filter array. The thinning process is performed by applying a filter assuming an optical low-pass filter mounted on the CCD image sensor 5. That is, the thinning process is performed assuming an actual optical system. Next, the number of bits is reduced for the thinned pixel values. In the system that actually performs the processing, if A / D conversion is performed with a precision of 6 bits and the gradation is raised to 8 bits in the subsequent classification adaptive processing, 8
A bit image is used as an input to the learning device 40, and the thinning circuit 41 drops the image to 6 bits. The output of the thinning circuit 41 is supplied to the class tap extraction circuit 42 and the prediction tap extraction circuit 44 as a student image signal.

The class tap extraction circuit 42 extracts a class tap used for class classification from the student image signal created by the thinning circuit 41.

The class classification circuit 43 applies the ADRC (Adapt) to the pixel values extracted by the class tap extraction circuit 42.
The image signal patterns are classified based on the feature information by processing such as ive dynamic range coding, and a class number (class code) indicating the classification result is output.

The predictive tap extracting circuit 44 extracts predictive taps in correspondence with class taps in the student image signal.

The prediction target pixel extraction circuit 45 extracts the pixel value to be predicted from the teacher image signal while keeping correspondence with the class tap and the prediction tap extracted from the student image signal.

The arithmetic circuit 46, for each class number output from the class classification circuit 43, predicts the pixel value of the prediction tap extracted by the prediction tap extraction circuit 44 and the prediction target pixel extracted by the prediction target pixel extraction circuit 45. The pixel value is added to a normal equation for solving the least squares method, and an operation for generating data of a normal equation which is an equation having a prediction coefficient set as a solution is performed. The data of the normal equation generated by the arithmetic circuit 46, that is, the coefficients of the matrix are sequentially read and stored in the learning data memory 47.

The arithmetic circuit 48 executes a process for solving the matrix of normal equations stored in the learning data memory 47 by using a solution method such as Cholesky decomposition. As a result, a prediction coefficient set for each class is calculated. The calculated prediction coefficient set is stored in the coefficient memory 49 in association with the class. The stored contents of the coefficient memory 49 are loaded into the above-mentioned coefficient memory 33 and used when the class classification adaptation processing is performed.

Next, the operation of the learning device 40 will be described with reference to the flowchart of FIG.

The digital image signal input to the learning device 40 is an image signal that provides an image quality equivalent to the image captured by the three-plate type camera. The image signal (teacher image signal) obtained by the three-plate camera includes the three primary color signals of R, G, and B as the image signal of one pixel.
The image signal (student image signal) obtained with a single-chip camera is
An image signal of one pixel includes only one color signal of R, G, and B primary color signals.

In step S31, the thinning-out circuit 41 executes a thinning-out process in which a filter corresponding to the color coding filter 4 used in the CCD image sensor 5 of the single-chip camera is applied, whereby a teacher image signal such as a 3-plate CCD output phase is obtained. A student image signal such as a single-chip CCD output phase is created from the above, and at the same time, the number of bits is reduced.

In the next step S32, the class tap extraction circuit 42 and the prediction tap extraction circuit 44 extract the pixel values of the class tap and the prediction tap from the student image signal obtained by the thinning process. The prediction target pixel extraction circuit 45 extracts the pixel value of the prediction target pixel from the teacher image signal in association with the student image signal. In step S32, the CCD image sensor 5 of the single-chip camera is operated by the thinning circuit 41 for the teacher image signal that provides the image quality equivalent to the image captured by the three-plate camera.
A student image signal corresponding to the image signal output by the CCD image sensor 5 of the single-chip camera is generated from the teacher image signal by executing a thinning process for applying a filter corresponding to the color coding filter 4 used for The student image signal is output to the class tap extraction circuit 42.

In the next step S33, the class value of the class tap extracted by the class tap extraction circuit 42 is subjected to ADRC processing by the class classification circuit 43 and the class number is output.

In the next step S34, the pixel value of the prediction tap and the pixel value of the pixel to be predicted are added to the normal equation by the arithmetic circuit 46 according to the class number output from the class classification circuit 43. The normal equation is stored in the learning data memory 47.

In the next step S35, it is determined by the arithmetic circuit 46 whether or not the process of adding to the normal equation has been performed on all the learning target images. If all additions have been performed, the process proceeds to step S36, and if not completed, the process returns to step S32.

In the next step S36, the normal equation in which the pixel values of all the learning target images are added is used to calculate the arithmetic circuit 48.
Thus, the prediction coefficient set is calculated by executing a solving process using, for example, the sweeping method (Gauss-Jordan elimination method) or the Cholesky decomposition method. The prediction coefficient set calculated in this way is associated with the class code output by the class classification circuit 43 and stored in the coefficient memory 49.

Then, in the next step S37, it is determined whether or not the processing for solving the normal equations for all the classes has been executed in the arithmetic circuit 48, and if there is a class that has not been executed yet, the step is executed. The process returns to S36 and the subsequent processes are repeatedly executed. When it is determined in this step S37 that the process of solving the normal equations for all the classes is completed, the process ends.

The prediction coefficient set stored in the coefficient memory 49 in association with the class code in this way is stored in the coefficient memory 33 of the image signal processing unit 8 shown in FIG.

Next, the class classification method will be described.

In the class classification process, class taps are extracted from the input signal to be processed, the feature quantity existing between the class taps is extracted, and the classes are created by classifying the feature quantities. .

Here, as an example of class classification, a single plate CCD is used.
The case where the resolution is increased to 4 times the density and the gradation is increased from the output will be described.

This is an example until the user gets tired of it. However, FIGS. 18 to 21 show the relationship between the predicted pixel position and the arrangement of class taps for the image signal color-coded by the Bayer array color filter array. The four types have four modes of R, G, and B depending on the predicted pixel position.

When four pixels are formed around B of the single-chip CCD output as shown in FIG. 18A to make the density four times higher, a class tap as shown in FIG. 18B is used. Can be taken (mode 1). In this case, as a method of creating a class, among the class taps with various marks,
For example, 1-bit ADRC is performed between the marks of the same kind,
A value of 0 or 1 is given to each tap. Of the results, only the values of 9 taps surrounded by the dotted rectangle shown in FIG.
It will be possible to create street classes. In this case, the reason why separate ADRC processing is performed for each of R, G, and B is in consideration of incorporating the undulation of the waveform in each signal into the feature amount used for the class. In this class classification method, the tap arrangement in each mode is different, but the class creation method among R, G, and B is exactly the same.

Further, also in FIGS. 19 to 21, the class is classified by the same method.

That is, as shown in FIG. 19A, in the case where four pixels are formed around G adjacent to B in the horizontal row of the single-chip CCD output and the density is quadrupled, as shown in FIG. Take a class tap as shown in B) (mode 2).

As shown in FIG. 20A, in the case where four pixels are formed around G adjacent to R in the horizontal row of the single-plate CCD output to achieve 4 times density, FIG. 20B. It is possible to take a class tap as shown in (Mode 3).

When four pixels are formed around R in the horizontal row of the single-chip CCD output as shown in FIG. 21 (A) and the density is quadrupled, as shown in FIG. 21 (B). You can take a class tap (mode 4).

As an example of the prediction tap, FIG. 22 shows a structure in which 5 × 5 25 pixels are tapped around the prediction pixel position. Also in this case, although there are four modes depending on the predicted pixel position, the tap arrangement among R, G, and B is the same. Further, as a matter of course, the coefficients for obtaining four pixels are different from each other in each mode, and the coefficients are also different in the predictions of R, G, and B, respectively.

There are various possible arrangements of class taps and prediction taps. Also, it has been confirmed that the accuracy of processing can be improved by using a large number of class taps and prediction taps, but in reality, it is limited by the capacity of the arithmetic circuit or the memory storing prediction coefficients. Become.

In order to evaluate the effects of the above embodiments,
Simulations were performed assuming that a Bayer array color filter array was used.

An RGB image signal having a sufficiently high resolution is prepared and used as a teacher image signal for learning. The teacher image signal is thinned or a low-pass filter is applied to reduce the resolution, and the amount of light incident on each pixel of the CCD image sensor is taken into consideration in the AG
Noise is added assuming that the gain is increased by C. With respect to the image signal, the number of bits of each pixel value is reduced to obtain a learning student image signal. After setting the student image signal and the teacher image signal in this way, create the source code of the algorithm that performs the same operation as the learning device 40,
The prediction coefficient was obtained based on the above method. A simulation was actually performed using the prediction coefficient. In this process, three processes of single plate / three plate conversion, high resolution, and multi-gradation are collectively performed. Then, the student image signal in which the number of bits is not reduced by only adding noise to the processing result is compared with the image signal subjected to the class classification adaptive process of only single plate / three plate conversion and high resolution. went.

For the simulation, ITE (Institute o
9 high-definition standard images of f Television Engineers) were used, and the training of class classification adaptation processing was also performed with these 9 images. Also, the resolution when creating the student image signal is 1/4.
In order to reduce the number of bits, processing from 8 bits to 6 bits was performed. As a result, compared with the image signal in which the class classification adaptive processing is performed on the student image signal in which the bit number is not reduced, the image signal in which the noise is considerably reduced is obtained.
Further, when the effect as a noise removing method for the image signal of the same resolution which does not include the single-plate / three-plate conversion and the high resolution is confirmed, the effect is confirmed.

As described above, in the digital still camera 1, noise is removed by lowering the resolution of the A / D converter 7 to be lower than the resolution of the output image signal, and the image signal processing unit 8 removes the missing gradation. S / N can be improved by creating the class classification adaptive processing in.

FIG. 2 shows another embodiment of the present invention.
3 and FIG. 24.

The digital still camera 101 shown in FIG.
Is an improvement of the digital still camera 1 shown in FIG. 1 described above, and the same components are denoted by the same reference numerals and detailed description thereof will be omitted.

This digital still camera 101 has a CC
The gradation adjustment is adaptively created according to the amount of light incident on the D image sensor 5, and the signal adjustment unit 1 is controlled by the CPU 110.
C by monitoring the degree of amplification of AGC in 06
The amount of light incident on the CD image sensor 5 is detected, and the degree of amplification of AGC in the signal adjusting unit 106, that is, C
Depending on the amount of light incident on the CD image sensor 5, how many gradations are to be performed is determined.

Specifically, the controller 118 uses the CP
Under the control of U110, the AG in the signal conditioning unit 106
Depending on the degree of amplification of C, A by the A / D conversion unit 107
By manipulating the number of bits of the D / D conversion output and changing the coefficient in the class classification adaptive processing in the image signal processing unit 108 according to the number of bits, an adaptive gradation according to the amount of light incident on the CCD image sensor 5 is obtained. Create.

In this digital still camera 101, A
The coefficients of the class classification adaptation processing are prepared for the number of bits that can be selected in the / D conversion unit 107, and when the subject is very bright, gradation creation is not performed at all, and processing is performed. When the subject is dark, optimum gradation creation is performed in which noise is not emphasized in correspondence with the darkness. That is, gradation creation that is adaptive to the shooting environment is performed.

That is, this digital still camera 10
1, the signal adjusting unit 106, regarding the image signal obtained as the image pickup output by the CCD image sensor 5,
AGC that adjusts the gain so that the signal level is constant
The input signal level in processing is detected, and the value is detected by CPU
It is supposed to be given to 110.

Then, the A / D conversion unit 107 receives information indicating how many bits the pixel value is to be output from the CPU 110 via the controller 118, and according to the instruction, the image signal output from the signal adjustment unit 106. Is converted from an analog signal to a digital signal.

Further, the image signal processing unit 108 stores the prediction coefficient set corresponding to the class number classified by the class classification circuit 32 in the coefficient memory 133 by the kind of the number of bits selectable by the A / D conversion unit 107. There is. Then, in the image signal processing unit 108, as shown in FIG. 24, the information indicating how many bits the A / D conversion has been performed is CP.
The class number received from U110 via the controller 118 and classified by the class classification circuit 32, and A /
A prediction coefficient set corresponding to the number of bits after D conversion is read from the coefficient memory 133, and the adaptive processing circuit 135 multiplies the prediction coefficient corresponding to each pixel value of the prediction tap,
The predicted pixel value is processed using a linear sum, and the calculated predicted pixel value is output.

Here, the prediction coefficient set is obtained in advance by learning and is obtained by the learning process by the learning device 40 shown in FIG. Learning device 40
In (1), processing is performed by the number of patterns that the number of bits after A / D conversion can take. For example, the output of the entire imaging system is 8
The number of bits that can be taken after A / D conversion is 4, 5,
If it is any one of 6, 7, and 8 bits, the coefficient set of four patterns excluding the same 8 bits as the output is learned. Since only one coefficient set can be obtained in the series of learning processes according to the flowchart of FIG. 17 described above, it is necessary to perform learning by the number of patterns required for the system.

In order to evaluate the effect of the present invention, a single-plate CCD is used for the digital still camera 101 shown in FIG.
A simulation was performed assuming a Bayer array for the color filter array of the image sensor.

An RGB image signal having a sufficiently high resolution is prepared and used as a teacher image signal for learning. The teacher image signal is thinned or a low-pass filter is applied to reduce the resolution, and the amount of light incident on each pixel of the CCD image sensor is taken into consideration in the AG
Noise is added assuming that the gain is increased by C. As for the noise here, some noises having different intensities are prepared, and a plurality of noise image signals to which the noises are added are created. For those image signals, the number of bits is reduced according to the noise amount of each pixel value (a large amount of noise is reduced for a large amount of noise, and a small amount is reduced for a small amount of noise). Reduce the number of bits), and use as a student image signal for learning. After preparing a plurality of patterns of the student image signal and the teacher image signal in this way, the source code of the algorithm that performs the same operation as the learning device was created, and the prediction coefficient of each pattern was obtained based on the above method. A simulation was actually performed using the prediction coefficient. In this process, three processes of single plate / three plate conversion, high resolution, and multi-gradation are collectively performed. Then, the student image signal in which the number of bits is not reduced by only adding noise to the processing result is compared with the image signal subjected to the class classification adaptive process of only single plate / three plate conversion and high resolution. This was done for each pattern.

Nine high-definition standard images of ITE were used for the simulation, and the learning of the class classification adaptation processing was also performed on the nine images. In addition, the resolution when creating the student image signal was reduced to 1/4, and the number of bits was reduced from 8 bits to 4 bits and from 8 bits to 6 bits according to the noise amount. As a result, it was confirmed that noise was considerably reduced in both patterns as compared with the image signal in which the class classification adaptive processing was performed on the student image signal in which the number of bits was not reduced. Further, when the effect as a noise removing method is confirmed for the image signal of the same resolution that does not include the single-plate / three-plate conversion and the resolution enhancement, the same effect is confirmed.

In the above description, the case of using the Bayer array as the color coding filter 4 has been described, but the present invention can be applied to the case of using other color coding filters. is there.

Here, this digital still camera 1, 1
25A to 25N show a configuration example of a color filter array constituting the color coding filter 4 that can be used for the CCD image sensor 5 of No. 01.

25A to 25G show the primary colors (R, G,
B) Green (G) and red (R) in the color coding filter 4 composed of a primary color filter array that passes the components.
An example of a blue (B) color arrangement is shown.

FIG. 25A shows a Bayer array, and FIG.
Shows an interline arrangement, and FIG. 25C shows a G stripe R.
FIG. 25D shows a B checkered arrangement, FIG. 25D shows a G stripe RB perfect checkered arrangement, and FIG. 25E shows a stripe arrangement.
5F shows a diagonal stripe arrangement, and FIG. 25G shows a primary color difference arrangement.

Further, FIGS. 25H to 25N show complementary colors (M,
The color arrangement of magenta (M), yellow (Y), cyan (C), and white (W) in the color coding filter 4 configured by the complementary color filter array that passes the Y, C, W, and G) components is shown. FIG. 25H shows a field color difference sequential arrangement, FIG. 25I shows a frame color difference sequential arrangement, and FIG.
FIG. 25K shows an improved MOS type array,
FIG. 25L shows a frame interleaved array, and FIG.
M shows a field interleave arrangement, and FIG. 25N shows a stripe arrangement.

The complementary color (M, Y, C, W, G) components are given by Y = G + R M = R + B C = G + B W = R + G + B Further, each color (Y that passes through the color coding filter 4 corresponding to the frame color difference order shown in FIG.
(M, YG, CM, CG) components are given by YM = Y + M = 2R + G + B, CG = C + G = 2G + B, YG = Y + G = R + 2G CM = C + M = R + G + 2R.

In addition to the digital still camera, the present invention is also applied to a video device such as a VTR with a built-in camera, an image processing apparatus used in, for example, a broadcasting business, and further, for example, a printer or a scanner. be able to.

Further, the class classification adaptive processing in the image signal processing unit 8 and the learning processing for obtaining the prediction coefficient set in the learning device 40 are performed by a CPU (connected to the bus 311) as shown in FIG. Central Process
ing unit) 312, a memory 313, an input interface 314, a user interface 315, an output interface 316, and the like. The computer program that executes the above processing is a computer controllable program that performs image signal processing recorded on a recording medium and processing an input image signal having a color component that represents one of a plurality of pixel positions. Is provided to the user as a recording medium in which is recorded, or a recording medium in which a computer controllable program that performs a learning process for generating a prediction coefficient set according to a class is recorded. The recording medium includes information recording media such as magnetic disks and CD-ROMs, as well as transmission media through networks such as the Internet and digital satellites.

[0136]

As described above, according to the present invention, the A / D
Noise can be removed by lowering the resolution of the conversion means to be lower than the resolution of the output image signal, and the S / N can be improved by creating a missing gradation by the classification adaptive processing of the image signal processing means. it can.

Further, according to the present invention, the number of output bits in the A / D conversion means is determined according to the amount of light incident on the image sensor, and the coefficient in the class classification adaptive processing in the image signal processing means is determined by the number of bits. By changing the S, the gradation can be created adaptively to the shooting environment, and the S / N
Can be improved.

[Brief description of drawings]

FIG. 1 is a conceptual diagram of high gradation using a class classification adaptive process.

FIG. 2 is a conceptual diagram of high gradation and single plate / three plate conversion using a class classification adaptive process.

[FIG. 3] High gradation using class classification adaptive processing, single plate 3
It is a conceptual diagram of plate conversion and resolution enhancement.

FIG. 4 is a block diagram showing a configuration of a digital still camera to which the present invention has been applied.

FIG. 5 is a flowchart showing an operation example of the digital still camera.

FIG. 6 is a block diagram showing a configuration of an image signal processing circuit in the digital still camera.

FIG. 7 is a flowchart showing the operation of the first image processing performed by the image signal processing circuit in the operation example of the digital still camera.

FIG. 8 is a flowchart showing the operation of the first image processing performed by the image signal processing circuit in the operation example of the digital still camera.

FIG. 9 is a flowchart showing another operation example of the digital still camera.

FIG. 10 is a flowchart showing the operation of the first image processing performed by the image signal processing circuit in another operation example of the digital still camera.

FIG. 11 is a flowchart showing an operation of second image processing performed on the image signal processing circuit in another operation example of the digital still camera.

FIG. 12 is a flowchart showing an operation of third image processing performed by the image signal processing circuit in another operation example of the digital still camera.

FIG. 13 is a flowchart showing another operation example of the digital still camera.

FIG. 14 is a block diagram showing a configuration of a class classification adaptive processing unit in the image signal processing circuit.

FIG. 15 is a flowchart showing an operation of the class classification adaptation processing section.

FIG. 16 is a block diagram showing a configuration of a learning device that learns a coefficient set used in the class classification adaptive processing.

FIG. 17 is a flowchart showing an operation of the learning device.

FIG. 18 is a diagram schematically showing the relationship (mode 1) between the predicted pixel position and the placement of class taps in the class classification adaptive processing.

FIG. 19 is a diagram schematically showing the relationship (mode 2) between the predicted pixel position and the placement of class taps in the class classification adaptation process.

FIG. 20 is a diagram schematically showing the relationship (mode 3) between the predicted pixel position and the placement of class taps in the class classification adaptation process.

FIG. 21 is a diagram schematically showing the relationship (mode 4) between the predicted pixel position and the placement of class taps in the class classification adaptive processing.

FIG. 22 is a diagram schematically showing a relationship (modes 1 to 4) between predicted pixel positions and predicted tap arrangements in the class classification adaptive process.

FIG. 23 is a block diagram showing another configuration of a digital still camera to which the present invention has been applied.

FIG. 24 is a block diagram showing a configuration of a class classification adaptation processing unit in the image signal processing circuit of the digital still camera.

FIG. 25 is a diagram schematically showing a configuration example of a color filter array that constitutes a color coding filter that can be used in the CCD image sensor of the digital still camera.

FIG. 26 is a block diagram showing a configuration of a computer system used in a learning process for obtaining the prediction coefficient set.

[Explanation of symbols]

1 digital still camera, 2 lenses, 3 iris, 4 color coding filter, 5 CCD image sensor, 6 signal adjusting section, 7 A / D converting section, 8 image signal processing section, 9 timing generator, 10 CP
U, 11 motor, 12 motor, 13 flash,
15 memories, 14 interfaces, 16 memories,
17 recording medium, 18 controller, 19 power supply unit,
19A battery, 19B DC / DC converter, 2
0 operation unit, 21 defect correction circuit, 22 clamp circuit, 23 white balance adjustment circuit, 24 switch, 25 class classification adaptive processing unit, 26 gamma correction circuit, 27 correction circuit, 28 space conversion circuit, 31 class tap extraction circuit, 32 Class classification circuit, 33 coefficient memory, 34 prediction tap extraction circuit, 35 adaptive processing circuit, 40 learning device, 41 thinning circuit, 42 class tap extraction circuit, 43 class classification circuit, 44 prediction tap extraction circuit, 45 prediction target pixel extraction circuit , 46 arithmetic circuit, 47 learning data memory, 48 arithmetic circuit, 49
Coefficient memory, 101 Digital still camera, 110
CPU, 106 signal adjusting section, 118 controller, 107 A / D converting section, 108 image signal processing section, 1
33 coefficient memory, 135 adaptive processing circuit, 310 computer system, 311 bus, 312 CPU,
313 memory, 314 input interface, 31
5 user interface, 316 output interface

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 1/40 G09G 5/00 520A 5C082 5/14 5/36 520A 9/07 H04N 1/40 103Z (72 ) Inventor Takashi Sawao 6-7-35 Kitashinagawa, Shinagawa-ku, Tokyo Sony F-term (reference) 5B057 BA02 BA12 CA01 CA08 CA16 CB01 CB08 CB16 CE11 CE16 CH01 CH11 5C021 PA85 PA92 RB03 RB07 RB09 XA34 XA35 YA01 YA06 YC01 YC07 5C065 AA01 BB22 BB48 CC01 DD01 GG08 GG13 GG15 GG22 5C066 AA01 AA11 BA01 CA05 CA06 CA07 GA01 GA05 GB01 KA01 KM01 KM02 5C077 MM03 MP08 NN02 PP32 PP33 PP54 PP55 PQ12 PQ22 RR01 RR05 5C082 AA27 BA12 BA35 BA36 BB15 BB25 BB53 CA11 CB03 CB05 DA53 DA73 DA86 MM10

Claims (11)

[Claims] 1. An analog / digital conversion step for converting an input image signal into a digital signal with a resolution lower than that of the output image signal, and an adaptive class classification process for the input image signal converted into the digital signal. An image signal processing method comprising: a gradation creation processing step of performing gradation creation processing; and obtaining an output image signal having multiple gradations. 2. In the gradation creation processing step, a pixel value at a class tap position and a pixel value at a predicted tap position are extracted from a region of interest of the input image signal, and an image is extracted from the pixel values at the extracted class tap position. A method of classifying a signal pattern, multiplying each pixel value of the extracted prediction taps by a prediction coefficient set corresponding to the classified image signal pattern class, and using the linear sum as a prediction pixel value. The image signal processing method according to item 1. 3. The image signal processing method according to claim 2, wherein, in the gradation creation processing step, a class classification processing is performed for classifying a class by ADRC processing on pixel values extracted as class taps. 4. An analog / digital conversion means for converting an input image signal into a digital signal from an analog signal with a resolution lower than that of the output image signal, and an input image signal converted into a digital signal by the analog / digital conversion means. An image signal processing device for performing gradation creation processing by adaptive class classification processing to obtain an output image signal having multiple gradations. 5. The image signal processing means, class tap extracting means for extracting a pixel value at a designated class tap position from the attention area of the input image signal, and prediction designated from the attention area of the input image signal. Predictive tap extraction means for extracting pixel values from tap positions, class classification means for classifying image signal patterns into classes for pixel values extracted by the class tap extraction means, and each class classified by the class classification means A coefficient storage means for storing each prediction coefficient set and a prediction coefficient set corresponding to the class classification of the image signal patterns classified by the class classification means are given from the coefficient storage means and extracted by the prediction tap extraction means. Multiply each pixel value of the prediction taps by the above-mentioned prediction coefficient set, perform a process to obtain a prediction pixel value by linear sum, and obtain 5. The image signal processing apparatus according to claim 4, further comprising an adaptive processing unit that outputs the predicted pixel value. 6. The class classification means is ADRC (Adapt
The image signal processing device according to claim 5, wherein the image signal pattern is classified based on the characteristic information by ive dynamic range coding processing. 7. An image pickup means for picking up an image of a subject, and an analog / digital conversion means for converting an input image signal obtained as an image pickup signal by the image pickup means from an analog signal to a digital signal with a resolution lower than that of an output image signal. And image signal processing means for performing gradation creation processing by adaptive class classification processing on the input image signal converted into a digital signal by the analog / digital conversion means to obtain an output image signal with multiple gradations. An imaging device characterized by. 8. The image pickup means is a Bayer array CCD.
An image sensor according to claim 7,
The imaging device according to the item. 9. An analog / digital conversion step of converting an input image signal into a digital signal with a resolution lower than that of the output image signal, and an adaptive class classification process for the input image signal converted into the digital signal. A gradation control processing step for performing gradation generation processing, and computer controllable to perform an image signal processing for generating an output image signal having multiple gradations from an input image signal by gradation generation processing. A recording medium on which a program is recorded. 10. In the gradation creation processing step, a pixel value at a class tap position and a pixel value at a predicted tap position are extracted from a region of interest of the input image signal, and the pixel value at the extracted class tap position is imaged. A computer characterized by classifying a signal pattern, multiplying each pixel value of the extracted prediction taps by a prediction coefficient set corresponding to the classified image signal pattern class, and using the linear sum as a prediction pixel value. The recording medium according to claim 9, wherein a controllable program is recorded. 11. A computer-controllable program recorded in the gradation creation processing step, which performs a class classification process for classifying a class by ADRC processing on pixel values extracted as class taps. Item 10. The recording medium according to item 10.