JP2002335454A - Image processing unit, image processing method and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a natural image without a sense of incongruity with a minimum function configuration by correcting deviation in density caused on a border face in image data comprising a plurality of blocks. SOLUTION: The image processing unit that processes a pixel signal received from an image pickup section (101) including a plurality of photoelectric conversion elements and outputting the pixel signal obtained by applying photoelectric conversion to an object optical image with a plurality of the photoelectric conversion elements to each of a plurality of areas of an object image, includes a dark current correction section (114) that corrects a difference of the pixel signals caused by the characteristic of a plurality of the photoelectric conversion elements, a screen correlation extract section (112) that acquires a correlation between the pixel signals outputted by each of a plurality of the areas based on the pixel signals corrected by the dark current correction section, and a screen matching section that corrects the pixel signal corrected by the dark current correction section based on the correlation value acquired by the screen correlation extract section.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method, and an image processing apparatus for performing image processing on an image divided and output from a photographing apparatus into a plurality of blocks.

[0002]

2. Description of the Related Art Conventionally, an image pickup apparatus such as a digital video camera or a digital still camera using a solid-state image pickup device such as a CCD is disclosed in, for example, JP-A-5-137059 and JP-A-6-141246. Then, the subject is partially imaged using a plurality of CCDs,
There has been known a method of increasing the resolution by synthesizing those partial images in image processing to obtain a captured image of the entire subject.

In recent years, an imaging system using a CCD having millions of pixels has been developed, and various devices have been devised for performing high-speed signal processing.

FIG. 13 shows a configuration example of a conventional image pickup device.

FIG. 13 is a diagram showing a schematic configuration of a CCD generally used. Mainly, a light receiving section 1 composed of a plurality of photoelectric conversion elements, light shielding sections 2L and 2R in which light shielding members are arranged on the upper surfaces of the plurality of photoelectric conversion elements to prevent light from entering some photoelectric conversion elements, and horizontal transfer. CCD (HCCD) 4
And a vertical transfer CCD (VCCD) (not shown), a charge-voltage conversion amplifier 5, and the like.

The driving method is as follows. After the electric charges generated in the light receiving section 1 are transferred to the VCCD, the electric charges are transferred in units of one row in the horizontal direction.
Charge transfer is performed in order to read from CCD4, and HCC
The output from D4 is charge-voltage converted by the charge-voltage conversion amplifier 105 and output.

FIG. 14A shows a configuration example of another conventional image sensor.

FIG. 14A is a diagram showing a configuration in which the photoelectric conversion output from the light receiving section 1 is divided into two left and right horizontal transfer CCDs (HCCDs) 4L and 4R and simultaneously read out. FIG.
Also in the imaging device having the configuration shown in FIG. 4A, charge transfer of the photoelectric conversion output is performed by the HCCDs 4L and 4R in units of one row in the horizontal direction.
The charge is transferred to the left by the CCD 4L and to the right by the photoelectric conversion output of the right half by the HCCD 4R, and is further subjected to charge-voltage conversion by individually prepared charge-voltage conversion amplifiers 5L and 5R.

In this configuration, two HCCDs 4L and 4R are used.
1 can be read in parallel with each other.
The read driving of one frame can be performed in about half the driving time of the configuration shown in FIG.

[0010] Japanese Patent Application Laid-Open No. 3-79991 discloses that
An image in which a plurality of image sensors are provided to divide and read image information, an image pickup range of an adjacent image sensor is overlapped at a boundary portion, and images captured by both image sensors are stuck at the boundary portion to obtain an image of the entire image information. In the reading device, a pixel position having a small spatial density change between adjacent pixels is detected from the image data to be formed, and the image data forming the captured image of one image sensor at the pixel position and the other are detected. An image reading apparatus is shown in which image data constituting an image picked up by an image sensor is stuck.

[0011]

As described above, when a subject is partially divided and imaged by a plurality of image sensors, and the captured images are pasted together to obtain a captured image of the entire subject, the distance between each image sensor is reduced. If the sensitivity difference is different, a step in the image density occurs due to the sensitivity difference at the bonding portion (boundary portion), and a problem occurs in that a captured image with unnatural image quality is obtained.

The above-mentioned Japanese Patent Application Laid-Open Nos. 5-137059 and 6-114646 disclose a technique in which a plurality of image pickup devices are used to partially image a subject. There is no description of the problem of the density step occurring at the boundary of the captured image due to the sensitivity difference, and no solution to this problem is disclosed.

In order to obtain a picked-up image in which the bonding is not conspicuous, Japanese Patent Application Laid-Open No. 3-79991 discloses a method for improving image quality defects such as image disturbance at a boundary portion. Since the position where the density change at the boundary portion is small is detected and the two partial images are simply pasted, the pasting process is complicated and the processing time becomes long.

Further, since this method merely controls the bonding position of two picked-up images on a line-by-line basis, it is difficult to effectively suppress a sharp change in density at the boundary in the vertical direction. is there. In particular, it is difficult to effectively reduce a density step generated at a boundary portion due to a sensitivity difference between image pickup devices and obtain a picked-up image in which bonding is not conspicuous.

As a method for improving this, Japanese Patent Application Laid-Open No.
In Japanese Patent Application Laid-Open No. 055558, a subject is divided into two parts by a CCD so as to be divided into right and left parts so that a boundary part is overlapped, and the two parts are stored in an image memory via an analog signal processing circuit and an A / D converter. Further, there has been proposed a digital camera in which a shading correction unit corrects a variation in sensitivity distribution in an imaging surface, and an image synthesis unit attaches a boundary portion to generate a captured image of the entire subject.

In this method, the image of the boundary portion in which the density step at the boundary portion is reduced is generated by processing such as the average value calculation of the image of the boundary portion of the left and right partial images. It is generated by synthesizing the image and the partial images excluding the image of the left and right boundary parts, but it is necessary to divide the subject light image into a plurality of partially overlapping images and then lead it to a plurality of image sensors In addition, an increase in the shape due to the three-dimensional space, assembling accuracy, and an accompanying cost increase occur.

Further, in the configuration shown in FIG. 14A, since the imaging unit is realized by one CCD, there is no need for means such as optical path division, and the configuration of the imaging system can be simplified and the cost can be reduced. Is possible, but the left and right regions of the captured image are independent (ie, HCCD 4L and amplifier 5L,
Since reading is performed separately by the HCCD 4R and the amplifier 5R), a step at the boundary surface becomes conspicuous.

FIG. 14B is an example showing the photoelectric conversion output for one horizontal line indicated by aa 'in FIG. 14A.

FIG. 14A shows an example of a scene of a sunny day, in which the sun, mountains, trees, and grass are shown. FIG.
14B shows an output from the light-shielding portion 2R in FIG. 14A, level B shows an output from the light-shielding portion 2L, and level C shows an output of a portion corresponding to the sun. The difference D is a level difference generated by reading by the left and right independent systems described above.

A method of making the level difference look uncomfortable on the screen is called screen matching. As one of the methods, there is a method of making the density step on the boundary surface inconspicuous. This is to correct the density step on the boundary surface based on the degree of correlation obtained from the image data on the boundary surface.
It is necessary to take measures against dark current that causes a decrease in N.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and has a function of correcting a density deviation occurring at a boundary surface in image data composed of a plurality of blocks so that a natural image without a sense of incongruity can be minimized. It is intended to be realized by a configuration.

It is another object of the present invention to make it possible to cope with an increase in the number of pixels of an image sensor.

[0023]

In order to achieve the above object, a plurality of photoelectric conversion elements are provided, and a pixel signal obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements is converted into a pixel signal of the subject image. The image processing apparatus of the present invention, which performs processing of a pixel signal input from an imaging unit that outputs for each of a plurality of regions, includes a first correction unit configured to correct a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements. A correlation value acquisition unit for acquiring a correlation value between pixel signals output for each of the plurality of regions based on the pixel signal corrected by the first correction unit; and a correlation acquired by the correlation value acquisition unit. A second correction unit for correcting the pixel signal corrected by the first correction unit based on the value.

Also, a plurality of photoelectric conversion elements are provided, and pixel signals obtained by photoelectrically converting the subject optical image by the plurality of photoelectric conversion elements are input from an image pickup means for outputting a plurality of regions of the subject image for each of a plurality of regions. The image processing method of the present invention for processing pixel signals includes a first correction step of correcting a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements, and a pixel corrected in the first correction step. A correlation value acquisition step of acquiring a correlation value between pixel signals output for each of the plurality of regions based on the signal, and a first correction step based on the correlation value acquired by the correlation value acquisition step. A second correction step of correcting the corrected pixel signal.

According to the above configuration, when performing the correction processing by the screen matching, the image data for which the correction value is to be obtained is subjected to the dark current correction in advance, whereby the screen matching processing with high accuracy can be performed. Good image data can be obtained.

According to a preferred embodiment of the present invention, a storage means for storing at least one frame of pixel signals output from the image pickup means, and a pixel signal for one frame stored in the storage means. Wherein the first correction means corrects a pixel signal near a boundary between the plurality of regions, outputs the corrected pixel signal to the correlation value obtaining means, and outputs the corrected pixel signal to the imaging means by the first correction means. 1 output from
There is further provided control means for correcting the pixel signals for the frames and controlling the corrected pixel signals to be corrected by the second correction means.

Also, a plurality of photoelectric conversion elements are provided, and pixel signals obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements are input from an image pickup means for outputting a plurality of regions of the subject image for each of a plurality of regions. Another image processing method according to the present invention for processing a pixel signal includes a storage step of storing at least one frame of a pixel signal output from the imaging unit, and a pixel signal of one frame stored in the storage step. A first correction step of correcting a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements for a pixel signal near a boundary between the plurality of regions by the first correction unit; A correlation value acquiring step of acquiring a correlation value between the pixel signals output for each of the plurality of regions based on the pixel signals corrected in the correcting step; A second correction step of correcting a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements with respect to the raw signal, and a second correction step based on the correlation value acquired in the correlation value acquisition step. And a third correction step of correcting the corrected pixel signal.

According to the above configuration, when a correction value for performing the correction process by the screen matching is obtained, only the necessary part is inputted from the image data to select and obtain the correction value, thereby simplifying the operation of the memory control. A highly accurate screen matching process can be performed by the system configuration.

According to a preferred aspect of the present invention, the apparatus further comprises switching means for switching whether or not the first correction means is used, and the switching means switches the first correction means to non-use. In this case, the correlation value acquisition unit and the second correction unit perform processing using a pixel signal input from the imaging unit.

Further, the method further comprises a switching step of switching whether or not to perform the first correction step or the first and second correction steps. In the correlation value acquiring step, the second correction step or the third correction step, processing is performed using a pixel signal input from the imaging unit.

According to the above configuration, for example, when the accumulation time is short, or when the temperature of the imaging unit is not high and the difference between the individual dark current outputs of the plurality of photoelectric conversion elements is relatively small, the correction effect of the dark current correction is obtained. Because the effects of negative effects such as processing time and current consumption are greater than in the case of, especially when the camera is operated in the continuous shooting mode, the shooting interval increases, so the screen correlation according to these factors Optimal processing can be performed by selectively performing the dark current correction processing on the main image data to be extracted.

[0032]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

<First Embodiment> FIG. 1 shows a first embodiment of the present invention.
FIG. 2 is a schematic block diagram illustrating a configuration of an imaging device such as a digital camera according to the embodiment.

As shown in FIG. 1, the image pickup apparatus includes an image pickup unit 101 including a CCD area sensor, a microcomputer PRS105 for controlling the entire image processing, and an original image processing unit 110 for processing an image signal from the image pickup unit 101. And an image processing unit 104 that receives an output from the original image processing unit 110 and performs processing such as white balance and color interpolation; a storage unit 103 including a non-volatile storage member that stores a plurality of image data; And a display unit such as a display memory 121 and an LCD, and a JPEG compression processing unit 125 and a storage device 130 for storing processed data.

The microcomputer PRS105 includes, for example, a CPU (central processing unit), a RAM, a ROM, an EEP
It is a one-chip computer (hereinafter abbreviated as a microcomputer) in which a ROM (electrically erasable programmable ROM), input / output ports and the like are arranged.

The microcomputer PRS105 performs a series of operations based on a sequence program stored in the ROM. Since the feature of the present invention resides in the original image processing unit 110, arrows are added in FIG. 1 to clearly show that the original image processing unit 110 is under the control of the microcomputer PRS 105. It is controlled by the PRS 105.

In the image pickup section 101, a light beam from a subject is formed on an image pickup device through an optical system (not shown). The image pickup device photoelectrically converts the light beam into an electric signal, and further converts the electric signal into a digital value. To the original image processing unit 110. The imaging unit 101 will be described later in detail with reference to FIG.

The original image processing unit 110 receives the image pickup signal output from the image pickup unit 101, and extracts pixel data of a light-shielding unit (optical black) in preparation for performing dark correction by the OB clamp unit 111. Are temporarily stored in the buffer memory of the storage unit 103. The buffer memory has a capacity capable of storing a plurality of image data of one screen read from the image sensor as one unit.

The buffer memory is used by the image processing unit 1 when performing a plurality of photographing operations in a short time, for example, a continuous photographing operation.
Although it is provided to eliminate or minimize the influence of the processing speed after 04, in the first embodiment, the dark output of the image sensor used for performing the dark current correction of the pixel data is also stored. Has functions.

The input / output of data between the original image processing unit 110 and the storage unit 103 is controlled by the memory control unit 113.

The pixel data once stored in the storage unit 103 is read into the original image processing unit 110, and the original image processing unit 110
Current correction unit 114 and screen correlation extraction unit 11
2. Screen matching unit 116, shading correction unit 1
17, signal processing is performed by the flaw correction unit 118 and the like. The switch 141 supplies data by selectively switching between a case where a screen correlation is extracted and a case where signal processing for output to the image processing unit 104 is performed.

The signal processing procedure in the original image processing section 110 will be described later in detail with reference to FIG.

The image processing unit 104 includes an original image processing unit 110
Performs a series of image processing related to so-called picture making, such as white balance, gamma correction, color interpolation, and color correction, on the pixel data processed by, and outputs signals for each of RGB color components. Therefore, the output data amount from the image processing unit 104 has been increased to an integral multiple of the output data amount from the original image processing unit 110 in order to perform color interpolation processing.

For example, when the image pickup device of the image pickup section 101 is covered with a Bayer array color filter, R (red)
A row in which a signal and G (green) are output alternately;
A row for alternately outputting a B (blue) signal exists every other row in units of one horizontal line. Since the data output from the original image processing unit 110 has the same Bayer arrangement order, when the image processing unit 104 performs the color interpolation processing on the input data and outputs the RGB data for each pixel, the input data is tripled. This is the amount of image data. FIG. 2 conceptually illustrates this.

The output from the image processing unit 104 is reduced in size by the display processing unit 120 including the filtering process so that the output becomes the optimum size to be displayed on the display unit 122 such as an LCD provided in the imaging device. Is stored in the display memory 121. here,
L stored for each color component of RGB and provided in the camera
The interface with the drive unit of the CD display unit 122 is also RG.
B is performed. The output is performed in line units like the input.

The JPEG compression processing section 125 comprises a YUV conversion section and a JPEG encoding processing section. YU
The V conversion unit converts the RGB image data input from the image processing unit 104 into a luminance component and a color difference component, and performs image compression processing by a JPEG encoding processing unit. The JPEG encoding unit compresses the input image data by JPEG encoding. Here, DCT (Discrete Cosine Transform) and Huffman Transform are performed, and a signal composed of luminance and color difference is converted into a luminance signal of 8 × for DCT processing.
The horizontal unit is 16 pixels and the color difference signal is 8 × 8 pixels as the minimum unit.
The function for converting from raster scan to zigzag scan is also included in this part.

The data subjected to the compression processing is stored in the storage device 1 in the subsequent stage.
30 are sequentially stored. Repeat the above encoding process,
The processing of one screen of the imaging data from the imaging unit 101 is performed.
Before and after this JPEG encoding process, the microcomputer PRS10
Reference numeral 5 generates a header and a footer such as shooting date and time information in an optional format as necessary, and stores the header and footer in the storage device 130 together with the image data compressed by encoding.

At this time, two pieces of image data having different image sizes are stored in the storage device 130 based on the same image pickup data. The first image data is not subjected to the image reduction processing by the image reduction processing unit in the image processing unit 104, and the second image data is the image data subjected to the image reduction processing by the image reduction processing unit in the image processing unit 104. is there. The first image data is generated as image data representing a captured image, and the second data is generated mainly as management information for editing or confirming image data such as a thumbnail. Although the first image data and the second image data are managed as independent image files,
An association has been made.

The compression ratio required for JPEG encoding and the set values such as conversion reference data required for the JPEG encoding are set at the time when the shutter of the imaging device is pressed by the microcomputer PRS10.
5 completes the setting.

The storage device 130 includes a volatile storage member having a relatively high writing speed capable of storing a plurality of image data, and a removable nonvolatile storage member having a relatively low writing speed. The output from the JPEG compression processing unit 125 is first stored in a volatile storage member. At this time, two pieces of image data generated from the same image data and having different image sizes are stored. Thereafter, the image data is sequentially stored in the non-volatile storage member at intervals of the operation of the imaging device.

The non-volatile storage member is detachable from the imaging device, and after the data is written from the volatile storage member for one or a plurality of screens in a state where the non-volatile storage member is mounted on the imaging device, the non-volatile storage member is removed from the imaging device, and Data stored and attached to another system that can be read in the data format of the device can be reproduced, edited, and saved. Both the first image data and the second image data are managed in the storage device 130 in a format that satisfies the above object.

FIG. 3 is a schematic diagram showing the internal configuration of the image pickup unit 101 shown in FIG. In FIG. 3, reference numeral 90 denotes an image pickup device such as a CCD area sensor, and 81 and 82 denote known CDSs.
(Double correlation sampling) / AGC (Automatic gain adjustment)
The units 83 and 84 are known A / D converters. Also,
A sensor driving unit (not shown) for driving the image sensor 90 is also provided. With this configuration, after performing analog processing suitable for the read photoelectric conversion output, the signal is converted into a digital signal and output.

The image sensor 90 is a known Bayer array type CC
D, and 91L and 91R are photoelectric conversion elements existing in the left half and right half areas of the screen, respectively, and 92L and 92R are vertical transfer elements existing in the left half and right half areas of the screen, respectively. CCD (VCCD).
Each photoelectric conversion element and VCCD are paired, and a plurality of the pairs are arranged two-dimensionally to form an imaging area, and an image is captured by converting a light flux from a subject into electric charges. is there. 4L is VCCD92
Horizontal transfer CCDs (HCCDs) 4R for transferring charges sequentially transferred from L in the horizontal direction are HCCDs for transferring charges sequentially transferred from the VCCD 92R in the horizontal direction.

The light receiving surface of the image sensor 90 that forms the image pickup area is covered with R, G, and B primary color filters.
Are arranged in a checkered pattern, and R and B are alternately arranged for each line between G filters (Bayer arrangement). Further, the pixels are formed in a square lattice, so that the calculation after the image is captured is easy.

The charge generated by the photoelectric conversion element 91L is VC
After being transferred to the CD 92L, the data is sequentially transferred in the vertical direction to the HCCD 4L in units of one row in the horizontal direction. Thereafter, the charges are transferred in the direction B by the HCCD 4L, and are converted from charges to voltages by the charge-voltage conversion amplifier 95.
/ AGC unit 81, and further processed by A / D converter 83
After being converted to digital data, the data is output to the original image processing unit 110 shown in FIG.

On the other hand, the charges generated in the photoelectric conversion unit 91R are transferred to the VCCD 92R, and then sequentially transferred to the 9HCCD 4R in the vertical direction in units of one horizontal line. Then, it is transferred in the direction A by the HCCD 4R,
The charge-voltage conversion amplifier 96 converts the charge into a voltage,
The signal is processed by the CDS / AGC unit 82 and the A / D converter 8
After being converted into digital data in step 4, the data is output to the original image processing unit 110 shown in FIG. In this manner, the captured image data is output separately for the left and right regions.

The VCCD 9 constituting the image pickup device 90
Each of the 2L and 92R, the HCCDs 4L and 4R, the photoelectric conversion element of the light receiving section, and the photoelectric conversion element of the light shielding section is constituted by a larger number than those shown in FIG. For example, in FIG. 3, the photoelectric conversion element of the light-shielding portion is represented by one each at the right end and the left end in one horizontal direction, but is actually constituted by a plurality.

FIG. 4 shows a state in which the viewfinder is viewed through a camera eyepiece (not shown). AFP1 to AFP3 show three distance measuring points, and show a state where the focus is adjusted at the distance measuring point AFP2. ing. Also, in FIG. 4, each display shown at the lower end shows a setting state of the camera such as a shutter speed and a focus determination of focus adjustment, and shows a state in which all display units are turned on for explanation. During camera operation,
Lighting or extinguishing is performed independently in accordance with the operation state, so that all the displays are not turned on as shown in FIG.
When each of the focus detection points AFP1 to AFP3 is selected for focus adjustment, the outer and inner quadrilaterals are surrounded by an optical system and an illumination device (not shown) in a short time that can be sufficiently confirmed by a photographer. The lit area is lit red.

At this time, the light receiving section of the image pickup device included in the image pickup section 101 of FIG. 1 has K × L pixels as shown in FIG. Further, as described above, the signal of K × L pixels shown in FIG. 5 is processed by the original image processing unit 110, and the image processing unit 104 performs a series of signal processing such as white balance, color correction, and color interpolation. When performed for each of the RGB colors, an output image composed of K × L pixels is formed for each of the R, G, and B colors, as shown in FIG. In other words, the data amount from the image sensor composed of K × L pixels is tripled by performing signal processing by the original image processing unit 110 and the image processing unit 104.

FIG. 6 is a diagram for explaining a correction process when the output difference between the left and right screens is obtained from a plurality of photoelectric conversion outputs near the boundary.

FIG. 6A is a simplified view of the image sensor 90 shown in FIG. In the example shown in FIG. 6A, a landscape on a sunny day is photographed, and the sun, mountains, trees, and grass are photographed. Note that, in the configuration shown in FIG.
The same components as those shown in FIG. 4A are denoted by the same reference numerals, and description thereof will not be repeated.

FIG. 6B shows an example of photoelectric conversion output for one horizontal line indicated by bb 'in FIG. 6A. The level A in FIG. 6B is the light shielding unit 2R read from the HCCD 4R.
6B, and the level B is the HCCD of FIG.
The output level of the light shielding unit 2L read from the 4L is shown.
The difference D indicates the level difference between the right screen and the left screen.

The correlation extraction units A and B shown in FIG. 6A are imaging data areas used for obtaining a correction amount for correcting the difference D. In each case, the same number (plural) of photoelectric conversion element outputs are extracted from the left and right boundaries of the imaging region, and correlation values are obtained to generate correction values.

Here, a method of generating a correction value according to the present invention will be described.

First, the pixel data of the correlation extraction unit A read via the HCCD 4R and the amplifier 5R is corrected for dark current, and the corrected data AVGA is averaged.
Correlation extractor B read via 4L and amplifier 5L
Is corrected for dark current, and an average value AVGB of the corrected data is obtained. When the number of pixels in the horizontal direction of the correlation extraction units A and B is N, AS (N), BS (N)
Are the HCCD4R, the amplifier 5R, and the HCCD4, respectively.
The output is obtained by performing dark current correction on the pixel data of the correlation extraction unit A and the correlation extraction unit B read by L and 5L. AVGA = (AS (1) + AS (2) + ... + AS (N)) / N AVGB = (BS (1) + BS (2) + ... + BS (N)) / N (1)

Is determined by Next, the average value AVGA, A
A correlation value is obtained based on VGB. Correlation value = AVGB-AVGA (2) Since the correlation value obtained as described above can be used as a correction value as it is, here, dark current correction is performed on pixel data read through the HCCD 4L and the amplifier 5L. Calculate with the data after That is, B (n) is H
Assuming that the pixel data read through the CCD 4L and the amplifier 5L is output after performing dark current correction, first, BB (n) = B (n) + correction value (correlation value) (3)

Is calculated. As a result, the level corresponding to the difference D can be corrected. Note that n indicates a pixel number, and 1 ≦ n ≦ K / 2, where K is the number of photoelectric conversion elements forming one row in the horizontal direction. Next, the offset value C is added to the pixel data of the left and right screens. Note that A
(N) is an output obtained by performing dark current correction on the pixel data read via the HCCD 4R and the amplifier 5R. FB (n) = BB (n) + C (4) FA (n) = A (n) + C (5) Thus, the waveform shown in FIG. 6C can be obtained.

The offset value C corresponding to the level C in FIG. 6C is an offset value commonly added to the corrected pixel data so that digital signal processing can be easily performed by the image processing unit 104 shown in FIG. is there.

The output waveforms shown in FIGS. 6 (B) and 6 (C) show the output in the display image, and are image data composed of each color component of RGB obtained from the image sensor. Therefore, the correction value generation is performed independently for each of the RGB color components.

Next, the processing performed in the original image processing section 110 of FIG. 1 will be described in detail with reference to FIGS. FIG. 7 is a flowchart showing the operation of the original image processing unit 110.

As shown in FIG. 5, pixel data input from the image pickup unit 101 to the original image processing unit 110 is represented by K in the horizontal direction.
And L in the vertical direction. The output is composed of the outputs of the photoelectric conversion elements in the light receiving section and the outputs of the plurality of photoelectric conversion elements in the light shielding sections 2L and 2R shown in FIG. As shown in FIG. 3, the outputs of these photoelectric conversion elements are output through the A / D converter 83 on the left screen and output from the right screen via the A / D converter 84. 110 are input in parallel. The original image processing unit 110 performs parallel processing on these two sets of digital pixel data.

First, processing for acquiring dark current correction data used for dark current correction is performed.

The microcomputer PRS105 controls the image pickup unit and resets the output of the photoelectric conversion element to an initial value in a state where light does not enter the image pickup unit 101, and then performs a charge accumulation operation for a predetermined time. Do. Step S1
At 0, the digital pixel data output from the A / D converters 83 and 84 are input in parallel after the completion of the charge storage operation.

In step S11, pixel data corresponding to the dark current outputs of the plurality of photoelectric conversion elements in the light shielding portions 2L and 2R shown in FIG. 6A is temporarily stored and used for clamping by a known method. Determine digital data. Subsequently, a clamping process with pixel data obtained from the photoelectric conversion element in the area of the light receiving unit 1 in FIG. 6A is performed by a known method.

Next, in step S12, the clamped data (dark current correction data) is stored in the dark current correction data storage area secured in advance in the storage unit 103 via the memory control unit 113. Note that the storage order of the pixel data transferred in the direction A by the HCCD 4R is opposite to the reading order (that is, in the direction of the direction), so that the subsequent data can be easily handled at the time of storage in the storage unit 103. (In order of B) by the memory control unit 113.

The processing of steps S10 to S12 is repeated L times, which is the number of horizontal lines (that is, YE in step S13).
(Until S is reached). This completes the capture of the dark current correction data for one frame.

Subsequently, pixel data obtained by normal photographing of a subject is input by the imaging unit 101 (step S1).
4). This pixel data is also clamped (step S15), and the pixel data after the clamp processing is stored in the pixel data storage area secured in advance in the storage unit 103 via the memory control unit 113 (step S16).
The operations of steps S14 to S16 are repeated L times (step S1
7 is repeated, that is, until the processing of the image data for one frame is completed).

Next, switch 1 is used to perform dark current correction.
41, the image data and dark current correction data read from the storage unit 103 via the memory control unit 113 are input to the dark current correction processing unit 114, and dark current correction is performed on the pixel data (step S18). . Under the control of the microcomputer PRS105, the dark current correction unit 114 operates on the dark current correction data based on each accumulation time ratio so that the dark current correction data corresponds to the accumulation time when the pixel data is acquired. The adjustment is performed by performing the processing, and the adjusted dark current correction data is subtracted from the pixel data. As a result, it is possible to remove the dark current component included in the output of each photoelectric conversion element constituting the pixel data, and generate pixel data having good S / N characteristics. The raw data that has been subjected to the dark current correction as described above is input to the screen correlation extraction unit 112 and is also stored in the storage unit 103.

The screen correlation extraction unit 112 performs a screen correlation extraction process in order to obtain a correlation value serving as a reference for obtaining a correction value necessary for the image matching unit 116 to correct the level difference between the right screen and the left screen. (Step S1)
9). More specifically, FIG.
When the image data of the correlation extraction unit A and the correlation extraction unit B shown in (1) are input, the variable AVGA and the variable AVGB are obtained by the above equation (1), and the correlation value is calculated by the above equation (2).

The correlation value output from the screen correlation extraction unit 112 is read into the microcomputer PRS 105, processed according to the processing contents prepared in advance, and reflected in the deviation correction processing control by the screen matching unit 116. You.

Subsequently, screen matching is performed by the screen matching processing unit 116 using the correction value generated by the microcomputer PRS105 based on the obtained correlation value of the image (step S20). At this time, the switch 141 has been switched to the screen matching unit 116 side, and the dark current corrected pixel data is read from the storage unit 103 via the memory control unit 113. In the first embodiment, it is assumed that screen matching is performed by performing correction processing on image data on the left screen in FIG.

In step S20, the image data of the right screen in FIG.
Is input to The screen matching unit 116 adds the value (offset value) corresponding to the level C shown in FIG. On the other hand, when the image data of the left screen in FIG. 3 is input to the screen matching unit 116 via the switch 141, the microcomputer PRS105 uses the correction value prepared in advance based on the correlation value of the input line. The calculations shown in the above equations (3) and (4) are performed to correct the level shift.

After being subjected to the matching processing by the screen matching unit 116, the image data separately output by the two systems of A / D converters 83 and 84 in FIG. It is summarized as.

Subsequently, in step S21, the shading correction processing section 117 corrects the distortion of the image data caused by optical factors by multiplying the image data subjected to the screen matching processing by a coefficient prepared in advance.

Next, in step S22, the flaw correction processing unit 118 replaces a part where an appropriate photoelectric conversion output cannot be obtained due to dust or flaw with a value calculated based on the surrounding output. Since there is a difference between the output of the surrounding pixels and a portion having dust or scratches, it can be determined that dust or scratches exist when the difference is larger than a predetermined value.

The image data that has been processed by the flaw correction unit 118 is sequentially output to the image processing unit 104.

Screen matching processing (step S20),
Since the shading correction processing (step S21) and the flaw correction processing (step S22) are also performed in units of horizontal lines, these processings are repeated L times (YES in step S23), so that the photoelectric conversion of L horizontal lines is performed. With respect to the output of the conversion element, the original image processing for one frame is completed.

With the above processing, the output waveform shown in FIG. 6B is corrected to the output waveform shown in FIG. 6C, and the difference D shown in FIG. 6B, which is the level difference between the right screen and the left screen. Is corrected, and it is possible to obtain image data having no step on the left and right screens.

As described above, in the first embodiment, when the correction processing is performed by the screen matching, the dark current correction is performed in advance on the image data for which the correction value is to be obtained. Can be applied.

<Second Embodiment> Hereinafter, a second embodiment of the present invention will be described with reference to the drawings.

FIG. 8 is a schematic block diagram showing the configuration of an imaging device such as a digital camera according to the second embodiment of the present invention. In the configuration shown in FIG. 8 and the configuration shown in FIG.
The difference is that the positions of the switch unit 141 and the dark current correction unit 114 are interchanged. In FIG. 8, the same components as those in FIG. 1 are denoted by the same reference numerals, and the detailed operation of each component will not be described.

In the first embodiment, the original image processing unit 11
0, the dark current correction of the image data is performed for one frame as preprocessing of the screen correlation extraction, and the storage unit 103 stores the image data after the dark current correction. In the second embodiment, a case will be described in which pixel data requiring processing is subjected to dark current correction each time.

The operation of the original image processing section 110 'in the second embodiment will be described below with reference to FIG.

The processing performed in steps S10 to S17 is the same as the processing performed in FIG. 7, and a description thereof will not be repeated.

In step S30, the clamped image data of the correlation extraction unit A and the correlation extraction unit B shown in FIG. 6A and the dark current correction data are read from the storage unit 103 via the memory control unit 113. . Book second
In the embodiment, since the switch 141 is not provided between the dark current correction unit 114 and the memory control unit 113, the memory control unit 113 sends the dark current correction unit 11
4 is directly input with data. The dark current correction processing operation in the dark current correction unit 114 is the same as the operation described in step S18 of FIG.

In order to obtain a correlation value in the next step S 31, the switch 141 is switched so as to output the image data after the dark current correction to the screen correlation extracting unit 112. Therefore, the dark current correction unit 114 outputs the image data of the correlation extraction unit A and the correlation extraction unit B after the dark current correction to the screen correlation extraction processing unit 112, and the screen correlation extraction processing unit 112 outputs the data based on the input data. The correlation value is obtained by the same operation as in step S19 in FIG. Here, since the dark current is corrected by the dark current correction unit 114 and the image data input to the screen correlation extraction unit 112 is only the area of the correlation extraction unit A and the correlation extraction unit B, the processing in steps S30 and S31 The process is completed in a considerably short time as compared with the first embodiment.

Subsequently, processing such as screen matching by the screen matching processing unit 116 is performed by using the correction value generated by the microcomputer PRS105 based on the obtained correlation value of the image. In the second embodiment, Before that, the dark current correction processing by the dark current correction processing unit 114 is performed (step S32).

Since the storage unit 103 has already stored one frame of dark current correction data and image data,
These data are transferred from the storage unit 103 to the dark current correction unit 114.
Is read out via the memory control unit 113. Using the data read in this manner, step S18 in FIG.
Similarly, the dark current correction processing is performed on the image data for one frame.

Here, the switch 141 is switched so that data is output to the screen matching unit 116, and the image data corrected for dark current is input to the screen sequential matching unit 116. In step S33, the same screen matching as in step S20 in FIG. 7 is performed using the correction value generated by the microcomputer PRS105, and when the matching process is completed for one line of image data, HC
The order of the image data of the right screen transferred in the direction A by the CD4R is reversed (that is, the data order is changed to the order of the direction B), and the image data of the left screen is combined with the image data of the left screen to form one line of image data. Output to the unit 117.

Note that the order of the image data on the right screen may be converted by reversing the order of reading from the storage unit 103 by the memory control unit 113 when performing the dark current correction in step S32.

The shading correction processing (Step S34) and the dust / scratch correction processing (Step S3) to be performed thereafter
Step 5) is the same as the operation in steps S21 and S22 in FIG. 7, respectively, and thus the description is omitted. The above steps S32 to S35 are repeated L times (that is, the number of horizontal lines) (step S36), and the original image processing for one frame is completed.

As described above, according to the second embodiment, when a correction value for performing screen matching is obtained, a correction value is obtained by selecting only a necessary portion from image data for one frame. Thus, with a simple system configuration of the memory control process, it is possible to perform a highly accurate screen matching process as in the first embodiment.

<Third Embodiment> Hereinafter, a third embodiment of the present invention will be described with reference to the drawings.

In the first and second embodiments, the dark current correction is always performed on the image data as a pre-process for extracting the screen correlation. On the other hand, in the third embodiment, a case will be described in which it is selectively switched whether dark current correction, which is preprocessing for screen correlation extraction, is performed or not.

FIG. 10 is a diagram showing a schematic configuration of an imaging apparatus according to the third embodiment. The difference between FIG. 1 and FIG. 1 is that a switch unit 142 is added. However, since the functions of the other components are the same as those shown in FIG. I do.

Next, referring to FIG.
The operation of "0" will be described. The same processes as those shown in FIG.

In step S40, it is determined whether dark current correction is to be performed. When performing dark current processing, the switch 141 is set to 1 and data is stored in the dark current correction unit 1.
144, and the process proceeds to step S18. If not performed, the switch 141 is set to 0 and the switch 142
Is set to 1 to bypass the dark current correction unit 114 and directly output image data to the screen correlation extraction unit 112,
Proceed to step S19.

An example of the operation for determining whether or not to execute the dark current correction in step S40 will be described with reference to the flowchart in FIG.

In step S41, it is determined whether the operation mode of the camera is continuous shooting or single shooting. If continuous shooting, it is determined that dark current correction processing is to be performed (step S44). If the camera operation mode is single shooting, in step S42, the imaging unit 101
It is determined whether the charge accumulation time is longer or shorter than a predetermined time T, and if it is longer, it is determined that the dark current correction process is to be performed (step S44). On the other hand, when it is short,
In step S43, the current temperature of the imaging unit 101 or the vicinity of the imaging unit 101 is measured by a thermometer (not shown), and the result is compared with a predetermined temperature Temp to determine whether the temperature is high or low. If it is determined that the dark current correction processing is to be performed (step S44), it is determined that the correction is not to be performed if it is low (step S45).

When the accumulation time is short, or when the temperature of the imaging unit is low and the difference between the individual dark current outputs of the plurality of photoelectric conversion elements is relatively small, the processing time and the correction time are compared with the correction effect of the dark current correction. The effect of negative effects such as current consumption increases. In particular, when the camera is operated in the continuous shooting mode, the shooting interval increases. Therefore, optimal processing can be performed by selectively performing or not performing the dark current correction processing on the image data on which the screen correlation extraction is performed in accordance with the above factors.

[0111]

[Other Embodiments] In the first to third embodiments, a case has been described in which the correction is performed so that the level of the image data of the left screen is matched with the image data of the right screen in FIG. The level of the image data of the right screen may be adjusted to the data, or the image data of the left and right screens may be adjusted to match the intermediate level. These operations can be easily performed by changing the correction value based on the correlation value obtained by the screen correlation extraction unit 112.

The present invention is not limited to the configuration described in the above embodiment. For example, the input signal does not have to be an RGB color component, a luminance component, or a color difference component. It does not need to be a CCD image sensor. In addition, even if one captured screen may be an image block larger than two, and the method of screen matching is different from that described in the above embodiment such as statistical processing, the present invention It goes without saying that a configuration that can achieve the function is sufficient.

Further, the present invention is not limited to a digital camera, but also processes an image signal obtained by image pickup by a two-dimensionally arranged photoelectric conversion element such as an optical device other than a camera or another device, and performs final processing. For example, any apparatus that outputs an image to an output device such as a monitor or a printer can be applied.

In the above embodiment, the image signal obtained by the area sensor is output to a plurality of output systems (HCCD,
Although the case of separately reading out each predetermined area via an amplifier or the like has been described, the present invention is not limited to this. For example, a plurality of images may be shot by shifting the shooting area, or a plurality of area sensors may be read. In the case where a plurality of images are photographed by using and the obtained plurality of images are combined to obtain an image of a wider subject area, an image signal output for each predetermined area is combined to form one image. It can be applied to various cases of acquisition.

Further, even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a scanner, a camera head, etc.), a device including one device (for example, a digital camera, a video camera, Camera).

Further, an object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. Or CPU and MPU) read and execute the program code stored in the storage medium,
It goes without saying that this is achieved. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
In addition, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program code,
The operating system (OS) running on the computer performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included. Here, as a storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, a ROM,
RAM, magnetic tape, nonvolatile memory card, CD-
ROMs, CD-Rs, DVDs, optical disks, magneto-optical disks, MOs, and the like are conceivable.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the program code is read based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

When the present invention is applied to the above-mentioned storage medium, the storage medium includes the above-described FIG. 7, FIG. 9 or FIG.
The program code corresponding to the flowchart shown in FIG.

[0119]

As described above, according to the present invention, the density deviation occurring at the boundary surface in the image data composed of a plurality of blocks is corrected, and a natural image without a sense of incongruity is obtained by the minimum necessary functional configuration. Can be realized.

Further, it is possible to cope with an increase in the number of pixels of the image sensor.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating a configuration of an imaging device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a concept of output data from an image processing unit according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a configuration of an imaging unit illustrated in FIG. 1;

FIG. 4 is a diagram illustrating a state in which the user looks through a finder of the imaging apparatus according to the first embodiment of the present invention.

FIG. 5 is a diagram illustrating a light receiving unit of an imaging device included in the imaging unit of FIG.

FIG. 6 is a diagram illustrating a correction process according to the first embodiment of the present invention.

FIG. 7 is a flowchart illustrating a processing procedure of an original image processing unit according to the first embodiment of the present invention.

FIG. 8 is a schematic block diagram illustrating a configuration of an imaging device according to a second embodiment of the present invention.

FIG. 9 is a flowchart illustrating a processing procedure of an original image processing unit according to the second embodiment of the present invention.

FIG. 10 is a schematic block diagram illustrating a configuration of an imaging device according to a third embodiment of the present invention.

FIG. 11 is a flowchart illustrating a processing procedure of an original image processing unit according to the third embodiment of the present invention.

FIG. 12 is a flowchart illustrating an operation of determining whether or not to execute dark current correction according to the third embodiment of the present invention.

FIG. 13 is a diagram illustrating a configuration example of a conventional image sensor.

FIG. 14 is a diagram illustrating a configuration example of another conventional imaging element.

[Explanation of symbols]

 101 imaging unit 103 storage unit 104 image processing unit 105 microcomputer PRS 110, 110 ', 110 "original image processing unit 111 OB clamp unit 112 screen correlation extraction unit 113 memory control unit 114 dark current correction unit 116 screen matching unit 117 shading correction unit 118 Scratch correction unit 120 Display processing unit 121 Display memory 122 Display unit 125 JPEG compression unit 130 Storage device 141 Switch

Claims (31)

[Claims] 1. An image pickup apparatus comprising: a plurality of photoelectric conversion elements; and a pixel signal obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements is input from an imaging unit for outputting a plurality of areas for each of a plurality of regions of a subject image. An image processing apparatus for processing a pixel signal, comprising: a first correction unit configured to correct a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements; and a pixel signal corrected by the first correction unit. A correlation value acquisition unit that acquires a correlation value between pixel signals output for each of the plurality of regions, based on the correlation value acquired by the correlation value acquisition unit,
An image processing apparatus comprising: a second correction unit configured to correct the pixel signal corrected by the first correction unit. 2. The image processing apparatus according to claim 1, wherein the first correction unit performs a dark current correction process. 3. The method according to claim 2, wherein the second correction unit corrects a level of a pixel signal output from another area so as to match a level of the pixel signal output from one of the plurality of areas. The image processing apparatus according to claim 1, wherein: 4. The apparatus according to claim 1, wherein the second correction unit corrects the level of the pixel signal so that the level of the pixel signal becomes an intermediate value between the levels of the pixel signals output for each of the plurality of regions. 3. The image processing device according to 1 or 2. 5. The correlation value acquiring unit acquires a correlation value based on a pixel signal near a boundary between the plurality of regions among pixel signals obtained by the photoelectric conversion element. An image processing apparatus according to any one of claims 1 to 4. 6. A storage unit, wherein the first correction unit corrects a pixel signal for one frame output from the imaging unit, and stores the corrected pixel signal in the storage unit and the correlation value. 2. The control device according to claim 1, further comprising a control unit that controls the correction unit to output the corrected pixel signal stored in the storage unit to the acquired pixel signal by the second correction unit. The image processing apparatus according to any one of claims 1 to 5, 7. A storage means for storing at least one frame of pixel signals output from the image pickup means, and the plurality of pixel signals of one frame stored in the storage means are provided by the first correction means. The pixel signal near the boundary between the regions is corrected, the corrected pixel signal is output to the correlation value acquisition unit, and the pixel signal for one frame output from the imaging unit by the first correction unit is output. 6. The image processing apparatus according to claim 1, further comprising control means for performing correction and performing control so that the corrected pixel signal is corrected by the second correction means. apparatus. 8. The apparatus according to claim 1, further comprising: a switching unit configured to switch whether or not the first correction unit is used. When the switching unit switches the first correction unit to non-use, the correlation value acquisition unit and the correlation value acquisition unit are connected to each other. The image processing apparatus according to claim 1, wherein the second correction unit performs processing using a pixel signal input from the imaging unit. 9. The apparatus according to claim 8, wherein the switching unit switches whether to use the first correction unit based on at least one of a camera mode and an ambient temperature at the time of shooting. Image processing device. 10. A shading correction unit and a flaw correction unit, wherein the shading correction unit and the flaw correction unit perform processing on the electric signal corrected by the second correction unit. The image processing apparatus according to claim 1, wherein: 11. An imaging unit having a plurality of photoelectric conversion elements, and outputting a pixel signal obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements for each of a plurality of regions of a subject image. An imaging device comprising: the image processing device according to any one of Items 1 to 10. 12. The image pickup apparatus according to claim 11, wherein the image pickup means outputs the signal through a plurality of output systems respectively corresponding to a plurality of predetermined regions of the photoelectric conversion element. 13. A plurality of photoelectric conversion elements, and a pixel signal obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements is input from an imaging unit that outputs a plurality of regions of a subject image for each of a plurality of regions. An image processing method for processing a pixel signal, comprising: a first correction step of correcting a difference between pixel signals caused by characteristics of the plurality of photoelectric conversion elements; and a pixel signal corrected in the first correction step. On the basis of the,
A correlation value acquisition step of acquiring a correlation value between pixel signals output for each of the plurality of regions, based on the correlation value acquired by the correlation value acquisition step,
A second correction step of correcting the pixel signal corrected in the first correction step. 14. The image processing method according to claim 13, wherein a dark current correction process is performed in the first correction step. 15. In the second correction step, the level of a pixel signal output from another area is corrected so as to match the level of a pixel signal output from one of the plurality of areas. A feature according to claim 13 or claim 14.
The image processing method according to 1. 16. The method according to claim 16, wherein in the second correcting step, the level of the pixel signal is corrected so that the level of the pixel signal becomes an intermediate value between the levels of the pixel signals output for each of the plurality of regions. 15. The image processing method according to 13 or 14. 17. The correlation value acquiring step, wherein a correlation value is acquired based on a pixel signal near a boundary between the plurality of regions among pixel signals obtained by the photoelectric conversion element. 17. The image processing method according to any one of 13 to 16. 18. The image processing method according to claim 13, wherein in the first correction step, a pixel signal for one frame output from the imaging unit is corrected. 19. The method according to claim 19, further comprising: a switching step of switching whether or not to perform the first correction step. When the first correction step is not performed due to the switching of the switching step, the correlation value acquisition step and the 19. The image processing method according to claim 13, wherein in the second correction step, processing is performed using a pixel signal input from the imaging unit. 20. The apparatus according to claim 19, wherein in the switching step, whether or not to perform the first correction step is switched based on at least one of a camera mode and an ambient temperature at the time of shooting. Image processing method. 21. A shading correction step and a flaw correction step, wherein the shading correction step and the flaw correction step include:
21. The image processing method according to claim 13, wherein processing is performed on the electric signal after the correction in the second correction step. 22. A pixel signal which has a plurality of photoelectric conversion elements and is obtained by photoelectrically converting a subject optical image by the plurality of photoelectric conversion elements is input from an image pickup means which outputs each of a plurality of regions of a subject image. An image processing method for processing a pixel signal, comprising: a storage step of storing at least one frame of a pixel signal output from the imaging unit; and a pixel signal of one frame stored in the storage step, A first correction unit that corrects a difference between pixel signals due to characteristics of the plurality of photoelectric conversion elements for a pixel signal near a boundary between the plurality of regions by a first correction unit; and the first correction step. Based on the pixel signal corrected in
A correlation value obtaining step of obtaining a correlation value between pixel signals output for each of the plurality of regions; and a pixel signal for one frame stored in the storage step, which is caused by characteristics of the plurality of photoelectric conversion elements. A second correction step of correcting the difference between the pixel signals, based on the correlation value obtained in the correlation value obtaining step,
A third correction step of correcting the pixel signal corrected in the second correction step. 23. The image processing method according to claim 22, wherein in the first and second correction steps, a dark current correction process is performed. 24. In the third correction step, the level of a pixel signal output from another area is corrected so as to match the level of a pixel signal output from one of the plurality of areas. 24. The method according to claim 22, wherein
The image processing method according to 1. 25. The third correction step, wherein the level of the pixel signal is corrected so as to have an intermediate value between the levels of the pixel signals output for each of the plurality of regions. 24. The image processing method according to 22 or 23. 26. The apparatus according to claim 26, further comprising: a switching step of switching whether or not to perform the first and second correction steps. When the switching of the switching step does not perform the first and second correction steps, 26. The image processing method according to claim 22, wherein in the correlation value obtaining step and the third correction step, processing is performed using the pixel signals stored in the storage step. 27. The method according to claim 27, wherein, in the switching step, whether to perform the first and second correction steps is switched based on at least one of a camera mode and an ambient temperature at the time of shooting. 27. The image processing method according to 26. 28. The method further comprising: a shading correction step; and a flaw correction step, wherein the shading correction step and the flaw correction step include:
28. The image processing method according to claim 22, wherein processing is performed on the electrical signal after the correction in the third correction step. 29. A program executable by the information processing device, the information processing device executing the program being:
A non-transitory computer-readable storage medium storing a program that functions as the image processing apparatus according to claim 1. 30. A program executable by an information processing apparatus having a program code for realizing the image processing method according to claim 13. 31. A storage medium storing the program according to claim 29.