JPH1155682A - Digital camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent production of a false color in a paste processing of plural partial images. SOLUTION: A subject is photographed while being divided horizontally in a way that a border is duplicated by two single board color CCDs 12, 13. Both the photographed images are stored in an image memory 193 via an analog signal processing circuit 191 and an A/D converter section 192 for each color component. A shading correction section 194 corrects dispersion in the sensitivity distribution in a photographed image and an image data interpolation section 195 supplements image data at a pixel position of deficient data through interpolation processing and an image synthesis section 196 pastes borders to generate the photographed image of the entire object. The interpolation processing of partial image data of each color component is conducted before paste processing of the partial image to prevent production of a false color resulting from a sensitivity difference of the CCDs 12, 13 at synthesis processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention divides a subject into a plurality of blocks, captures each block with a plurality of image sensors, and combines the captured images by image processing to generate a captured image of the entire subject. It concerns digital cameras.

[0002]

2. Description of the Related Art Conventionally, an image pickup apparatus such as a digital video camera or a digital still camera using a solid-state image pickup device such as a CCD is disclosed in, for example, JP-A-5-137059 and JP-A-6-141246. Then, the subject is partially imaged using a plurality of CCDs,
There has been known a method of increasing the resolution by synthesizing those partial images in image processing to obtain a captured image of the entire subject.

[0003] Japanese Patent Application Laid-Open No. 3-79991 discloses that
An image in which a plurality of image sensors are provided to divide and read image information, an image pickup range of an adjacent image sensor is overlapped at a boundary portion, and images captured by both image sensors are stuck at the boundary portion to obtain an image of the entire image information. In the reading device, a pixel position having a small spatial density change between adjacent pixels is detected from the image data to be formed, and an image captured by one image sensor and an image captured by the other image sensor are detected at the pixel position. Is shown in the figure.

[0004]

By the way, in a digital camera in which the pixel density of an image pickup screen is increased by using a plurality of image pickup devices, when a single-plate type color image pickup device is used as the image pickup device, the color image pickup device is not used. The image data constituting the output image of each color component is composed of pixel data at different pixel positions, and depending on the type of the color image sensor, the pixel density of the color components is also different. Cathode RayTube) or LC
In order to display the image data on a display device such as a D (Liquid Crystal Display) or to output the image data using a printer, an interpolation process for supplementing image data at a pixel position where image data is insufficient for each color component is required.

In this case, as an image processing procedure of a captured image, a plurality of captured images (partial captured images) are attached to each color component to generate a captured image of the entire subject, and interpolation of image data is performed on the captured image. When the processing is performed, the image bonding process can be sped up, but a false color may be generated in a bonded portion of the image after bonding as described below.

For example, in a configuration in which the photographing screen is divided into two parts, the left and right objects are each imaged by two single-plate color CCDs, the boundary part of the divided objects has achromatic uniform luminance. , When there is a sensitivity difference between the two CCDs (when the sensitivity of the left CCD is higher than the sensitivity of the right CCD)
Will be described as an example. R at the boundary of the captured image on the left
(Red), G (green), and B (blue) signal levels are denoted by R1, G1, and B1, and R1 at the boundary of the right-side captured image
The signal levels of the (red), G (green), and B (blue) color components are R2, G2, and B2, and R1 = G1 = B1 = D1, R2
= G2 = B2 = D2, and D2 <D1.

[0007] In order to make the density step at the boundary of the bonded image inconspicuous, the image data constituting the boundary between the left and right picked-up images is alternately mixed and arranged at each pixel position to form the boundary for bonding. If an image is generated and the image of the boundary portion and the image excluding the boundary portion of the left and right captured images are combined to generate a captured image of the entire subject, the image of the boundary portion for bonding is a signal Since image data forming two boundary portions having different levels are generated by alternately mixing and arranging them at respective pixel positions, for example, when the color CCD is a single-panel color CCD of the Bayer type, image data of the G color component Is D1, while the signal level of the image data of the R and B color components is D2 (<D1), and a false color is generated at the border portion which was originally achromatic.

The above-mentioned publication discloses a technique in which a subject is divided into a plurality of blocks by a plurality of image pickup elements and a part of the image is captured, and these partial images are attached to obtain a captured image of the entire subject. However, there is no disclosure of the above-described problem in the case where a plurality of image pickup devices are formed of a single-plate type color image pickup device and a method of solving the problem.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and provides a digital camera in which a false color does not occur due to a bonding process and the image quality of a captured image is not deteriorated.

[0010]

According to the present invention, a subject light image is divided into a plurality of partial light images such that boundaries are overlapped with each other, and each partial light image is picked up by a plurality of color image pickup means. In a digital camera that generates a captured image of the entire subject light image by bonding captured images at the boundary portions for each color component, the color imaging unit includes three primary color filters arranged in a predetermined positional relationship on an imaging surface. Image data interpolating means for supplementing image data at a pixel position where a captured image of each color component is insufficient by interpolation processing using image data output from the color image capturing means; An image processing control means for performing a bonding process of the captured images after the interpolation process of the captured images of the components (claim 1).

According to the above arrangement, the subject light image is divided into a plurality of partial light images such that the boundary portions overlap, and each of the partial light images is separated into three color components by the single-plate color imaging means. And are photoelectrically converted and captured. A captured image of each color component is output from a color imaging unit.
Using the image data of the color component of the color, the image data at the insufficient pixel position is supplemented by the interpolation processing. Then, the plurality of partial images after the interpolation processing are pasted together at the boundary portion to generate a captured image of the entire subject light image. That is, of the plurality of partial images, using the image forming the boundary portion, an image of the boundary portion for synthesis in which the density difference between the two partial images sandwiching the boundary is reduced for each color component is generated. An image of the entire subject is generated by synthesizing (combining) the image of the synthesizing boundary portion and the image excluding the boundary portion among the plurality of partial images.

[0012]

FIG. 1 is a perspective view showing an appearance of a digital camera according to an embodiment of the present invention, and FIG. 2 shows a schematic configuration of an image pickup optical system provided in the digital camera. FIG.

In FIG. 1, a digital camera 1 is provided with a photographing lens 2 composed of a zoom lens substantially at the center of the front surface, and a light projecting window 4 for measuring a subject distance by an active distance measuring method, and a light receiving window above the photographing lens 2. A window 5 is provided, and a photometric window 3 for measuring the brightness of the subject is provided between the two windows. Further, a finder objective window 6 is provided on the left side of the light projecting window 4.

The light projecting window 4 is a window for irradiating the subject with infrared light, and the light receiving window 5 is a window for receiving the reflected light of the infrared light from the subject. In the present embodiment, the active distance measuring method is adopted as the distance measuring method, but a passive distance measuring method may be used.

On the side of the digital camera 1, there is provided a card insertion slot 7 into which a hard disk card 10 (hereinafter, abbreviated as HD card 10) is inserted and removed, and the HD card 10 is ejected above the card insertion slot 7. A card removal button 9 for performing the operation is provided. A shutter button 9 is provided at the left end of the upper surface of the digital camera 1.

When printing out the photographed result, the card removal button 9 is pressed to remove the HD card 10 from the digital camera 1, and the HD card 10 can be mounted on a printer to which the HD card 10 can be mounted and printed out. .

The digital camera 1 is provided with a SCSI cable interface, and the digital camera 1 and the printer are connected by a SCSI cable.
, The image data may be transferred to a printer and the photographed image may be printed out directly.

In this embodiment, a hard disk card conforming to PCMCIA is adopted as a recording medium for image data. However, if a photographing result can be stored as image data, a memory card or a mini disk (MD) is used.
And other recording media.

As shown in FIG. 2, at the position behind the taking lens 2 of the camera body, an optical path separating section 11 composed of, for example, a prism and a color image pickup element 12, 13 composed of a pair of CCD color area sensors. And an imaging optical system comprising:

As shown in FIG. 3, the image pickup optical system has a horizontally long rectangular photographing screen. The subject light image A in this photographing screen has a right end portion of the left half light image A1 and a right end portion. The optical path separating unit 11 divides the light image A1 into two in the horizontal direction such that the boundary C is included in the left end of the light image A2 of the half.
Is captured by the CCD 12, and the right half light image A2 is
3 is taken. And both CCD1
The image of the boundary portion C of the images captured by the images 2 and 13 is combined by image processing to obtain a captured image of the subject light image A (that is, to capture the entire subject in the shooting screen).

Therefore, as shown in FIG.
If the number of pixels constituting the imaging surface (horizontal imaging surface) S with respect to is defined as M (vertical) × (2N−W) (horizontal) and the number of pixels in the horizontal direction of the boundary portion C is defined as W, each of the CCDs 12 and 13 The number of pixels constituting the imaging surfaces S1 and S2 is M (vertical) × N (horizontal).

The CCDs 12 and 13 are single-panel color CCDs of the Bayer type, and R, G, and B color filters having the color arrangement shown in FIG. 5 are provided at each pixel position on the imaging surface. The color arrangement shown in FIG. 5 is an arrangement in which R (red), B (blue) and G (green) are arranged in a checkered pattern, and the pixel position is (i row (i = 1, 2,... M ), J columns (j = 1,2, ...
N)), the intersection of odd rows / even columns (2ζ +
1,2ξ + 2) and the intersection position of even rows / odd columns (2ζ +
2,2ξ + 1) (ζ = 0,1,2, ... M / 2, ξ = 0,1,2, ... N /
2), a G color filter is arranged, and an R color filter is arranged at an odd row / odd column intersection position (2ζ + 1, 2 + 1), and an even row / even column intersection position (2ζ + 2, 2ξ).
At +2), B color filters are arranged.

Further, as shown in FIGS. 6 and 7, a micro lens 14 composed of a convex lens is provided at each pixel position. The microlenses 14 condense a light beam in order to increase the light receiving sensitivity of each pixel.

FIG. 8 is a block diagram of a digital camera according to the present invention. In the figure, the same members as those shown in FIGS. 1 and 2 are denoted by the same reference numerals. Also,
The diaphragm 15 is a light amount adjusting member provided in the photographing lens 2. The lens drive control section 16 drives a focusing lens (not shown) in the photographing lens 2 to perform automatic focus adjustment. The focusing lens is provided, for example, at the front end of the photographing lens 2 so as to be movable in the optical axis direction, and is moved by a driving force of an electric motor (not shown). The lens drive control unit 16 controls the drive of the electric motor based on the AF control value input from the control unit 25 that centrally controls the shooting operation of the camera, and automatically adjusts the focus of the shooting lens 2.

The aperture drive controller 17 adjusts the aperture of the aperture 15. The aperture drive controller 17 controls the aperture of the aperture 15 based on the exposure control value (aperture value) input from the controller 25. The CCD drive control unit 18
It controls the imaging operation of the CDs 12 and 13 (charge storage and reading of stored charges). The CCD drive control unit 18
The exposure amount is controlled by controlling the charge accumulation time (integration time) of the CCDs 12, 13 based on the exposure control value (shutter speed) input from the control unit 25. CCD12,
When the exposure (charge accumulation) of 13 is completed, the accumulated charges are read out to the image processing unit 19 in the order of the CCD 12 and the CCD 13.

The image processing section 19 performs predetermined processing such as white balance, γ correction, shading correction and the like on the accumulated charges (image signals) read from the CCDs 12 and 13, as well as an image picked up by the CCD 12 (left half). Of the subject light image A1 (hereinafter referred to as left image G1) and an image captured by the CCD 13 (captured image of the right half subject image A2; hereinafter referred to as right image G2). (A process of generating a captured image for the entire light image A). Processing such as shading correction and generation of a captured image by the image processing unit 19 will be described later.

The HD card 10 is a recording medium for recording image data constituting an image. In the HD card 10,
A captured image (hereinafter, referred to as a captured image G) of the entire subject obtained by laminating the left image G1 and the right image G2 is recorded. The card drive control unit 20 controls the drive of the HD card 10 to record image data.

The distance measuring unit 21 detects a subject distance provided at a position behind the light projecting window 4 and the light receiving window 5.
The photometric unit 22 includes an SPC provided at a position behind the photometric window 3.
And the like to detect the subject brightness.

A ROM (Read Only Memory) 23 stores data necessary for driving control of the image pickup system and data necessary for performing image processing such as shading correction, image pasting processing, and image data interpolation processing described later. This is a memory in which a processing program is recorded. RAM (Random Access Memor
y) 24 is a memory for the control unit 25 to perform processing related to imaging.

FIG. 9 is a diagram showing a block configuration relating to processing such as shading correction in the image processing section 19, generation of a photographed image, interpolation of image data, and the like.

The image processing unit 19 includes analog signal processing units 191 and A / A as processing units for performing processes such as shading correction, generation of photographed images, and interpolation of image data.
D conversion section 192, image memory 193, shading correction section 194, image data interpolation section 195, image synthesis section 19
6 and an output I / F (interface) 197 are provided.

The analog signal processor 191 includes a CD (not shown).
It has a signal processing circuit such as an S circuit (Correlative Double Sampling) circuit and an analog amplifier, and performs noise reduction and level adjustment (amplification) on an image signal composed of analog signals of R, G, and B color components output from the CCDs 12 and 13. ) And the like. A / D converter 1
92 converts the image signal output from the analog signal processing unit 191 into image data composed of, for example, an 8-bit digital signal. The image memory 19
Reference numeral 3 stores image data output from the A / D conversion unit 192. The image memory 193 includes the CCD 12
And a storage area for storing image data of an image captured by the CCD 13 and image data of an image captured by the CCD 13, respectively. Note that image data is stored in each storage area for each of R, G, and B color components.

The shading correction section 194 includes the CCD 1
This is for correcting variations in output levels between pixels constituting the imaging surfaces 2 and 13. The output level variation is
This is performed by correcting the light reception level output from each pixel using a preset shading correction table. This shading correction table is set in advance using the pupil position of the photographing lens 2 and the aperture amount of the aperture 15 as parameters, and is stored in the ROM 23. At the time of photographing, a predetermined shading correction table corresponding to the detected pupil position of the photographing lens 2 and the aperture amount of the diaphragm 15 is set in the shading correction unit 194, and the images of the captured images of the CCDs 12 and 13 stored in the image memory 193. The data is level-adjusted in this shading correction table.

Here, the shading correction table will be briefly described. The shading correction is for correcting a variation in output level between pixels that occurs when a light image having a uniform density is uniformly projected on an imaging surface of a CCD by a photographing lens.

The variation in the output level between pixels mainly occurs due to the variation in the sensitivity characteristic of the pixel itself and the variation in the amount of light incident on each pixel. Variations in the sensitivity characteristics of the pixels themselves include not only those based on the difference in photoelectric conversion rate between pixels, but also those based on differences in the characteristics of the microlenses 14 provided on the light receiving surface of each pixel.
That is, as shown in FIGS. 11 to 13, variations in light-collecting characteristics based on the shape of the microlenses 14 and variations in optical characteristics based on the composition of the microlenses 14 such as the refractive index and the transmittance are caused. Output level variation occurs. FIG. 11 shows a case where the thickness of the microlens 14 is thinner than the standard one (see FIG. 7).
Is a case where the thickness of the microlens 14 is larger than the standard one. FIG. 13 shows a case where the position of the micro lens 14 on the light receiving surface of the pixel is shifted from the center position.

The variation in the amount of light incident on each pixel is shown in FIG.
As shown in (2), when the light beam 24 is incident on the light receiving surface of the pixel from an oblique direction (angle θ) with respect to the front direction n, such a variation in the amount of light based on the change in the incident direction is different from the light receiving surface of the pixel. This is caused by a change in the pupil position of the photographing lens 2 and a change in the aperture amount.

That is, taking the CCD 12 as an example,
As shown in FIG. 15, taking into consideration the incident light flux at the center portion Z and the upper and lower ends C and D on a vertical vertical plane passing through the center of the imaging surface S1 of the CCD 12, the imaging lens 2 is positioned with respect to the imaging surface S1. As the pupil position becomes longer, as shown in FIG.
D is incident from the outside with respect to the front direction of the imaging surface S1, and as the pupil position of the photographing lens 2 becomes closer to the imaging surface S1, as shown in FIG. The light beam centers 26C and 26D are incident from the inside with respect to the front direction of the imaging surface S1. When the aperture of the diaphragm 15 is reduced at the same pupil position as in FIGS. 16 and 17, the width (incident range) of the incident light beam is reduced as shown in FIGS. 18 and 19, respectively. The amount of incident light decreases. In FIGS. 16 to 19, the dotted lines indicate the range of each incident light beam.

For this reason, at the center Z of the imaging surface S1 of the CCD 12, the center 26Z of the incident light beam is incident substantially perpendicularly to the imaging surface S1, but at both ends C and D, it is inclined with respect to the imaging surface S1. 14 and the pixels in this portion are shown in FIG.
As shown in FIG. 7, the light beam 24 is incident on the light receiving surface in an inclined state. Therefore, even if there is no variation in the sensitivity characteristics of the pixel itself, even if there is no variation in the sensitivity characteristic of the pixel itself, the pixel of the central portion Z of the imaging surface S1 and both ends C , D, there will be a difference in output level. The same applies to a vertical plane in the horizontal direction passing through the center of the imaging surface S1 of the CCD 12 (a plane including A, Z, and B in FIG. 15).

Therefore, when a uniformly illuminated subject of uniform density is photographed, the output level of the image data captured by the CCD 12 is lower at the periphery of the imaging surface S1 than at the center, as shown in FIG. It takes shape. The shading correction table is for converting the waveform of the image data into a flat waveform passing through the output level Lz of the center Z indicated by the imaginary line in FIG. 20, and calculating a correction coefficient (magnification coefficient) K for each pixel. is there. For example, assuming that the output level of the pixel at point A is La, the correction coefficient K for the pixel at point A is
_A is K _A = Lz / La.

Assuming that the output level at an arbitrary pixel position (i, j) is L (i, j) and the correction coefficient is K (i, j), the shading correction table has M × N correction levels. Coefficient K (i, j) = Lz / L (i, j) (i = 1, 2,...)
M, j = 1, 2,... N). If the variation in the output level between pixels is caused only by the variation in the photoelectric conversion characteristics unique to the CCD 12 including the characteristics of the microlenses 14, it is sufficient to prepare one shading correction table, but as described above. Since the output level of the image data captured by the CCD 12 changes depending on the pupil position of the photographing lens 2 and the aperture amount of the aperture 15,
It is necessary to prepare a plurality of shading correction tables using the pupil position of the photographing lens 2 and the aperture amount of the aperture 15 as parameters.

However, if the correction coefficients K (i, j) for all the pixels are prepared corresponding to all the parameters, the capacity of the shading correction table becomes large. In this embodiment, as described below, The capacity of the shading correction table is reduced by a method such as reducing the number of correction coefficients K (i, j) itself or reducing the number of tables corresponding to the parameters.

First, in the first method, as shown in FIG. 21, the imaging surface S1 of the CCD 12 is divided into a plurality of pixels (for example, 1 pixel).
A block B (h, k) (h = 6 × 24 pixels)
1, 2,..., K = 1, 2,...) And the correction coefficient K (h, k) is set for each block to reduce the capacity of the shading correction table itself.

The second method is to reduce the number of shading correction tables using the aperture value as a parameter. That is, when the change in the output level of the CCD 12 with respect to the aperture amount of the aperture 15 can be approximated to be substantially linear, a shading correction table is prepared for two points when the aperture amount of the aperture 15 is maximized and minimized. With respect to the intermediate aperture amount, a correction coefficient K (h, k) for each block is calculated by using these shading correction tables.

For example, if correction coefficients K (h, k) _{F = 2} and K (h, k) _{F = 32} are set for the aperture values _{F = 2} and _{F = 32} , 2 <x <32 The correction coefficient K (h, k) _{F = x} for an arbitrary aperture value F = x is calculated and set by the following equation (1).

[0045]

(Equation 1)

The CCD 1 with respect to the aperture amount of the aperture 15
Even if the changes in the output levels of the outputs 2 and 13 are non-linear, the function K (h, k) _{F = x} = f (K (h, k) _{F = 2} , K (h, k) _{F = 32} ) is known. If so, the function K (h, k) _{F = x} = f (K (h, k) _{F = 2} , K
(h, k) _{F = 32} ), and an arbitrary aperture value F = 2 <x <32
A correction coefficient K (h, k) _{F = x} for _x can be calculated and set.

The third method is to reduce the number of shading correction tables using the pupil position of the photographing lens 2 as a parameter. That is, the CCD 12
In the peripheral portion of the imaging surface S1, the incident angle θ of the incident light beam changes depending on the pupil position of the photographing lens 2, and the output level also changes. However, since the pupil position and the incident angle θ change in a fixed relationship, For example, a shading correction table at the pupil position where the incident angle θ is the largest is prepared, and the shading correction tables for the other pupil positions are calculated and set by calculation using the shading correction table at the pupil position where the incident angle θ is the largest. Is what you do.

At a pupil position other than the pupil position at which the incident angle θ becomes the largest, the incident angle θ becomes smaller than the maximum incident angle θmax. Therefore, at the pupil position, the shading correction table at the pupil position at which the incident angle θ becomes the largest is obtained. Correction coefficient K
(h, k), the correction coefficient K of the portion corresponding to the incident angle θ
The correction coefficient K (h, k) ′ can be calculated and set using (h, k).

That is, as shown in FIG. 22, the relationship between the incident light beam at the pupil position where the incident angle θ of the incident light beam is smaller than the maximum incident angle θmax and the imaging surface S1 of the CCD 12 is equivalently photographed. It is conceivable that the imaging surface S1 was moved in a direction away from the photographing lens 2 with the lens 2 fixed at the pupil position where the incident angle θ was maximized. At this pupil position, the incident angle of the incident light beam becomes smaller from θ to θ ′ (<θ). Therefore, as shown in FIG. 23, of the correction coefficient K (h, k) of the shading correction table, the incident angle θ ′ The correction coefficient K (h, k) 'for the pupil position can be calculated by enlarging a part of the correction coefficient K (h, k) included in the area S1' corresponding to the pupil position. .

In FIG. 23, the point z is the image pickup surface S1.
Is the center position. The small rectangular block B (h, k) in the imaging surface S1 is a block B (h, k) in which the correction coefficient (h, k) is set by the shading correction table.
The area S1 'is a correction coefficient K of the shading correction table when the pupil position of the photographing lens 2 is changed to a position where the incident angle θ of the incident light flux is smaller than the maximum value.
This is the area where (h, k) is applied.

Next, a method of calculating the correction coefficient K (h, k) 'for a pupil position other than the pupil position at which the incident angle θ becomes the largest will be described.

FIG. 24 is a diagram showing an example of a change in the size of the block B (h, k) when the area S1 'is enlarged to the size of the imaging surface S1.

FIG. 17 shows an enlarged state of the four blocks Ba to Bd (blocks shown in FIG. 23) included in the upper left area when the imaging surface S1 is divided into four parts by the orthogonal coordinates passing through the center z. It is a thing. Further, in the same figure, blocks Ba to Bd indicated by solid lines are imaging surfaces S1.
, And the blocks Ba ′ to Bd ′ indicated by the alternate long and short dash line are enlarged sizes.

Assuming that the correction coefficients preset for the blocks Ba to Bd are Ka to Kd and the correction coefficients calculated for the change in the pupil position are Ka 'to Kd', respectively, as shown in FIG. And the block Ba is the block B
Since it is included in a ', the correction coefficient Ka for the block Ba is applied as it is, and Ka' = Ka.

On the other hand, blocks Bb to Bb other than block Ba
Since Bd is composed of a plurality of blocks after the enlargement including the enlarged block, the weighted average value of the correction coefficients set for the block using the area ratio of the constituted blocks as a weight coefficient Is calculated, the correction coefficient of the block is calculated.

For example, the block Bd is composed of four blocks Ba 'to Bd' after the enlargement, and the enlargement magnification is Q
If (> 1), the area ratio of each of the constituent blocks Ba ′ to Bd ′ is Ba ′: Bb ′: Bc ′: Bd ′ = (1−Q)
^{2: Q (1-Q)} : Q (1-Q): from the Q ^2, correction coefficient Kd 'is calculated by the following equation (2).

[0057]

(Equation 2)

In the same manner, the correction coefficients Kb 'and Kc' for the blocks Bb and Bc are calculated by the following equations (3) and (4).

Kb ′ = (1−Q) · Ka + Q · Kc (3) Kc ′ = (1−Q) · Ka + Q · Kb (4) The block B (h, k) in the peripheral portion of the imaging surface S1 ), A block B (h, k) obtained by expanding the block (h, k)
k) one or more expanded blocks B not including '
(h ′, k ′) ′ (h ′ ≠ h, k ′ ≠ k), but in this case, similarly to the above-described blocks Bb to Kd, the configured blocks after expansion By calculating a weighted average value of the correction coefficients set for the block using the area ratio of the block as a weight coefficient, the correction coefficient of the block is calculated.

Although the shading correction table has been described using the CCD 12 as an example, the same applies to the CCD 13.

Therefore, in this embodiment, four basic shading correction tables (hereinafter, referred to as (A) to (D)) corresponding to parameters such as the aperture amount and the pupil position are shown. These tables are referred to as basic correction tables).
When the pupil position and the aperture amount are different from the parameters of the basic correction table, the shading correction table of the parameters is calculated by the above-described calculation method.

(A) In the case where the aperture of the diaphragm 15 is maximized at the pupil position where the incident angle of the incident light beam from the outside with respect to the imaging surface is maximum (the case corresponding to FIG. 16). In the case where the aperture of the diaphragm 15 is maximized at the pupil position where the incident angle of the incident light beam from the inside becomes maximum (a case corresponding to FIG. 17) (C) The incident light beam from the outside with respect to the imaging surface In the case where the aperture amount of the stop 15 is minimized at the pupil position where the incident angle of the image becomes maximum (the case corresponding to FIG. 18). (D) The pupil where the incident angle of the incident light beam from the inside with respect to the imaging surface becomes the maximum. At the position, when the opening amount of the diaphragm 15 is minimized (a case corresponding to FIG. 19). Then, the microlens set in advance in the shading correction table calculated in accordance with the aperture value and the pupil position using the basic correction table CC with properties 12, 13 specific final shading correction table is multiplied by a correction table for correcting the variation of the photoelectric conversion characteristics calculated and set. Note that the correction table for correcting the variation in the photoelectric conversion characteristics is the same as the basic correction table.
It is stored in the OM23.

Referring back to FIG. 9, the image data interpolation unit 195
For each of the R, G, and B color components, image data at a pixel position where data is insufficient is supplemented by interpolation processing. CCDs 12 and 13 are Bayer type single-plate color C
Since a CD is used, the image of each of the R, G, and B color components has a pixel position where no image data exists (see FIG. 5), and the pixel densities between the color components do not match. The image data interpolation unit 195 matches the pixel density of the image of each color component of R, G, and B, and enables the captured image to be displayed on a color CRT or printed by a printer.

That is, the image data interpolation unit 195
For the color component image, the odd-numbered column image data and the image data at the intersection of the even-numbered row / even-numbered column are supplemented. For the B-color component image, the even-numbered column image data and the odd-numbered row /
The image data at the intersecting position of the odd-numbered column is interpolated. G
For the color component image, the image data at the odd row / even column intersection and the odd row / even column intersection are interpolated.

The image of each color component of R, G, and B is subjected to the above-described interpolation processing of the image data according to the processing procedure shown in FIG. 25, for example.

In FIG. 25, in the image data 27R, 27G, and 27B of each color component of R, G, and B, the symbols "R", "G", and "B" in the matrix indicate the image at the pixel position. The presence of data indicates that a blank pixel position indicates that no image data exists (being a pixel position to be interpolated). The image data 27
R ', 27G', and 27B 'indicate the interpolated image data, and the symbols "R'", "G '", and "B'" in the matrix indicate that the interpolated image data exists. Is shown. The color difference data 28R is obtained by converting the image data 27R of the R color component into color difference data using the image data 27G 'of the G color component after interpolation, and the color difference data 28R' is obtained by converting the color difference data 28R. This is obtained by performing interpolation processing using a preset interpolation filter 29. Similarly, the color difference data 28B is obtained by converting the image data 27B of the B color component into color difference data using the image data 27G 'of the G color component after interpolation, and the color difference data 28B'
Is obtained by converting the color difference data 28B into a preset interpolation filter 3
This is the result of interpolation processing using 0. The symbols "Cr" and "Cb" in the matrix indicate that color difference data exists, and the symbols "Cr '" and "Cb'" include interpolated color difference data. It is shown that.

For the image data 27G of the G color component, pixel positions (2 補 間 +1, 2 （+1), (2
The image data of {+2, 2} +2) is interpolated by the average value of the two image data excluding the maximum value and the minimum value among the four image data at the pixel positions adjacent to the pixel position.
For example, the image data G (2,2) at the pixel position (2,2) is the adjacent image data G (1,2), G (2,3), G (3,2), G (2,1)
If G (1,2) and G (2,3) are the maximum value or the minimum value, it is calculated by (G (3,2) + G (2,1)) / 2. In a peripheral pixel position where there are two adjacent image data, such as the pixel position (1, 1), the average value of the two image data is used, and as in the pixel position (3, 1), At a pixel position where there are three adjacent image data, the image data at that pixel position is interpolated by the average value of the two image data excluding the maximum value or the minimum value.

The image data 27 of the R and B color components
As for R and 27B, the R and G color data are converted into color difference data Cr and Cb using the image data of the G color component image data 27G 'after interpolation.
Interpolation processing is performed by predetermined interpolation filters 29 and 30 preset for R and 28B. Then, the interpolated color difference data 28R 'and 28B' are re-converted into R and G color data by using the interpolated G color component image data 27G 'image data, and the interpolated R and B color data. Component image data 27R 'and 27B' are obtained.

For example, in the interpolation processing of the pixel position (2, 2) of the image data 27R of the R color component, first, the pixel position (1, 2)
1), (1,3), (3,1), (3,3)
The image data R (1,1), R (1,3), R (3,
The image data G (1, 3) of the G color component after interpolation from R (3, 3).
1) ', G (1,3)', G (3,1) ', G (3,3)' are subtracted to obtain color difference data Cr (1,1), Cr (1,3), Cr (3 , 1), Cr (3,3)
Is generated. Subsequently, these color difference data Cr (1,
1), Cr (1,3), Cr (3,1), and Cr (3,3) are subjected to a filtering process by the interpolation filter 29, so that the color difference data Cr (2,2) at the pixel position (2,2) is obtained. Is replenished. In the interpolation filter 29 according to the present embodiment, Cr
Since the filter coefficient for (1,1), Cr (1,3), Cr (3,1), Cr (3,3) is “1”, the color difference data Cr (2,2)
Is Cr (2,2) = Cr (1,1) + Cr (1,3) + Cr (3,1) + C
r (3,3). Subsequently, the interpolated color difference data Cr (2,2)
Is added to the image data G (2,2) 'of the G color component after interpolation to generate image data R (2,2)' composed of R color data.

The image data 27R ', 27 of each of the R, G, and B color components subjected to the interpolation processing in the image data interpolation unit 195.
G 'and 27B' are input to the image synthesizing unit 196 and stored in a memory (not shown).

In this embodiment, an image synthesizing section 196 is arranged at the subsequent stage of the image data interpolating section 195, and R, G,
After the interpolation processing of the image data of each color component of B is performed, the bonding processing of the left image G1 and the right image G2 is performed. As shown in FIG.
And the arrangement of the image synthesizing unit 196 are exchanged.
The image data interpolation unit 195 is arranged at the subsequent stage, and the left image G
1 and the right side image G2, then R, G, B
The interpolation processing of the image data of each color component may be performed.

The method of performing the interpolation processing of the image data before the image synthesizing processing has an advantage that occurrence of a false color in the bonding processing can be prevented. That is, for example, when the boundary portion C has achromatic uniform luminance, the CCD 12 and the CC
The sensitivity is different from D13, for example, the sensitivity ratio is 100:
If it is 93, R,
The output levels Lr1, Lr of the image at the boundary between the G and B color components
Assuming that Lg1 and Lb1 are Lr1 = Lg1 = Lb1 = 255, the output levels Lr2, Lg2, and Lb2 of the image of the boundary between the R, G, and B color components captured by the CCD 13 are Lr2.
= Lg2 = Lb2 = 237 (≒ 255 × 0.93).
In this case, if image synthesis processing is performed by, for example, random mixing of simple image data before the interpolation processing of the image data, G image data with an output level of “255” and R, B with an output level of “237” Is mixed with the image data of
A false color will occur in the generated boundary image Gc.
On the other hand, if the interpolation processing of the image data is performed before the image synthesis processing, the R, R having the output level “255”
G, B image data and R, whose output level is “237”
The image data of G and B are mixed at random, and generation of a false color can be suppressed.

On the other hand, the method of performing the image synthesizing process before the interpolation process of the image data has an advantage that the processing time until the captured image G of the entire subject is generated can be shortened.

Returning to FIG. 9, the image synthesizing section 196 is a CCD
The left image G1 picked up at 12 and the right image G2 picked up by the CCD 13 are bonded together at the boundary C to generate a picked-up image G of the entire subject. The output I /
F197 is a captured image G generated by the image combining unit 196.
Is an interface for outputting the image data constituting the image data to the HD card 10.

The image synthesizing section 196 forms an image of the boundary portion C in the left image G1 (hereinafter referred to as left boundary image Gc1) and an image of the boundary portion C in the right image G2 (hereinafter referred to as right boundary image Gc2). The image of the boundary portion C for synthesis such that the density continuously changes from the left image G1 to the right image G2 (that is, an image in which the density change of the bonded portion is inconspicuous. Hereinafter, the boundary image Gc) .)
The captured image G is generated by synthesizing the boundary image Gc and the left image G1 and the right image G2 except for the boundary images Gc1 and Gc2. The synthesis processing of the captured image G is performed for each of the R, G, and B color components. Then, the image data constituting the captured image G of the entire subject generated by the image synthesizing unit 195 is transferred to the HD card 10 via the output I / F 197 and recorded.

Various methods are conceivable as a method of synthesizing the left boundary image Gc1 and the right boundary image Gc2 so that the density change of the boundary image Gc is not conspicuous. Here, a method of synthesizing the boundary image Gc for synthesis for generating the captured image G using the right boundary image Gc1 and the right boundary image Gc2 will be described.

FIG. 26 is a diagram showing a first embodiment of the boundary image generation method. The boundary image generation method shown in FIG.
The left boundary image Gc1 and the right boundary image Gc2 are simply pasted together such that the boundary line 31 is zigzag in the vertical direction. FIG. 7A shows the boundary line 31 in a sawtooth shape, and FIG. 7B shows the boundary line 31 in a sine wave shape. The more the zigzag shape in the vertical direction of the boundary line 31 changes, the more the sharp change in density is alleviated. Therefore, it is preferable that the zigzag shape has no regularity and has a large change. Note that the vertical zigzag shape of the boundary line 31 is not limited to the example shown in FIG. 26, and various shapes other than a linear shape can be adopted.

When this method is adopted, the image synthesizing unit 1
Reference numeral 96 denotes an image constituting the left boundary image Gc1 and the right boundary image Gc2 stored in the memory provided in the image synthesizing unit 196 such that a vertical zigzag shape of the preset boundary line 31 is generated. Part of the image data up to a predetermined bonding position is read out from the data for each line, and these are simply connected to generate a boundary image Gc.

Now, assuming that the number of pixels in the horizontal direction of the boundary image Gc is m, the address p of the pixel position in the horizontal direction of the boundary images Gc1 and Gc2 is p = p from the left end to the right end as shown in FIG.
1, 2,... M. Assuming that the bonding position (the address of the pixel adjacent to the boundary line 31 of the left boundary image Gc1) of the image data in the i-th row is r (i), the image at the pixel position (i, p) forming the boundary image Gc The data Dc (i, p) is as follows: 1 ≦ p ≦ r (i) Dc (i, p) = Dc1 (i, p) r (i) <p ≦ m Dc (i, p) = Dc2 (i, p) ). Here, Dc1 (i, p) is the image data of the pixel position (i, p) forming the left boundary image Gc1, and Dc2 (i, p) is the image data of the pixel position (i, p) forming the right boundary image Gc2. Data.

FIG. 28 is a diagram showing a second embodiment of the boundary image generating method. The boundary image generation method according to the second embodiment includes the image data Dc1 that forms the left boundary image Gc1.
And image data Dc2 constituting the right boundary image Gc2 in a checkered pattern. FIG. 28 shows the boundary image Gc.
The image data D constituting the left boundary image Gc1 is shown at the pixel position of the stippled portion, for example.
c1 is arranged, and image data Dc2 constituting the right boundary image Gc2 is arranged at the pixel position of the white portion.

Therefore, the image data Dc (i, p) constituting the boundary image Gc is configured as shown in the following (a1) to (d1).

[0082]

(Equation 3)

FIG. 29 is a modified example of the boundary image generating method according to the second embodiment. The boundary image generation method shown in the figure combines the image data forming the left boundary image Gc1 and the image data forming the right boundary image Gc2 in a checkered pattern in block units. In the figure, one block is 2
The image data Dc constituting the boundary image Gc in the case where one block is composed of K × L (K = L = 2, 3, 4,...) Pixels, although this is a case where the image is composed of four × 2 pixels (i, p)
Is configured as shown in the following (a2) to (d2), where s is a block number in the row direction and t is a block number in the column direction.

[0084]

(Equation 4)

In order to apply the boundary image generation method according to the second embodiment to the case where the Bayer single-plate color CCD according to the present embodiment is used, the CCD 12,
Taking the pixel positions of the R, G, and B color components into consideration, the image data Dc1 forming the left boundary image Gc1 and the image data Dc2 forming the right boundary image Gc2 are combined as shown in FIG. Good.

FIG. 30 shows the R, G, and B color components.
The image data Dc1 forming the left boundary image Gc1 and the image data Dc2 forming the right boundary image Gc2 are combined so as to be arranged alternately in the vertical direction. Image data Dc1 is arranged in the stippling portion, and image data Dc2 is arranged in the white portion.

The pixels of the R color component are arranged only in the odd rows / even columns, and the pixels of the B color component are arranged only in the even rows / odd columns. ,
As shown in FIGS. 31 and 32, the image data Dc1 forming the left boundary image Gc1 and the image data Dc2 forming the right boundary image Gc2 are alternately arranged only in the vertical direction. On the other hand, since the pixels of the G color component are arranged in odd rows / odd columns and even rows / even columns, the pixels of the G color component are left and right in both the vertical and horizontal directions as shown in FIG. The image data Dc1 constituting the boundary image Gc1 and the image data Dc2 constituting the right boundary image Gc2 are arranged alternately.

Therefore, the image data Dc (i, p) constituting the boundary image Gc shown in FIG. 30 is represented by the following (a3) to (d3)
It is configured as follows.

[0089]

(Equation 5)

As described above, for each of the R, G, and B color components, the image data Dc1 forming the left boundary image Gc1 and the image data Dc2 forming the right boundary image Gc2 are alternately arranged in the vertical or horizontal direction. 28, the density change at the boundary portion C can be made inconspicuous, as in the case of FIG.

By the way, the boundary image generating method according to the second embodiment uses the image data Dc1 and the right boundary image Gc constituting the left boundary image Gc1 in pixel units or block units.
Although the image data Dc2 constituting c2 is synthesized in a checkered pattern, the boundary image Gc is divided into a plurality of blocks, and the image data Dc1 and the right boundary image constituting the left boundary image Gc1 in each block. Image data Dc2 constituting Gc2
May be combined. In this boundary image generation method (the boundary image generation method according to the third embodiment), the area ratio (the ratio of the number of pixels) between the left boundary image Gc1 and the right boundary image Gc2 and the pixel arrangement of both images are determined by the block position. By changing the density, the density change of the boundary portion C can be made inconspicuous.

Various methods can be considered as a method for synthesizing the image data Dc1 constituting the left boundary image Gc1 and the image data Dc2 constituting the right boundary image Gc2 in the block for making the density change inconspicuous.

In the first method, the boundary portion C is divided into two regions in the horizontal direction. In the region on the left image G1 side, the number of pixels of the image data Dc1 constituting the left boundary image Gc1 included in a certain block is calculated. N1, assuming that the number of pixels of the image data Dc2 constituting the right boundary image Gc2 is N2, as the block becomes closer to the boundary with the left image G1, the pixel number ratio N1 / N2 is increased, and the right boundary within the block is increased. The image data Dc2 forming the image Gc2 is arranged at a position far from the boundary with the left image G1 (position near the center), and conversely, in the region on the right image G2 side, the closer the block is to the boundary with the right image G2. , The pixel number ratio N2 / N1 is increased, and the image data Dc1 constituting the left boundary image Gc1 in the block is arranged at a position far from the boundary with the right image G2 (a position near the center). Is placed.

FIG. 34 shows an example in which the block is composed of 16 pixels of 4 × 4. In the block B1 adjacent to the left image G1, N1 = 15 and N2 = 1, and 1
The image data Dc2 constituting the right boundary image Gc2 are arranged at the pixel position of the upper right corner in the block B1, and the block B
In the block B2 adjacent to No. 1, N = 14 and N2 = 2, and the image data Dc2 forming the two right boundary images Gc2
Are arranged at the pixel position at the upper right corner in the block B1 and the pixel position thereunder. In a block Bt adjacent to the right image G2, N2 = 15 and N1 = 1, and the image data Dc1 constituting one left boundary image Gc1 is a block Bt.
1 is located at the pixel position at the upper left corner. Note that FIG.
In FIG. 4, “C1” indicates that the image data Dc1 forms the left boundary image Gc1, and “C2” indicates that the image data Dc2 forms the right boundary image Gc2. As is clear from the figure, the blocks having the right and left symmetric positional relationship with respect to the center have a relationship in which the right boundary image Gc1 and the left boundary image Gc2 are interchanged.

In the second method, for example, as the block moves away from the boundary with the left image G1, the number of images of the image data Dc2 constituting the right boundary image Gc2 in the block is increased, and the pixel position is randomized. It is designed to be arranged in.

FIG. 35 is a diagram showing each pixel position when a pixel in the block is replaced with image data Dc2 constituting the right boundary image Gc2 from a block composed of image data Dc1 in which all pixels constitute the left boundary image Gc1. Is a matrix table showing the order of substitution. In this matrix table, “□” indicates a pixel position in a block, and a number in the “□” indicates a replacement order.

In the matrix table shown in FIG. 35, the number of pixels in the horizontal direction of the boundary portion C is 64, and the block is 4 × 4 1
This is a case where it is composed of six pixels. This matrix table is applied when the boundary image Gc is divided into 16 blocks in the horizontal direction. As in the case of FIG. 34, the right image G2 is shifted from the boundary with the left image G1.
B1, B2,... B1 across the boundary
If blocks B5 and B16 are arranged, these blocks B
, B2,..., B16, all pixels have the left boundary image G
Each pixel in the block constituted by the image data Dc1 constituting c1 is replaced with image data Dc2 constituting the right boundary image Gc2 in accordance with the order of the matrix table shown in FIG. Accordingly, for example, the center block B8 of the boundary portion C is, as shown in FIG.
And the image data Dc2 forming the right boundary image Gc2 are arranged in a checkered pattern. In FIG. 36, “C1” indicates that the image data Dc1 forms the left boundary image Gc1, and “C2” indicates that the image data Dc2 forms the right boundary image Gc2.

FIG. 37 is a matrix table corresponding to FIG. 35 when the above-described method is applied to the case where a Bayer single-plate color CCD is used. In the figure, "□"
(R), (G), and (B) indicate color components, and numerals indicate the order of replacement.

In the figure, the image data at each pixel position is sequentially replaced by repeating the order of the colors G, G, R, and B from the periphery to the center of the block. . Therefore, in the example of the description in FIG. 35, for example, the image data forming the blocks B2 and B8 are as shown in FIGS. 38 and 39, respectively.

The color order of (G, G, R, B) is
Other orders such as (G, R, B, G) and (R, B, G, G) may be used. If the permutation of the color order of (G, G, R, B) is changed, the example shown in FIG. In addition to the above, 11 conversion methods can be adopted.

FIG. 40 is a diagram for explaining a fourth embodiment of the boundary image generation method. The figure shows the boundary image Gc
The horizontal address indicates the horizontal signal level of the boundary image Gc. A signal waveform 32 indicated by a solid line indicates a signal waveform when the left image G1 and the right image G2 are combined, and a signal waveform 33 indicated by a dashed line indicates a left boundary image G
The waveform of the image data Dc1 forming c1 is shown, and the signal waveform 33 indicated by a dashed line indicates the waveform of the image data Dc2 forming the right boundary image Gc2.

In the boundary image generation method according to the fourth embodiment, the image data Dc1 (i, p) constituting the left boundary image Gc1 is used.
Image data Dc constituting the boundary image Gc by an average value of the image data Dc2 (i, p) constituting the right boundary image Gc2
(i, p) is generated. That is, Dc
(i, p) = {Dc1 (i, p) + Dc2 (i, p)} / 2, p = 1,2, ...
m.

Since this method simply averages the image data Dc1 and Dc2 of the left boundary image Gc1 and the right boundary image Gc2, there is an advantage that the arithmetic processing for generating the boundary image Gc is simplified. However, when the density difference between the density of the left boundary image Gc1 and the density of the right boundary image Gc2 increases, the density step at the boundary portion C becomes conspicuous.
This is an effective method when the density difference between the density of the left boundary image Gc1 and the density of the right boundary image Gc2 is relatively small.

FIG. 41 is a diagram for explaining a fifth embodiment of the boundary image generating method. The contents of the notation in this figure are the same as those in FIG. In the boundary image generation method according to the fifth embodiment, the left boundary image G is changed so that the density of the boundary image Gc continuously changes with respect to the density of the image of the other part.
The image data Dc1 (i, p) constituting the c1 and the right boundary image Gc2
The image data Dc (i, p) constituting the boundary image Gc is generated by a weighted average value with the image data Dc2 (i, p) constituting the image data Dc2 (i, p).

The weight coefficient for the image data Dc1 (i, p) is w1 (i, p), and the weight coefficient for the image data Dc2 (i, p) is w
2 (i, p), image data Dc (i, p) at pixel position (i, p)
Is Dc (i, p) = ｛w1 (i, p) · Dc1 (i, p) + w2 (i, p) ·
Dc2 (i, p)}.

In this case, in order to facilitate the arithmetic processing,
The number m of pixels in the horizontal direction of the boundary image Gc is a power of 2 (m =
2 ⁿ ). For example, if the number of pixels m is 2 ⁶ = 64, the weighting factors w1 (i, p) and w2 (i, p) are w1
(i, p) = (65−p) / 65, w2 (i, p) = p / 65, p
= 1, 2,... 64, and the image data Dc (i, p) is
Dc (i, p) = ｛(65−p) · Dc1 (i, p) + p · Dc2 (i,
p) It is calculated by｝ / 65. Note that simply, Dc (i, p) =
The calculation formula {{64−p) · Dc1 (i, p) + p · Dc2 (i, p)} / 64 may be used.

FIG. 42 is a diagram for explaining the sixth embodiment of the boundary image generation method, and is a diagram showing an example of the waveform of the horizontal output level of the image data at the boundary between the CCDs 12 and 13. FIG. 43 is also a diagram for explaining the sixth embodiment of the boundary image generation method, and shows an example of the output level waveform when the boundary images Gc are synthesized after the output levels of the CCDs 12 and 13 are adjusted. FIG.

The notation in both figures is the same as that in FIG. 40 or FIG. In both figures, a signal waveform 35 indicated by a solid line is a signal waveform of the left image G1 captured by the CCD 12, and a signal waveform 36 indicated by a solid line is a signal waveform of the right image G2 captured by the CCD 13. The signal waveform 37 indicated by the dotted line is obtained by adjusting the level so that the average level of the right image G2 is substantially the same as the average level of the left image G1.

In FIG. 43, a signal waveform 38 indicated by a chain line is generated by an average value of the image data Dc1 forming the left boundary image Gc1 and the image data Dc2 'forming the right boundary image Gc2 whose level has been adjusted. Boundary image G
It is a signal waveform of c.

In the boundary image generation method according to the sixth embodiment, when there is a difference in the average sensitivities (including those caused by the optical system) between the CCD 12 and the CCD 13, the left boundary image G
After adjusting the average level of the image data Dc1 forming the c1 and the average level of the image data Dc2 forming the right boundary image Gc2, the boundary image generation method according to the fourth or fifth embodiment is applied. .

The boundary image generating method according to the fourth or fifth embodiment is mainly intended to alleviate a sharp change in density at the boundary between the boundary image Gc and the left and right images G1 and G2 on both sides thereof. Therefore, the CCD 12 and the CCD 12
In the case where the difference between the average sensitivity and the average sensitivity is large, there is a certain limit in making the density change at the boundary portion C inconspicuous. Sixth
The boundary image generating method according to the embodiment includes a CCD 12 and a C
By generating the boundary image Gc after performing the level correction corresponding to the shading correction with the CD 13, the effect of making the density change in the boundary portion C inconspicuous can be surely and more effectively performed.

In the image data of the i-th row, the image data Dc1 (i, p) (p = 1,
2,... M) ｛Dc1 (i, 1) + Dc1 (i, 2) +... + Dc1 (i,
m)｝ / m is Va, and the average value ｛Dc2 (i, 1) of image data Dc2 (i, p) (p = 1, 2,... m) constituting the right boundary image Gc2.
+ Dc2 (i, 2) +... + Dc2 (i, m)｝ / m as Vb, FIG.
2, the signal waveform 37 indicates that all image data D2 (i, j) (j = 1, 2,.
m,... N) times Va / Vb.

As described above, by adjusting the level of the image data D2 (i, j) of the i-th row taken in by the CCD 13, the right image G2 picked up by the CCD 13 and the CCD 12
The sensitivity difference between the left image G1 and the left image G1 is corrected. As a result, the image data Dc2 (i, j) ′ (= (Va / V) constituting the right boundary image Gc2 of the level-adjusted right image G2.
b) · Dc2 (i, j)) and left boundary image Gc1 of left image G1
Is combined with the image data Dc1 (i, j) constituting the image data to make the density change at the boundary portion C less noticeable (FIG. 4).
0, see FIG. 41 and FIG. 43).

In the examples of FIGS. 42 and 43, the CCD 1
3, the signal level of the left image G2 is
Is adjusted to the signal level of the right image G1 captured in the step (1). Alternatively, the signal level of the left image G1 may be adjusted to the signal level of the right image G2. Each of the signal levels may be adjusted to a specific level.

In FIG. 43, the image data Dc forming the boundary image Gc is generated by the average value calculation. However, the image data Dc forming the boundary image Gc is generated by the other method described above. It may be.

In the examples of FIGS. 42 and 43, the magnification coefficient K (= Va / Vb) for level adjustment is calculated for each horizontal line using the image data of the line. Line noise may occur in the subsequent image. Therefore, as shown in FIG. 44, in a block in which two adjacent front and rear horizontal lines are combined, image data Dc1 (i, p) (p = 1, 2,...) Constituting the left boundary image Gc1.
m) and the average value Vb 'of the image data Dc2 (i, p) constituting the right boundary image Gc2 are calculated, and the magnification coefficient K' (= Va ') is calculated from both the average values Va' and Vb '. / Vb ')
May be calculated to prevent generation of line noise.

In the example of FIG. 44, the magnification coefficient K 'in the k-th line is (k-2), (k-1), k,
Left boundary image Gc in each row of (k + 2) and (k + 1)
Image data Dc1 (i, p) constituting 1 (p = 1, 2,... M)
Are the average values of Va (k-2), Va (k-1), Va (k), Va (k + 1), V
a (k + 2), and image data D constituting the right border image Gc2
The average value of c2 (i, p) (p = 1, 2,... m) is calculated as Vb (k-2), Vb
Assuming that (k-1), Vb (k), Vb (k + 1), and Vb (k + 2), it is calculated by the following equation (5).

[0118]

(Equation 6)

Next, shooting control of the digital camera 1 will be described with reference to the flowcharts of FIGS.

When the main switch is turned on and the digital camera 1 is started, the digital camera 1 enters a photographable state (# 2 loop). When the shutter button 9 is operated by the photographer and a photographing instruction signal is input (YES in # 2), the distance measuring unit 2
The data of the subject distance is fetched by 1 and the focus control value of the photographing lens 2 is calculated from the data (# 4).
Subsequently, the data of the subject brightness is taken in by the photometry unit 22 (# 6), and the exposure control values (aperture value and shutter speed) are calculated using the data of the subject brightness (# 6).
8).

Subsequently, the data of the calculated aperture value is output to the aperture drive control unit 17, the aperture of the aperture 15 is adjusted, and the data of the calculated shutter speed is output to the CCD drive control unit 18. (# 10). Further, the calculated focus control value is output to the lens drive control unit 17, and the focusing lens of the photographing lens 21 is moved to perform focus adjustment (# 12).

Subsequently, the type of the photographing lens 2, the focal length,
The pupil position of the photographing lens 2 is calculated based on the lens extension amount and the like (# 14).
3 is selected, and RO correction is performed.
It is read from M23 (# 16). In the ROM 23,
The above four types of basic correction tables (A) to (D) are C
When the pupil position is close, the data is stored in correspondence with the CDs 12 and 13, (A) (table corresponding to the case in FIG. 15) and (C) (table corresponding to the case in FIG. 17).
Is selected. When the pupil position is close, (B) (table corresponding to the case of FIG. 16)
And (D) (the table corresponding to the case of FIG. 18) are selected.

Subsequently, a shading correction table corresponding to the aperture value of the aperture 15 calculated using the selected basic correction table is calculated (# 18), and in accordance with the calculated pupil position of the photographing lens 2. The calculated shading correction table is calculated (# 20), and the final shading correction table is set by multiplying the shading correction table by a correction table for correcting variations in the photoelectric conversion characteristics of the CCDs 12 and 13 (# 2).
2).

Subsequently, the CCDs 12 and 13 are driven for a predetermined time (integrated for a time corresponding to the shutter speed), and an image of a subject is taken (# 24). CCD12,
The image signals constituting the left image G1 and the right image G2 captured by the A / D converter 13 are subjected to predetermined signal processing by the analog signal processor 191 for each of the R, G, and B color components, and then the A / D converter 192. Is converted to image data by the image memory 193
Is stored.

The left image G stored in the image memory 193
1, the image data constituting the right image G2 is R, G, B
After the shading correction is performed by the shading correction unit 194 using the shading correction table set in step # 22 for each of the color components (# 26), the image data interpolating unit 195 performs the above-described interpolation processing to perform image data shortage. Image data at the selected pixel position is supplemented (# 2
8). Further, the image combining unit 196 uses the image data of the left boundary image Gc1 of the left image G1 and the right boundary image Gc2 of the right image G2 by the above-described boundary image generation method to make the density change at the boundary portion C inconspicuous. Image data of the boundary image Gc for use is generated (# 30), and the image data forming the boundary image Gc and the image data forming the part of the left image G1 and the right image G2 excluding the boundary images Gc1 and Gc2. Are combined to generate image data constituting a captured image G of the entire subject (# 32).

Then, the image data forming the captured image G is output to the HD card 10 via the output I / F 197 and recorded (# 34). This completes the imaging operation for one sheet, and returns to step # 2 to perform the next imaging operation.

In the above embodiment, two CCDs are used.
However, the present invention can be applied to a case where the subject light image A is divided and photographed using three or more CCDs. In addition, the present invention is not limited to color photography, and can be applied to monochrome photography.

Further, in the above embodiment, the digital still camera has been described, but the present invention can be applied to a digital video camera.

[0129]

As described above, according to the present invention,
The subject light image is divided into a plurality of partial light images such that the boundary portions overlap each other, and each of the partial light images is captured by a plurality of single-plate color imaging units. In a digital camera that generates a captured image of the entire light image of a subject by laminating the image data of the partial images output from the color imaging unit for each color component, the image data of each color component is generated before generating a captured image of the entire subject. Since the image data at the pixel position where the image data of the partial image is insufficient is supplemented by the interpolation processing, false colors do not occur in the composite image due to the difference in sensitivity of the color imaging means.

[Brief description of the drawings]

FIG. 1 is a perspective view showing an appearance of a digital camera according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a schematic configuration of an imaging optical system provided in the digital camera according to the present invention.

FIG. 3 is a diagram illustrating a relationship between a subject light image in an imaging screen and a partial light image of the subject light image formed on an imaging surface of a color imaging device.

FIG. 4 is a diagram illustrating a relationship between the number of pixels of a color imaging element and the number of pixels of an imaging screen.

FIG. 5 is a diagram illustrating an array of Bayer-type color filters provided on an imaging surface of a color imaging element.

FIG. 6 is a plan view showing a microlens provided in each pixel of the color imaging device.

FIG. 7 is a cross-sectional view showing a microlens provided for each pixel of the color image sensor.

FIG. 8 is a diagram showing an embodiment of a block configuration of a digital camera according to the present invention.

FIG. 9 is a diagram illustrating a block configuration related to processing such as shading correction, generation of a captured image, and interpolation of image data in an image processing unit.

FIG. 10 is a diagram illustrating a modified example of a block configuration related to processes such as shading correction, generation of a captured image, and interpolation of image data in an image processing unit.

FIG. 11 is a cross-sectional view of a microlens thinner than a standard.

FIG. 12 is a cross-sectional view of a microlens thicker than a standard.

FIG. 13 is a cross-sectional view of a microlens having a positional shift.

FIG. 14 is a diagram illustrating a state in which an incident light beam is obliquely incident on a microlens.

FIG. 15 is a plan view illustrating an imaging surface of an imaging element.

FIG. 16 is a diagram illustrating an incident direction of an incident light beam when a pupil position of a photographing lens is far from an imaging surface of an imaging element.

FIG. 17 is a diagram illustrating an incident direction of an incident light beam when a pupil position of a photographing lens is close to an imaging surface.

FIG. 18 is a diagram illustrating a change in a light beam width of an incident light beam when the stop is stopped down in the case of FIG. 16;

FIG. 19 is a diagram showing a change in a light beam width of an incident light beam when the stop is opened in the case of FIG. 17;

FIG. 20 is a perspective view illustrating a variation in sensitivity of an imaging surface of an imaging element.

FIG. 21 is a diagram illustrating a state in which an imaging surface of an imaging element is divided into a plurality of blocks.

FIG. 22 is a diagram for explaining a change amount of an incident angle of an incident light beam when a pupil position of a photographing lens changes.

FIG. 23 is a diagram illustrating a correction coefficient K of a shading correction table on an imaging surface when the pupil position of the photographing lens is changed to a position where the angle of incidence of the incident light beam is smaller than the maximum value.
It is a figure showing the field to which (h, k) is applied.

FIG. 24 is a diagram for explaining a method of correcting the correction coefficient K (h, k) when the pupil position of the photographing lens is changed to a position where the incident angle of the incident light beam is smaller than the maximum value as a whole.

FIG. 25 is a diagram illustrating a procedure of interpolation processing of image data of an image of each of R, G, and B color components.

26A and 26B are diagrams illustrating a first embodiment of a boundary image generation method, in which FIG. 26A is a diagram in which the boundary is saw-toothed, and FIG. 26B is a diagram in which the boundary is sine-shaped.

FIG. 27 is a diagram showing addresses in the horizontal direction of image data forming a boundary portion.

FIG. 28 is a diagram illustrating a boundary image generation method according to a second embodiment.

FIG. 29 is a diagram illustrating a modification of the boundary image generation method according to the second embodiment.

FIG. 30 is a diagram for describing a method of applying the boundary image generation method according to the second embodiment to a case where a Bayer-type color imaging device is used.

FIG. 31 is a diagram showing image data forming an image of an R color component at a boundary portion.

FIG. 32 is a diagram showing image data constituting an image of a B color component at a boundary portion.

FIG. 33 is a diagram showing image data forming an image of an R color component at a boundary portion.

FIG. 34 is a diagram for explaining a boundary image generation method according to the third embodiment.

FIG. 35 is a matrix table showing a replacement order of each pixel position when replacing a pixel in the block with image data forming a right boundary image from a block including all pixels forming image data forming a left boundary image; It is.

36 is a diagram illustrating a configuration of image data of a block B8 at the center of a boundary portion generated based on the matrix table illustrated in FIG. 35;

FIG. 37 is a matrix table corresponding to FIG. 35 and applied when a Bayer single-plate color CCD is used.

38 is a diagram showing a configuration of image data of a block B2 generated based on the matrix table shown in FIG. 37.

39 is a diagram illustrating a configuration of image data of a block B8 at the center of a boundary portion generated based on the matrix table illustrated in FIG. 37.

FIG. 40 is a diagram illustrating a waveform of an output level of an image of a boundary portion generated by the boundary image generation method according to the fourth embodiment.

FIG. 41 is a diagram illustrating a waveform of an output level of an image of a boundary portion generated by the boundary image generation method according to the fifth embodiment.

FIG. 42 is a diagram illustrating a waveform of an output level at a boundary portion between two CCDs that capture a subject light image.

FIG. 43 is a diagram showing output level waveforms when both captured images are combined after adjusting the output levels of two CCDs.

FIG. 44 is a diagram for describing a method of calculating a magnification coefficient K for level adjustment when performing level adjustment on a line-by-line basis using image data included in several lines before and after including a target line.

FIG. 45 is a diagram showing a flowchart of photographing control.

FIG. 46 is a view showing a flowchart of photographing control.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Photographing lens 3 Metering window 4 Light-emitting window 5 Light-receiving window 6 Viewfinder objective window 7 Card slot 8 Card eject button 9 Shutter button 10 HD card 11 Optical path separation unit 12, 13 Color imaging element (color imaging means) 14 Microlens 15 Aperture 16 Lens drive controller 17 Aperture drive controller 18 CCD drive controller 19 Image processor (image processing controller) 191 Analog signal processor 192 A / D converter 193 Image memory 194 Shading corrector 195 Image data Interpolation unit (image data interpolation means) 196 Image synthesis unit 197 Output I / F 20 Card drive control unit 21 Distance measurement unit 22 Photometry unit 23 ROM 24 RAM 25 Control unit 26 Light beam center 27R-27B, 27R'-27B'R, G, B color component image data 28R, 8B, 28R ', 28B' R, color difference data 29, 30 interpolation filter 31 borders 32-38 signal waveforms of the color components of B

Claims (1)

[Claims] 1. An object light image is divided into a plurality of partial light images such that boundary portions overlap each other, and each of the partial light images is captured by a plurality of color imaging means. In a digital camera that generates a captured image of the entire subject light image by bonding at a boundary portion, the color imaging unit includes a single-plate color imaging unit in which three primary color filters are arranged in a predetermined positional relationship on an imaging surface. Become
An image data interpolating unit that supplements image data at a pixel position where a captured image of each color component is insufficient by an interpolation process using the image data output from the color imaging unit, and after the interpolation process of the captured image of each color component, A digital camera, comprising: an image processing control unit that performs the bonding process of the captured images.