JP2000333060A - Digital camera and storage medium for image signal processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect smear caused in image data of a digital camera. SOLUTION: An image processing section 431a amplifies an object image signal photographed by an image pickup element 214a up to a prescribed level. The object image signal photographed by an image pickup element 214b is amplified by an image processing section 431b up to a prescribed level. An A/D converter circuit 432 converts the image signals by the image pickup elements 214a, 214b into digital. signals, and a DSP 433 applies image processing to the digital signals. A CPU 439 discriminates whether or not an output value of pixel output data photographed by the image pickup element 214a is 250 or over, and when the CPU 439 discriminates that the output value is 250 or over, the CPU 439 discriminates whether or not an output value of pixel output data photographed by the corresponding image pickup element 214b is in excess of 200. When the pixel output, value by the image pickup element 214b is 200 or below, occurrence of smear is discriminated and the pixel output value being 250 or over by the image pickup element 214a can be corrected.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a digital camera capable of capturing a subject image and storing subject image data, and a storage medium storing an image signal processing program.

[0002]

2. Description of the Related Art An image pickup device such as a CCD for picking up an image of a subject passing through a photographing lens and outputting an image signal, an image processing device for image-processing the image signal for display, and processing the image data after the image processing. 2. Description of the Related Art A digital still camera including a storage device that compresses data by a method such as JPEG and stores the compressed data in a storage medium such as a flash memory is known.

[0003] CCD of such a digital still camera
Is known to cause a phenomenon called smear when an object with extremely high brightness exists in the field. Smear is observed as a white streak in the captured image.Charge overflows from the place where strong light hits on the CCD, it is on the same line as the place where the charge overflows, and accumulated on the CCD. This is caused by adding an overflowing charge to a charge read after the above-described portion when the charge is read.

[0004]

When the smear described above occurs, white streaks that do not exist in the subject appear in the image data, so that there has been a problem that a good image cannot be recorded.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a digital camera which detects, for example, when smear is generated in an image signal output by capturing a subject image. Another object of the present invention is to provide a storage medium storing, for example, a program for detecting smear, if it occurs, in subject image data stored in the storage medium.

[0006]

FIG. 1 shows an embodiment of the present invention.
The present invention will be described with reference to FIGS. (1) In a digital camera according to a first aspect of the present invention, an optical unit 2 for separating a subject image into a first subject image and a second subject image and forming the first subject image and the second subject image on the first and second imaging surfaces, respectively.
19, first and second imaging units 214a and 214b arranged on the first and second imaging planes and imaging the first and second subject images, respectively, and the first imaging unit 214
a detecting means 439 for detecting a first pixel whose output value a is equal to or greater than a first predetermined value 250, and an output value of a second pixel of the second imaging means 214b corresponding to the first pixel being a second value. The above-described object is achieved by providing the determination unit 439 for determining whether or not the predetermined value 201 is equal to or more than the predetermined value 201. (2) In the digital camera according to the first aspect, the output value of the first pixel may be smaller than the first predetermined value 250 based on the determination result by the determination unit 439. It is characterized by including a correction means 439 for correcting the output value of the first pixel. (3) The digital camera according to claim 1 or 2, wherein the first and second image pickup means 21
4a and 214b, when reading out the accumulated electric charge based on the respective subject images projected on the two image pickup means 214a and 214b,
14a and 214b are characterized by being opposite to each other. (4) According to the invention of claim 4, the subject image is divided into first subject image data and second subject image data obtained by imaging on mutually conjugate planes, and stored in a storage medium, respectively.
The present invention is applied to an image signal processing storage medium storing an image processing program for reading out first and second subject image data stored in a storage medium and performing image processing. Then, the image processing program detects a first data in which the luminance value of the first subject image data is equal to or more than the first predetermined value 250, and a second process corresponding to the first data.
And determining whether the luminance value of the second data of the subject image data is equal to or greater than a second predetermined value 201. (5) The image signal processing storage medium according to (4), wherein the brightness value of the first data is set to the first value based on the determination result of the determination processing by the image processing program. A correction process for correcting the luminance value of the first data so as to be smaller than the predetermined value 250 is performed.

[0007] In the section of the means for solving the above-mentioned problems, which explains the configuration of the present invention, the drawings of the embodiments are used to facilitate understanding of the present invention. However, the present invention is not limited to this.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS -First Embodiment- An embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the single-lens reflex digital still camera 100 according to the first embodiment includes a camera body 200, a finder device 50 detachable from the camera body 200,
The camera includes a photographing lens 1 which has a built-in lens 91 and a stop 92 and is detachably attached to the camera body 200. The subject light L passes through the photographing lens 1 and is incident on the camera body 200. The quick return mirror 3, which is a half mirror, transmits light L1 for focus detection (autofocus: AF) and reflected light L2 for finder observation. Is separated into The transmitted light L1 passes through the quick return mirror 3 in the viewfinder observation state and is collected by the field lens 209.
Further, the light is reflected by the mirror 210 and is later described as a mirror 211.
It is led to. On the other hand, the reflected light L2 is
The finder optical system 207 guides the subject image onto the finder mat 4 conjugated to the primary imaging surface 16 of the photographing lens 1 by being guided to the finder prism 4.
And is observed by the photographer 18.

When the digital still camera 100 is released, the half mirror 3 rotates to the position shown by the broken line 3 ', and the subject light L proceeds to the mirror 210 without being separated.

The above-mentioned field lens 209 is provided behind the quick return mirror 3 (right side in FIG. 1) and forward (left side in FIG. 1) of the primary imaging surface 16 of the taking lens 1. Behind the primary imaging plane 16,
Mirror 2 fixed at an angle θ of 55 degrees with respect to the horizontal direction
10 is provided, and reflects the luminous flux of the subject light L toward a mirror 211 provided in front of immediately below the mirror 210. The light beam incident on the mirror 211 is
At 1, the light is reflected horizontally in the depth direction of the drawing. As shown in FIG. 2 as seen through the digital still camera 100 from the back, the light beam reflected rightward in FIG. 2 by the mirror 211 enters a reduction optical system 212 described later.

FIG. 3 shows an image forming system including a reduction optical system 212, a beam splitter 219, and image pickup devices 214a and 214b which are arranged behind the above-mentioned primary image forming surface 16 of the photographing lens 1 (in FIG. 3, obliquely right above). It is a perspective view of an image optical system. Inside the reduction optical system 212, a shutter plate 208 is sandwiched between relay lenses 212a and 212b.
And an aperture plate 215 are provided adjacent to each other. The shutter plate 208 and the diaphragm plate 215 are formed in a disk shape, and the shutter plate 208 and the diaphragm plate 215 are rotated by step motors 408 and 415 (FIG. 2) provided at the rotation centers of the disks, respectively.

As shown in FIG. 3, the shutter plate 208 has a complete light blocking portion 208a for blocking all subject light beams, an opening portion 208b for passing all subject light beams, and a pair of opening portions arranged in the horizontal direction of the scene. 208c and 208 having
d, and a pair of openings 208 arranged in the vertical direction of the object scene
e and 208f, a pair of openings 208g and 208h arranged in a direction oblique to 45 degrees with respect to the horizontal direction of the scene, and a pair of openings 208 arranged in a direction orthogonal to the oblique 45 degrees.
i and 208j are provided. Then, the opening 208
c, opening 208e, opening 208g, and opening 2
08i includes polarizing plates 308c, 308e, 308g, and 308i that transmit the P-polarized component from the subject light beam, and the opening portions 208d, 208f, 208h, and 208j transmit the S-polarized component from the subject light beam. Polarizing plates 308d, 308f, 308h, and 308j are provided, respectively. Complete shading part 2 for normal shooting
08a and an opening 208b are used, and the openings 208c to 208j are used at the time of a pupil division type focus detection operation described later.

On the other hand, a diaphragm plate 215 has a diaphragm opening 215.
a to 215 g are provided. With reference to the area of the opening 215a through which all the subject light beams pass, the aperture area of the aperture openings 215b to 215g is
The opening area is set so as to be halved in order to 215 g. As shown in FIG. 3, by alternately arranging openings having a large opening area and openings having a small opening area, the distance between adjacent openings is not reduced. On the exit side of the reduction optical system 212 (upper right of the relay lens 212b in FIG. 3), the polarization beam splitter 21 is provided.
9 are provided. The polarization beam splitter 219 is
It transmits the P-polarized light component and reflects the S-polarized light component of the incident subject light beam. The P-polarized light component of the light beam transmitted through the polarization beam splitter 219 is incident on the image sensor 214a,
The S-polarized component of the light beam reflected by the polarization beam splitter 219 is incident on the image sensor 214b. Image sensor 214
a and 214b are imaged on the primary imaging surface 16
Is an image formed as an aerial image reduced by the action of the field lens 209 and the relay lenses 212a and 212b. Although not shown in FIG. 3, as shown in FIG.
4b and the polarization beam splitter 219, a CCD
Protective glasses 213a and 213b and optical low-pass filters 223a and 223b are provided, respectively.

FIG. 4 is a block diagram showing a circuit of the digital still camera 100 according to this embodiment. CPU4
When the control program stored in the ROM 443 is activated, the control unit 39 appropriately controls the blocks of each unit based on an operation signal input from the operation member 407.
The digital still camera 100 has two operation modes, a recording mode in which a subject image is captured and image data is recorded, and a reproduction mode in which recorded image data is read out and reproduced, as described later. Operation member 4
Switching is performed based on an operation signal input from the control unit 07.

The image sensor 214 already shown in FIG.
a and 214b are CCDs having the same number of pixels,
The optical image formed on each pixel is photoelectrically converted into an electrical image signal. A digital signal processor (hereinafter referred to as DSP) 433 supplies a horizontal drive signal to the image sensors 214a and 214b, controls an image sensor drive circuit 434, and supplies a vertical drive signal to the image sensors 214a and 214b. Supply.

The image processing units 431a and 431b have a CPU
439, the image sensors 214a and 214
The image signal photoelectrically converted in b is sampled at a predetermined timing and amplified to a predetermined signal level. The set value of the amplification factor for amplifying the image signal is changed between the time of shooting and the time of focus detection described later. In order to shorten the exposure time at the time of focus detection, for example, at the time of focus detection, it is set to about four times that at the time of shooting. An analog / digital conversion circuit (hereinafter, referred to as an A / D conversion circuit) 432 has a two-channel input port.
The amplified image signals output from 31a and 431b are input respectively. Then, the image data converted into the digital signal by the A / D conversion circuit 432 is
The image data is input to the image processing unit 33 and subjected to image processing such as contour compensation, gamma correction, and white balance adjustment.

The DSP 433 controls the data bus connected to the buffer memory 436 and the memory card 424, and temporarily stores the image data subjected to the image processing in the buffer memory 436.
The stored image data is read out from the
a and 214b are combined.
After that, for example, a predetermined format process is performed for JPEG compression, and the image data after the format process is data-compressed at a predetermined ratio by the JPEG method and recorded on the memory card 424. The DSP 433 stores the image data after the image processing in the frame memory 435, and stores the image data in the LCD 2 provided on the back of the camera body 200.
20 (FIG. 2) or read out and expand the captured image data recorded from the memory card 424, store the expanded captured image data in the frame memory 435 and display it on the LCD 220. In addition, DSP433
Is a buffer memory 4 for recording the above-described image data on the memory card 424 and for taking the expanded image data.
The timing management of data input / output in recording to 36 is performed.

The buffer memory 436 is a memory card 4
24 and the difference in the speed of input / output of image data
It is used to reduce the difference in processing speed between the U439 and the DSP433. The colorimetric element 417 measures the color temperature of the subject and its surroundings, and outputs the measurement result to the colorimetric circuit 452. The colorimetric circuit 452 performs a predetermined process on the analog signal, which is the colorimetric result output from the colorimetric element 417, to convert the analog signal into a digital signal, and outputs the converted digital signal to the CPU 439. The above-described white balance adjustment is performed based on the digital output signal. The timer 445 has a built-in clock circuit and outputs time data corresponding to the current time to the CPU 439. This time data is recorded on the memory card 424 together with the image data described above.

The diaphragm driving circuit 453 is provided with the diaphragm plate 2 described above.
A step motor 415 for rotating the aperture plate 215 is driven so that a predetermined one of the fifteen openings 215a to 215g is selected. The measurement of the exposure for setting the aperture is performed based on the image signals output from the image sensors 214a and 214b. The shutter drive circuit 454 controls the shutter open time by driving a step motor 408 that rotates the shutter plate 208 and switching between a complete light blocking portion 208a provided on the shutter plate 208 and an opening 208b through which all light beams pass. I do. At the time of focus detection, the openings 208c to 208j are switched.

The lens drive circuit 430 moves the photographing lens 1 to perform focus adjustment. The in-finder display circuit 440 causes the in-finder display element 426 to display the settings in the various operations described above. The interface 448 is provided so as to exchange predetermined data such as captured image data with a predetermined external device (not shown).

-Focus Detection-The focus detection operation of the digital still camera 100 will be described. The CPU 439 controls the aperture driving circuit 453
Is controlled to drive the stepping motor 415 and the diaphragm plate 21
Among the openings formed in 5, the opening 215b through which all light beams pass is set on the optical axis. Further, the shutter drive circuit 454 is controlled to drive the step motor 408 so that, for example, the centers of the openings 208 e and 208 f formed in the direction perpendicular to the optical axis of the shutter plate 208 are on the optical axis. set. That is, the opening 20
8e and 208f are arranged vertically side by side in the scene.

The primary image incident on the taking lens 1 and formed on the primary image plane 16 is reflected by mirrors 210 and 211 as a light beam of the subject and is incident on the shutter plate 208.
The incident subject light flux is emitted as a focus detection light flux of a P-polarized light component and a focus detection light flux of an S-polarized light component. That is, due to the action of the polarizing plate 308e (FIG. 3) provided in the opening 208e and transmitting P-polarized light, the light beam transmitted through the opening 208e becomes only the P-polarized component. On the other hand, the opening 20
By the operation of the polarizing plate 308f provided in 8f and transmitting S-polarized light, the light beam transmitted through the opening 208f becomes only the S-polarized component. For this reason, the focus detection light flux of the P-polarized light component transmitted through the opening 208 e is
The light is transmitted without being reflected at the imaging element 214a. The focus detection light flux of the S-polarized light component transmitted through the opening 208f is reflected without passing through the polarization beam splitter 209, and is incident on the image sensor 214b. As described above, the light beam transmitted through the opening 208e enters the image sensor 214a, and the light beam transmitted through the opening 208f enters the image sensor 214b.
By comparing the two subject image data based on the image signal output from 14b, focus detection can be performed as follows.

FIG. 5 is a diagram for explaining the principle of the focus detection operation by pupil division. An area 800 is an exit pupil of the photographing lens 1 (FIG. 1). Also, in FIG.
0, 211, the aperture plate 215 and the polarization beam splitter 219 are omitted. The regions 801 e and 801 f are provided with openings 208 e and 2
08f is a region where an image is projected back onto the region 800 by the field lens 209. Region 801e,
The light beam incident via the lens 801f focuses on the primary imaging surface 16 of the photographing lens 1 and then is focused on the field lens 2
09, relay lens 212a, openings 208e, 208
f and the relay lens 212b,
a, re-imaged on 214b.

A pair of subject images re-imaged on the image pickup devices 214a and 214b is a so-called front focus state in which the photographing lens 1 forms a sharp image of the subject before the primary imaging surface 16 (subject side). In contrast, in the so-called rear focus state in which a sharp image of the subject is formed after the primary imaging plane 16, they move away from each other. Then, the imaging elements 214a,
When the subject image formed on 14b has a predetermined interval, a sharp image of the subject is located on the primary imaging plane 16. Therefore, the pair of subject images are transferred to the imaging devices 214a and 214a.
In 4b, photoelectric conversion is performed to convert the signals into electric signals, and these signals are arithmetically processed to obtain a relative distance between a pair of subject images. Thus, a focus adjustment state of the photographing lens 1, that is, a sharp image is formed by the lens 1. The direction and how far the position is from the primary imaging plane 16, that is, the shift amount is obtained.

In FIG. 5, the focus detection areas are back-projected by the relay lenses 212a and 212b on the imaging elements 214a and 214b selected by the photographer.
This corresponds to an overlapping portion near the primary imaging plane 16. The photographer may use, for example, the image sensor 21 as the focus detection area.
Area including the subject closest to the camera in the screen by 4a and 214b, image sensors 214a and 214
The setting is performed by an operation member (not shown) so that any one of an area including a subject having the highest contrast in the screen b and an area to be arbitrarily set is selected. C
After detecting the focus of the selected focus detection area, the PU 439 controls the lens drive circuit 430 based on the focus detection data to drive the photographing lens 1 to the focus position. As described above, the time required for focus detection can be reduced by performing focus detection based on data in a predetermined specific focus detection area without using data of all areas of the imaging elements 214a and 214b.

In the optical system shown in FIG.
Observing the image transmitted through the apertures 208e and 208f of the E.8 corresponds to observing the object field by dividing the object field into two pupils shifted in the horizontal direction by the action of the mirrors 210 and 211. Called split. The direction of pupil division is changed by rotating the shutter plate 208. For example, the openings 208c and 208d formed in the horizontal direction with respect to the optical axis of the shutter plate 208 so as to be orthogonal to the direction in which the openings 208e and 208f are arranged.
Is set to be on the optical axis, which corresponds to observing the object scene by dividing the pupil in the vertical direction. If the object scene contains a horizontal line or the like and contrast cannot be obtained from the subject image divided horizontally in the pupil, that is, if it is determined that focus detection is difficult, change the direction of the pupil division in this way. The focus detection operation is performed. The direction of pupil division is horizontal and vertical with respect to the object field, and the aperture 208
A direction of 45 degrees oblique to the horizontal direction by g and 208h and a direction of -45 degrees oblique to the horizontal direction by the openings 208i and 208j can be selected, and these oblique directions are selected according to the subject. You.

The above-mentioned image pickup devices 214a and 214
Since b is not disposed on the same plane, the distance between a pair of subject images re-imaged on both image sensors is obtained as follows. As shown in FIG. 3, the image sensor 214a
And 214b are arranged on an image plane conjugate to each other, so that the
The distance between the pair of subject images can be obtained based on the respective pixel positions on the image sensor 214b where the subject image of 4a and the S-polarized component is detected.

Next, the operation of detecting the interval between a pair of subject images obtained by pupil division will be described. The detection operation is performed on the above-described focus detection area, that is, the subject image in a specific area selected by the photographer in the scene. FIG. 6 is an enlarged view of a portion corresponding to the focus detection area of the imaging elements 214a and 214b. FIG. 6 (a)
As shown in (2), among the data of the subject image detected on the image sensor 214a, the pixel data corresponding to the focus detection area is read out as the pixel group 1a, the pixel group 2a,. If these pixel groups are selected such that the direction of pupil division is horizontal with respect to the object scene, for example, a total of 10 pixels of 25 pixels in the vertical direction and 4 pixels in the horizontal direction
It is composed of 0 pixels. In each pixel group read out,
A total output of 100 pixels is calculated to obtain a total output value of N pixels. The adjacent pixel groups are set so as to be shifted from each other by two pixel pitches as shown in FIG. However,
When performing a precise detection operation, as shown in FIG.
The pixel group may be set so as to be shifted by the pixel pitch.

In the same manner as the subject image data detected on the image sensor 214a, the data of the subject image detected on the image sensor 214b includes pixel groups 1b, 2b,. , Pixel group Nb. As shown in FIG. 6C, the pixel pitch between adjacent pixel groups is set to be the same as the pixel groups 1a to Na. In each of the read pixel groups, the sum of outputs of 100 pixels is calculated to obtain a total output value of N pixels.

FIG. 7 shows an image sensor 214a and an image sensor 2
It is an example of the graph showing the relationship between the position of each pixel group detected on 14b, and the pixel total output value of each pixel group. The output from the pixel group based on the image sensor 214a (black circle) and the output from the pixel group based on the image sensor 214b (white circle) are:
Curves corresponding to the respective subjects are shown, and the interval between the two curves is the interval between a pair of subject images. For example, when the two curves are displaced by two pixel groups, the image sensor 214a
And the horizontal pixel pitch on 214b is 2
(Shift of two pixel groups) × 4 (the pixel group is composed of four pixels in the horizontal direction) = 8 times to obtain the relative distance of the subject image. The focus adjustment state of the photographing lens 1 is determined based on the determined relative distance between the pair of subject images.

The operation of the digital still camera 100 thus configured in the recording mode will be described. Fig. 8-
9 is a flowchart showing a program started by the half-press switch. The CPU 439 includes an operation member 40
7, a half-press operation signal and a full-press operation signal interlocked with a release button (not shown) are input. When it is determined that the half-press operation signal has been input (Y in step S1), the amplification factor for amplifying the image signal in the image processing units 431a and 431b is changed to a predetermined value at the time of focus detection. Subsequently, an opening 215b on the aperture plate 215 through which all light beams pass is set on the optical path (step S2), and openings 208e and 208f formed in a direction perpendicular to the optical axis of the shutter plate 208 are placed on the optical path. (Step S3). On the other hand, when it is not determined in step S1 that the half-press operation signal has been input (N in step S1), the process waits for the input of the half-press operation signal.

In step S4, image data based on the image signals picked up by the image pickup devices 214a and 214b is read out, and there is contrast in the image data in the focus detection area selected as described above. It is determined whether or not (Step S5). If it is determined that there is contrast (Y in step S5), step S15
When it is determined that there is no contrast (N in step S5), the shutter plate 208 is rotated, and the apertures 208c and 208d formed in the horizontal direction with respect to the optical axis are set to be on the optical path. (Step S6). In step S7, the image sensors 214a and 214b are again
The image data based on the image signal picked up in step is read out, and it is determined whether or not there is a contrast in the focus detection area in the image data (step S8). If it is determined that there is contrast (Y in step S8), the process proceeds to step S15, and if it is determined that there is no contrast (N in step S8), the shutter plate 208 is rotated again, and the shutter plate 208 is tilted with respect to the optical axis. Opening 20 formed
8g and 208h are set on the optical path (step S9).

In step S10, the image sensor 214 is again activated.
The image data based on the image signal a and the image signal captured by the image sensor 214b is read, and it is determined whether or not the image data has a contrast in the focus detection area (step S11). When it is determined that there is contrast (Y in step S11), the process proceeds to step S15, and when it is determined that there is no contrast (N in step S11), the shutter plate 208 is rotated again, and the opening 208g is opened.
Opening 208i formed so as to be orthogonal to and 208h
And 208j are set on the optical path (step S12). In step S13, the image data based on the image signals captured by the image sensor 214a and the image sensor 214b is read again, and it is determined whether or not there is a contrast in the focus detection area in the image data (step S14). ). When it is determined that there is contrast (Y in step S14), the process proceeds to step S15,
If it is determined that there is no contrast (N in step S14), a warning indicating that focusing is impossible is displayed on the display element 426 in the finder via the display circuit 440 in the finder (step S17), and the process returns to step S1.

In step S15, two image data based on the focus detection areas output from the image sensor 214a and the image sensor 214b are compared with each other, and the focus is detected by pupil division as described above. Then, based on the obtained focus detection data, the taking lens 1 is driven by the lens driving circuit 4.
The camera is driven to the in-focus position by 30 (step S16).
It is determined that the full-press operation signal has been input (step S
18), the quick return mirror 3 is flipped up, and a shooting sequence subsequent to step S19 is executed. On the other hand, when it is determined that the full-press operation signal is not input (N in step S18), the process returns to step S1.

In step S19, the shutter plate 208
Is set on the optical path, and based on the outputs from the image sensor 214a and the image sensor 214b, the shutter opening time and the aperture value are calculated so as to obtain an appropriate exposure (step S2
0). Then, the amplification factor for amplifying the image signal in the image processing units 431a and 431b is changed from the set value at the time of focus detection to a predetermined set value at the time of shooting. Based on the value calculated in step S20, the opening 215 on the diaphragm plate 215
Any of a to 215h is set on the optical path (step S21), and the image sensors 214a and 214b are exposed for a predetermined time (step S22). After the exposure is completed, in step S23, the complete light shielding portion 208a on the shutter plate 208
Are set on the optical path, and the image pickup devices 214a and 214
An image signal is read from b (step S24). The read image signal is subjected to predetermined image processing and synthesis by the DSP 433 as described above (step S2).
5) The data compressed according to the predetermined format is recorded on the memory card 424 (step S26).
9 are completed.

In the above description, when the focus is detected, the image sensor 2
When reading the subject image data detected on 14a and 214b and corresponding to the focus detection area, a total of 100 pixels consisting of 25 pixels in the vertical direction and 4 pixels in the horizontal direction with respect to the object scene
Although the subject image data is read for each pixel group of pixels, when the direction of pupil division is changed, the arrangement of pixels constituting the pixel group may be changed according to the direction of pupil division. That is, in the case of dividing the object field in the vertical direction, the configuration of the pixel group is set to 4 pixels in the vertical direction and 25 pixels in the horizontal direction, and the interval between adjacent pixel groups is set to be shifted by two pitches in the vertical direction. .

-Smear Reduction- The image pickup devices 214a and 214b are constituted by CCDs in which color filters of the same Bayer array are on-chip. By the operation of the polarization beam splitter 219, a mirror image of an image incident on the image sensor 214a is projected on the image sensor 214b. FIG. 3 shows how “P” is imaged on each of the imaging elements 214a and 214b when “P” is imaged on the primary image plane 16. Therefore, the image data captured by the image sensor 214b is rearranged in the reverse order of the pixels after the image is read, and is arranged in the pixel data of the image data captured by the image sensor 214a. FIG. 10A shows the arrangement of the pixels of the imaging element 214a, and FIG. 10B shows the arrangement of the pixels of the imaging element 214b. In the drawing, the arrangement of the pixels of G color (for example, G112 and G121) of the image sensor 214a is at a position where they interpolate with the arrangement of the pixels of G color (G211) of the image sensor 214b. Therefore, by synthesizing the image data based on the outputs from the two image sensors 214a and 214b,
G-color pixels can be obtained at all pixel positions. Similarly, the R color and the B color correspond to two image sensors 2
By synthesizing the image data based on the outputs from 14a and 214b, pixels of each color can be obtained every other vertical line.

The image sensor 214 constructed as described above
The reading directions of the pixels a and 214b are indicated by arrows in FIG. Although the imaging devices 214a and 214b are the same CCD, the order of the pixels shown in FIG. 10B is rearranged, so that the arrows indicating the readout directions in FIGS. 10A and 10B are reversed. Has become. In the figure, pixels marked with a circle (G152, R153, B16 in FIG. 10A)
2, G163 and R252, G25 in FIG.
3, G262, B263), when the accumulated electric charge is saturated and overflows due to the intense light incident thereon, the electric charge accumulated in each pixel is sent in the reading direction at the time of reading the image signal. , And an overflowing charge is added to the amount of charge passing through the portion marked with ○ to generate smear.

When the image signal of the image sensor 214a is read, the pixel position (G1
54, R155, G156, B164, G165, B1
When the charge amount corresponding to (66) passes through the pixel position marked with a circle, smear occurs. Similarly, the image sensor 21
When the image signal of FIG. 4B is read, the amount of charge corresponding to the pixel position (G251, B261) marked with a triangle in FIG. 10B passes through the pixel position marked with a circle, and smear occurs. Occurs.

In FIGS. 10 (a) and 10 (b), the pixel position where smear has occurred can be specified by comparing the pixel positions of ○ and Δ where the pixel output is saturated. That is, when the pixel output of one image sensor is saturated, the pixel output of the other image sensor corresponding to the mark O where strong light is incident is saturated, and smear occurs.
The pixel output of the other image sensor corresponding to the mark is not saturated.
By utilizing this phenomenon, the position of a pixel where smear has occurred is grasped, and the pixel output of one image sensor saturated by smear is interpolated with the other pixel output that is not saturated, thereby reducing the influence of smear on image data. .

The above-described smear reduction processing will be described with reference to the flowcharts of FIGS. FIG.
FIG. 12 is a flowchart illustrating a smear reduction process of the image sensor 214a, and FIG. 12 is a flowchart illustrating a smear reduction process of the image sensor 214b. These processes are performed after the photographing process is performed. Step S20
In 1 and S202, the counter of X indicating the pixel position in the horizontal direction and the counter of Y indicating the pixel position in the vertical direction are reset to 1. The pixel position (1,
It is determined whether the A / D conversion value of the pixel output at (X, Y) is equal to or greater than the substantially saturated level of 250 in 8-bit conversion (step S203). The pixel position of the image sensor 214a is represented by (1, X, Y), and the corresponding pixel position on the image sensor 214b is represented by (2, X, Y).

The pixel position (1, X, Y) of the image sensor 214a
Is determined to be less than 250 (N in step S203), X indicating the pixel position in the horizontal direction is advanced by one (step S204), and X is shifted in the horizontal direction of the image sensor. It is determined whether the number is equal to or less than the number of pixels (step S205). If it is determined that X is equal to or less than the number of pixels in the horizontal direction (Y in step S205), step S2
Return to 03. On the other hand, if it is determined that X exceeds the number of pixels in the horizontal direction (N in step S205), Y indicating the pixel position in the vertical direction is advanced by one (step S206).
Is smaller than or equal to the number of pixels in the vertical direction of the image sensor (step S207). If it is determined that Y is equal to or less than the number of pixels in the vertical direction (Y in step S207), the process returns to step S202. On the other hand, if it is determined that Y exceeds the number of pixels in the vertical direction (N in step S207), the process ends.

In step S203, the image sensor 21
If it is determined that the A / D conversion value of the pixel output at the pixel position (1, X, Y) at 4a is 250 or more (Y at step S203), the corresponding pixel position (2, 2) on the image sensor 214b is determined at step S208. It is determined whether the A / D conversion value of the pixel output of (X, Y) is, for example, 200 or less in 8-bit conversion. If it is determined that the pixel output data exceeds 200 (N in step 208), it is determined that the image is not a smear phenomenon, and the process returns to step S204. On the other hand, if the pixel output data is 2
If the difference is determined to be equal to or less than 00 (Y in step S208), the difference between the (1, X, Y) pixel output data of the image sensor 214a and the (2, X, Y) pixel output data of the image sensor 214b is a smear. It is determined to be due to the phenomenon. Then, in step S209, the (1, X, Y) pixel of the image sensor 214a is G
It is determined whether it is a color. If it is determined that the color is not the G color (N in step S209), the process proceeds to step S211. If it is determined that the color is the G color (Y in step S209), the process proceeds to step S21.
Go to 0.

In step S210, the image sensor 214b
The average of the four G color pixel output data arranged on the upper, lower, left, and right of the corresponding pixel above is calculated, and the product of the calculated average value and the correction coefficient f1 saturates the image sensor 21 which is saturated.
The pixel output data of (1, X, Y) 4a is replaced. For example, in FIGS. 10A and 10B, if the pixel of the saturated image sensor 214 a is 165 of G color, the image sensor 21
G264, G266, G255 and G275 on 4b
Of the four pixel output data and the correction coefficient f1 are calculated. The correction coefficient f1 is determined by the sensitivity difference between the image sensor 214a and the image sensor 214b and the polarization beam splitter 219.
Is provided to correct the difference between the transmittance and the reflectance of the image. In the polarization beam splitter, if the angle of the incident light beam is different, the spectral transmittance may change. Therefore, the correction coefficient f1 is prepared as table data so as to correspond to each pixel on the image sensor.

In step S211, the image sensor 214a
It is determined whether or not the pixel of (1, X, Y) is B color. When it is determined that the color is not the B color (N in step S211), the process proceeds to step S213, and the color is determined as the B color (step S21).
(Y of 1) and the process proceeds to step S212. Step S212
In, the average of the two B-color pixel output data arranged above and below the corresponding pixel on the image sensor 214b is calculated,
The product of the calculated average value and the correction coefficient f2 replaces the (1, X, Y) pixel output data of the saturated image sensor 214a. For example, in FIGS. 10A and 10B, if the pixel of the saturated image sensor 214a is 124 of B color, B223 and B225 on the image sensor 214b
The product of the average value of the two pixel output data and the correction coefficient f2 is calculated. The correction coefficient f2 is the same as the correction coefficient f1,
Provided for correction between 14a and the image sensor 214b,
Table data is prepared so as to correspond to each pixel on the image sensor.

In step S213, the image sensor 214a
It is determined whether the pixel of (1, X, Y) is the R color. When it is determined that the color is not the R color (N in step S213), the process returns to step S204, and is determined as the R color (step S21).
3) and the process proceeds to step S214. Step S214
In, the average of the pixel output data of two R colors arranged above and below the corresponding pixel on the image sensor 214b is calculated,
The product of the calculated average value and the correction coefficient f3 replaces the (1, X, Y) pixel output data of the saturated image sensor 214a. For example, in FIGS. 10A and 10B, if the pixel of the saturated image sensor 214a is 175 of R color, B274 and B276 on the image sensor 214b
The product of the average value of the two pixel output data and the correction coefficient f3 is calculated. The correction coefficient f3 is provided for correction between the image sensor 214a and the image sensor 214b as in the case of f1 and f2, and is prepared as table data so as to correspond to each pixel on the image sensor.

When the processing of FIG. 11 is completed,
Is performed. Since the processing in FIG. 12 is similar to the processing in FIG. 11, only different points will be described. In step S303, the pixel position of the image sensor 214b
If the A / D conversion value of the pixel output of (2, X, Y) is determined to be 250 or more (Y in step S303), step S3
08, the corresponding pixel position (1, 1) on the image sensor 214a
It is determined whether the A / D conversion value of the pixel output of (X, Y) is, for example, 200 or less in 8-bit conversion. If it is determined that the pixel output data exceeds 200 (N in step 308), it is determined that the pixel output data is not a smear phenomenon, and step S30 is performed.
Return to 4. On the other hand, if the pixel output data is determined to be 200 or less (Y in step S308), the image sensor 214b
(2, X, Y) pixel output data and the image sensor 214a
The difference from the pixel output data of (1, X, Y) is determined to be due to the smear phenomenon. Then, in step S309, it is determined whether or not the (2, X, Y) pixel of the image sensor 214b is the G color. If it is determined that the color is not the G color (N in step S309), the process proceeds to step 3211. If it is determined that the color is the G color (Y in step S309), the process proceeds to step S310.

In step S310, the image sensor 214a
The average of the pixel output data of the four G colors arranged on the upper, lower, left and right sides of the corresponding pixel above is calculated, and the product of the calculated average value and the correction coefficient 1 / f1 indicates the saturation of the image sensor 214b that is saturated. The pixel output data of (2, X, Y) is replaced. For example, in FIGS. 10A and 10B, if the pixel of the saturated image sensor 214b is 244 of G color, G143, G145, G134, and G on the image sensor 214a.
154 of the four pixel output data and the correction coefficient 1 /
The product with f1 is calculated. The correction coefficient 1 / f1 is provided for correcting a sensitivity difference between the image sensor 214b and the image sensor 214a, a difference between the reflectance and the transmittance of the polarization beam splitter 219, and the like, and is a reciprocal of the correction coefficient f1 described above. In addition, the B color in steps S312 and S314,
The replacement process for the R color is performed in the same manner. 1 / f2 and 1 / f3 are used as the correction coefficients, respectively.

In the above description, the pixel output data of one image sensor determined to be a smear phenomenon is replaced by the product of the average value of the pixel output data of the other image sensor corresponding to the pixel and the correction coefficient. However, replacement processing may be similarly performed on pixel output data of two pixels around a pixel determined to be a smear phenomenon.

In the above description, the pixel output data of one image sensor determined to be a smear phenomenon is replaced with the product of the average value of the pixel output data of the other image sensor corresponding to the pixel and the correction coefficient. However, the pixel output data may be replaced with a predetermined value smaller than 250.

The actual imaging devices 214a and 214a
4b are arranged on the image planes conjugate to each other with a half pixel pitch shifted for pixel shift interpolation to be described later. However, in the above description of smear reduction, for the sake of simplicity, they are arranged at the same position without pixel shift. It was described as being provided.

-Configuration of Field Lens- Regarding the configuration (FIG. 1) in which the above-described field lens 209 is disposed between the quick return mirror 3 and the primary imaging surface 16, the following conditions for the field lens 209 in the present embodiment will be described. This will be described with reference to equations (1) to (4).

N (D)> 1.60 (1) D / f> 0.01 (2) R1 / f> 0.2 (3) βfL <1.0 (4) where N (D): D line of the field lens 209 (wavelength 5
87.6 nm), D: distance between the primary imaging surface 16 of the taking lens 1 and the field lens 209, f: focal length of the field lens 209, R1: radius of curvature of the field lens 209 on the object side, βfL: This is the imaging magnification of the field lens 209.

Conditional expression (1) satisfies the field lens 209
Is given. Field lens 209
Is disposed on the object side (left side in FIG. 1) with respect to the primary imaging plane 16, if this lens is constituted by one lens, it is constituted by a biconvex lens having a surface having a small radius of curvature directed toward the object side, and is distorted. The refractive index is increased to suppress the occurrence of various aberrations including the aberration. In the present embodiment, N (D) = 1.805. Conditional expression (2) gives the distance D between the field lens 209 arranged on the object side and the primary imaging plane 16. If the field lens 209 is too close to the primary image plane, dust or scratches will be imprinted on the image. To avoid this phenomenon, use a field lens 2
09 and the primary imaging plane 16 must be separated by a predetermined distance or more. In the present embodiment, D / f = 0.7 / 48.
87 = 0.0143.

Conditional expression (3) is satisfied with the field lens 209.
Is given to the object-side radius of curvature R1. In order to suppress various aberrations of the field lens 209, the radius of curvature R
Set 1 to a predetermined value or more. In order to sufficiently exhibit the effects of the present invention, it is desirable to set the lower limit to 0.5. In the present embodiment, R1 / f = 42 / 48.87 =
0.859. Conditional expression (4) gives the magnification of the field lens 209. In the present embodiment, β
fL = 0.894.

FIG. 13 shows an imaging optical system using the field lens 209 as described above. (a) is an imaging optical system according to the present embodiment in which a field lens 209 is arranged on the object side with respect to the primary imaging plane 16, and (b) is a field lens on the image side with respect to the primary imaging plane 16. 9 'is a conventional imaging optical system in which 9' is arranged. Note that reference numeral 213 in the drawing denotes a protective glass for protecting the image sensor 214a, and 223 denotes a spatial frequency filter (optical low-pass filter) for preventing color moire. The magnification β of the entire imaging optical system of FIG.
At 91, −0.28 of the magnification β ′ of the optical system according to FIG.
5, but the distance between the primary imaging plane 16 and the imaging element 214a, which is the final image plane position, is shortened.
This is because the optical system shown in FIG. 13A uses the magnification of the field lens at the reduction magnification (βfL = 0.894), while the optical system shown in FIG. 13B uses the magnification (βfL ′ = 1). .05
4). As a result, the magnification of the imaging lens 212 of the reduction optical system is βk = −0.326 in the optical system of FIG. 13A according to the present embodiment, and the magnification βk of the optical system in FIG. The size can be increased compared to '= −0.270.

In general, when the distance from the principal point of the lens to the object point is a and the distance from the principal point of the lens to the image point is b, the imaging formulas of the following equations (5) to (6) are known. ing.

1 / fg = (1 / a)-(1 / b) (5) βg = b / a (6) where fg is the focal length of the lens, and βg is the imaging magnification of the lens. In the above equations (5) and (6), the conjugate length L = (a + b), which is the distance between the object point and the image point, is minimized when βg = −1. In other words, the conjugation length L decreases as the imaging magnification βg approaches −1. For the above reasons, in the optical system of FIG. 13A according to the present embodiment, the field lens 209 is disposed on the left side of the primary
And the magnification of the imaging lens 212 is set to -1.
, The overall length of the imaging optical system is shortened.

FIG. 14 shows a table of specifications of the imaging optical system of FIG. 13A according to the present embodiment, and FIG. 15 shows various aberrations of the imaging optical system. For comparison, FIG. 16 shows a specification table of the imaging optical system in FIG. 13B, and FIG. 17 shows various aberration diagrams of the imaging optical system. In each aberration diagram shown in FIG.
The solid line in the spherical aberration diagram indicates the spherical aberration, the dashed line indicates the sine condition, and the solid line in the astigmatism diagram indicates the sagittal image surface, and the dashed line indicates the meridional image surface. The coma diagram shows the image height Y = 4.15,
Coma at 3.3, 2.0, 1.0, and 0.0 mm.
As can be seen from the aberration diagrams, various aberrations of the imaging optical system according to the present embodiment are favorably corrected regardless of the overall length of the imaging optical system.

In the above description of FIGS. 13 (a) and 13 (b), the field lens 209 is constituted by one lens, but may be constituted by two lenses. Two field lenses can be arranged on the object side and the image side with the primary imaging plane 16 interposed therebetween. In this case, the total length of the imaging optical system is reduced as compared with the case where one field lens is arranged on the image side of the primary imaging plane 16, and the distortion is smaller than when one field lens is formed. Become.

The features of the first embodiment will be summarized. (1) At the time of photographing, the luminous flux of the subject light is separated into transmitted light and reflected light by the polarizing beam splitter 219, and is made incident on the two image pickup devices 214a and 214b disposed on the conjugate imaging plane. Therefore, the separated subject luminous flux can be simultaneously imaged by the imaging elements 214a and 214b. (2) In pupil division at the time of focus detection, a polarization filter of a P polarization component and a polarization filter of an S polarization component are provided for the pupil to be divided, and the polarized focus detection subject light flux transmitted through the polarization filter is applied to the pupil. Two photographing image sensors 21 according to the
4a and 214b. Then, focus detection is performed by comparing image data of the AF areas of the imaging elements 214a and 214b.
Therefore, it is not necessary to provide a dedicated light receiving element for focus detection, so that the cost can be reduced. Furthermore, since the range in which the subject light flux for focus detection is received is wider than when a light receiving element smaller than the imaging element for photographing is used, the focus detection area can be widened.

(3) A pair of apertures 208c to 208j for pupil division at the time of focus detection are at +45 degrees and -45 with respect to the object field in the horizontal direction, the vertical direction, and the horizontal direction.
It is provided in the direction of degree so that any one pair of openings can be selected at the time of the focus detection operation. As a result, when it is difficult to detect the contrast from the subject image based on the selected opening, it is possible to select an opening provided in another direction and to detect the contrast of the subject image again. Thus, the effect that the focus detection operation is performed correctly can be obtained. (4) When calculating the relative distance between a pair of subject images obtained by pupil division, the image data detected on the imaging elements 214a and 214b is a pixel group consisting of 25 pixels in the vertical direction and 4 pixels in the horizontal direction, totaling 100 pixels And the total output was calculated for each pixel group. As a result, the noise included in the data can be reduced, and the pixel output of each of the RGB colors can be used as the data.

(5) An image signal processing unit 431 for processing image signals output from the image sensors 214a and 214b
In a and 431b, the image signal amplification factor is changed between focus detection and photographing. Thus, the effects of shortening the exposure time of the imaging elements 214a and 214b during focus detection and reducing noise during signal amplification during shooting can be obtained. Even if noise occurs during signal amplification during focus detection, noise is reduced by calculating the total output for each pixel group as described above, and the subject image during focus detection is recorded on the memory card 424 as a photographed image. It doesn't matter because it's not done. (6) When a predetermined exposure time elapses when exposing the image sensors 214a and 214b, the subject light is blocked by the shutter plate 208, so that smear caused by exposing the image sensors 214a and 214b when reading image signals is reduced. Can be minimized.

(7) An image sensor 214a composed of a CCD in which color filters of the same Bayer array are on-chip.
And 214b are arranged so that the subject image projected on the image sensor is mirror-symmetrical, so that the charge discharging direction when reading out the image signal is opposite to the captured image. As a result, based on the image signals output from the two image sensors, the image sensor in which smear has occurred and the smear-occurring pixel on the image sensor can be specified, and the corresponding image sensor on the other image sensor in which smear has not occurred can be specified. The pixel output in which smear has occurred can be corrected based on the pixel output to be performed. (8) By arranging the field lens 209 on the object side of the primary imaging plane 16, the imaging magnification of the field lens 209 is set to the reduction magnification, and the magnification of the imaging lens 212 is made closer to the equal magnification (−1). . As a result, the focal length on the object side can be reduced as compared with the conventional imaging optical system, so that the entire length of the imaging optical system can be shortened, and the digital camera can be downsized. Further, since the focal length on the image side can be increased, it is effective in correcting off-axis aberrations.

(9) Since the luminous flux of the object is smaller on the object side than on the primary imaging surface 16, the lens diameter of the field lens 209 to be arranged can be reduced. As a result, effects of miniaturization and cost reduction can be obtained. (10) Since the mounting angle θ of the mirror 210 is set to 55 degrees with respect to the horizontal plane, the horizontal projection length H4 of the mirror 210 can be shortened, and the subject light L (when the quick return mirror 3 is rotated to the position of 3 ′). Is reflected forward (leftward in FIG. 1) of the mirror 210. As a result, the mirror 211 and the reduction optical system 212 can be arranged in the space below the quick return mirror 3, and
The length H3 in the depth direction of the camera body 200 is shorter than when the mounting angle θ of the mirror 210 is 45 degrees,
This has the effect of downsizing the camera.

Second Embodiment Subsequently, the display L of the digital still camera 100 is described.
The display operation on the CD 220 will be described. The image data finally recorded on the memory card 424 by the digital still camera 100 includes two image sensors 214a and 214
14b is obtained by synthesizing the image data captured in step 14b. However, in order to display a viewfinder or a monitor on the digital still camera 100, the display LCD 220
Is provided, an image before shooting and an image after shooting can be displayed on the LCD 220. In the present embodiment, when the digital still camera 100 is set to the recording mode, display is performed based on an image signal output from one of the image sensors 214a and 214b, and the digital camera 100 is set to the playback mode. When it is displayed, display is performed based on the image signals output from both the image sensors 214a and 214b.

-Display of Image Data- FIG. 18 is a flowchart illustrating a process of displaying image data on the display LCD 220 in the digital still camera 100 according to the present embodiment. Step S40
At 1, it is determined whether the digital still camera 100 is set to a recording mode or a reproduction mode. The digital still camera 100 is one of the operation members 407
When it is determined that the recording mode has been set by the mode change switch (not shown) (recording mode side in step S401), it is determined in step S402 whether a half-press operation signal has been input. On the other hand, if it is determined that the digital still camera 100 is set to the playback mode (the playback mode side of step S401), the process proceeds to step S4.
Proceed to 11. If it is determined in step S402 that a half-press operation signal has been input (Y in step S402), the process proceeds to step S403. If it is determined that no half-press operation signal has been input (N in step S402), a half-press operation signal is input. Wait for

In step S403, image data based on the output of the image sensor 214a is taken into the DSP 433. The image data captured by the DSP 433 is temporarily stored in the buffer memory 436 once predetermined image processing including pixel interpolation processing for calculating data of R, G, and B colors at each pixel position on the image sensor 214a is performed. After being stored, it is stored in the frame memory 435 and displayed on the display LCD 22.
0 is displayed (step S404). The image displayed at this time is a monitor subject image before shooting called a through image. In step S405, the shutter opening time and the aperture value are calculated based on the output from the image sensor 214a so that an appropriate exposure is obtained.
The image sensors 214a and 214b are exposed for a predetermined time,
The two image data based on the outputs from the imaging elements 214a and 214b are compared and calculated, and the focus detection is performed by pupil division as described above (step S406). Then, based on the obtained focus detection data, the photographing lens 1
Is driven by the lens drive circuit 430 to the in-focus position.

When it is determined that the full-press operation signal has been input (Y in step S407), the quick return mirror 3 jumps up, and the photographing process in step S408 is executed. On the other hand, when it is determined that the full press operation signal is not input (N in step S407), step S402 is performed.
Return to In step S408, the opening 208b of the shutter plate 208 through which all light beams pass is set on the optical path, and any of the openings 215a to 215h on the diaphragm plate 215 is placed on the optical path based on the value calculated in step S405. When set, the image sensors 214a and 214b
Is exposed for a predetermined time.

In step S409, the image sensor 21
The DSP 433 performs predetermined image processing on the image signals read from 4a and 214b, and synthesizes the image signals read from the two image sensors into one image data. Note that the above-described image processing includes a pixel shift interpolation process which is arranged on a conjugate image plane and shifted by a half pixel pitch and is based on pixel outputs of the two image sensors 214a and 214b. The subject image after the image processing and the synthesis processing is displayed on display LCD 220. This display image is a photographed subject image called a freeze image. As the image data after the image processing, for example, data compressed in a predetermined format such as JPEG is recorded on the memory card 424.

At step S410, it is determined whether or not the recording / reproducing mode has been changed. If it is determined that the mode has been changed to the reproducing mode (Y of step S410), the process proceeds to step S411. When it is determined that the recording mode is maintained (N in step S410), the process returns to step S402. When the composite image data finally recorded on the memory card 424 is displayed on the display LCD 220 in step S411, it is determined whether or not a key operation has been performed to display another frame (step S412). . When it is determined that no key operation has been performed (N in step S412), the display performed in step S411 is continued, and when it is determined that the key operation has been performed (Y in step S412), the key operation is followed. The image of the frame, for example, the image of the first recorded frame is displayed on display LCD 220 (step S413). In step S414, it is determined whether or not the recording / playback mode has been changed again. When it is determined that the mode has been changed to the recording mode (Y in step S414), the process returns to step S402, and it is determined that the playback mode is maintained. At this time (N in step S414), the process returns to step S412.

The features of the second embodiment will be summarized. (1) When displaying a through-the-lens image before photographing, the through image is displayed on the display LCD 220 based on an image signal output from one of the two image sensors, and a frozen image after the image is displayed. In this case, since the image signals from the two image sensors are combined and displayed, the combining process of combining the image signals from the two image sensors is not required during live view display, and quick display is performed. be able to.
The number of pixels that can be displayed on the display LCD 220 is, for example, 520 dots in the horizontal direction × 220 dots in the vertical direction, and is generally larger than the pixel number of the image pickup device provided in the digital still camera 100 (for example, 1400 horizontal dots × 1240 vertical dots). Since the number is much smaller, displaying image data based on the output from either one of the image sensors does not pose a practical problem. (2) Since the image sensors 214a and 214b use CCDs having the same number of pixels and are arranged on a conjugate image plane with a shift of 1/2 pixel pitch, the image signals output from both image sensors are shifted by a pixel and interpolated. By performing (Step S409), the image data can be converted into image data having twice the number of pixels in the horizontal and vertical directions of the screen as compared with the number of pixels in the case of one image sensor. As a result, the number of pixels of the imaging device used can be apparently increased.

In the above description, in step S403, the image data based on the output of the
The image before shooting is displayed L
Although the image is displayed on the CD 220 (step S404) and the exposure calculation is performed (step S405), it may be performed based on the output of the image sensor 214b instead of the image sensor 214a.

In the above description, if the release button is held in the state of the half-press operation input, step S40
In step 6, the focus detection operation is repeatedly performed. However, when an operation is performed so as to maintain the state once the subject is focused, as in one-shot AF, step S406 may be omitted after focusing. In this case, only one of the image sensors needs to be driven after focusing, so that power consumption can be reduced.

Third Embodiment In the second embodiment, when the digital still camera 100 is set to the recording mode, a through image is displayed based on the output of one of the image sensors. When the reproduction mode is set, the freeze image display is performed based on the outputs of both the image sensors 214a and 214b. However, in the third embodiment, either the recording mode or the reproduction mode is set. Even if you
The display is performed based on the output of one of the image sensors.

FIG. 19 is a flowchart for explaining a process for displaying image data on the display LCD 220. The difference from the flowchart of FIG.
a, S409a, S411a and Step S413a
Therefore, these steps will be mainly described. In step S408a, the image sensors 214a and 214b
Is captured by the DSP 433. Each of the image data captured by the DSP 433 is subjected to predetermined image processing including pixel interpolation processing for calculating R, G, and B data at each pixel position on the imaging elements 214a and 214b. Buffer memory 4
36 respectively.

In step S409a, the image data after the image processing is compressed in a predetermined format such as JPEG, and is recorded on the memory card 424 for each image data based on the two image sensors. As the freeze image displayed on the display LCD 220, image data based on the output of the image sensor 214a used for displaying the through image in step S403 is used.

In step S 411 a, of the image data finally recorded on the memory card 424,
Image data based on 14a is displayed on the display LCD 220. In step S413a, step S412
The image of the frame according to the key operation detected in, for example,
Of the image data of the first recorded frame, the image sensor 21
4a is displayed on the display LCD 220.

The features of the third embodiment will be summarized. (1) The freeze image display after photographing is also performed by the two image sensors 2
Display is performed on the display LCD 220 based on an image signal output from one of the image sensors 14a and 214b. As a result, when image data based on two image signals is respectively recorded on a memory card or the like, there is no need to perform a combining process of combining image data output from the two imaging elements 214a and 214b for displaying a frozen image. That is, quick freeze image display can be performed. (2) In the second embodiment (step S409), 2
The image data based on the outputs of the two image sensors 214a and 214b are pixel-shifted and interpolated (step S409).
This requires four times the memory capacity. However, since image data based on two image sensors is recorded on the memory card 424 (step S409a), the amount of memory used can be reduced.

In the above description, in step S409a, the image data based on the outputs of the image sensors 214a and 214b are JPEG-compressed, respectively,
The data was recorded on the memory card 424. Instead of the above, the image data based on the output of the image sensor 214a is JPEG-compressed, and the image data based on the output of the image sensor 214b takes the difference from the image data based on the output of the image sensor 214a. By performing JPEG compression, both JPEG data are stored in the memory card 424.
May be recorded, or both recording methods may be selectable.

Fourth Embodiment FIG. 20 is a view showing a fourth embodiment, and is disposed behind the primary imaging surface 16 of the photographing lens 1 (in FIG. 20, obliquely right above). Reduction optical system 512, beam splitter 5
FIG. 19 is a perspective view of an imaging optical system including the imaging device 19 and imaging elements 214a and 214b. A shutter plate 508 and a diaphragm plate 515 are provided adjacent to each other inside the reduction optical system 512, and the shutter plate 508 and the diaphragm plate 515 are sandwiched between relay lenses 512a and 512b. The shutter plate 508 and the aperture plate 515 are formed in a disk shape, and are driven by step motors (not shown) provided at respective rotation centers of the disk, as in the first embodiment.

As shown in FIG. 20, the shutter plate 5
Reference numeral 08 denotes a complete light blocking portion 508a for blocking all light beams, an opening 508b for passing all light beams, and an opening 508c at a position shifted vertically, horizontally, and obliquely with respect to the optical axis.
To 508h. The openings 508a and 508b are used during normal imaging, and the openings 508c to 508h are used during a pupil division type focus detection operation.

On the other hand, the diaphragm plate 515 has a diaphragm opening 515.
a to 515 g are provided. With reference to the area of the aperture 515a through which all the subject light beams pass, the aperture areas of the aperture apertures 515b to 515g are equal to the apertures 515b to 515g.
The opening area is set so that the opening area becomes half in order up to 515 g. The exit side of the reduction optical system 512 (FIG. 20)
A beam splitter 519 having no polarization action is provided on the upper right of the relay lens 512b), and separates the incident light beam into transmitted light 1 and reflected light 4. Beam splitter 5
The light beam transmitted through the light source 19 is incident on the image sensor 214a, and the light beam reflected by the beam splitter 519 is reflected by the image sensor 21a.
4b.

The focus detection operation in the fourth embodiment is performed in a time-division manner by using openings 508c to 508h provided on the shutter plate 508 at positions shifted from the optical axis. For example, in the optical system shown in FIG. 20, observing an image transmitted through the openings 508e and 508f of the shutter plate 508 provided at a position deviated vertically with respect to the optical axis is performed by the action of the mirror 210 and the mirror 211. This is equivalent to dividing the field of view into two horizontally displaced pupils for observation. Therefore, first, a subject image based on the light beam incident from the opening 508e is read out from the image sensor 214b, subjected to predetermined image processing by the DSP 433, and temporarily recorded in the buffer memory 436.

Next, the shutter plate 508 is rotated to read out the subject image by the light beam incident from the opening 508f from the image pickup device 214b, perform predetermined image processing by the DSP 433, and temporarily record it in the buffer memory 436. The two subject images recorded in the buffer memory 436, that is, the subject image captured by the image sensor 214b based on the light beam incident from the opening 508e and the opening within the predetermined specific AF area. 5
08f based on the light beam incident from the imaging element 214b
The focus detection operation according to the fourth embodiment is performed by calculating the interval between a pair of subject images formed of the subject images captured in step (1). The process of detecting the interval between a pair of subject images is performed in the same manner as the process in the first embodiment.

When the focus detection operation is performed in a time-division manner as described above, the digital still camera 100 or the subject may move due to the elapse of time, resulting in blurring. When blur occurs, the focus adjustment state of the photographing lens 1, that is, the shift amount cannot be accurately obtained. In order to prevent the focus detection failure due to such blurring, first, as described above, the opening 508e of the shutter plate 508 is formed.
The imaging lens 1 is determined based on the distance between the subject image detected on the image sensor 214b based on the light beam incident from the camera and the subject image detected on the image sensor 214b based on the light beam incident from the opening 508f. Is calculated. Next, the subject image detected on the image sensor 214b based on the light beam incident again from the opening 508e is recorded in the buffer memory 436.
The focus shift of the photographic lens 1 is calculated again based on the distance from the subject image detected on the image sensor 214b recorded in the image sensor 214b. Then, a focus detection operation is performed by taking the average of the two obtained shift amounts as the shift amount in this embodiment.

Next, display of image data according to the fourth embodiment will be described. Image sensors 214a and 2
14b, there is a difference in the amount of incident light due to the difference between the transmittance and the reflectance of the beam splitter 519. That is, the ratio of the transmittance to the reflectance of the polarizing beam splitter 219 according to the first embodiment is larger than the transmittance and the reflectance is 4. Therefore, the image pickup device 214a
A larger amount of light is incident, and a smaller amount of light is incident on the image sensor 214a than on the image sensor 214b. Therefore, when photographing a subject having a large difference in luminance, the image of the dark part uses image data from the imaging element 214b having a large amount of light,
One image data is synthesized from the image of the portion of the image sensor 214b where the amount of light is large and overexposure occurs due to the small amount of light and the image data of the image sensor 214a that is less likely to overexpose. The synthesized image data is displayed on the display LCD 220 as a through image before photographing.

FIG. 21 is a flowchart for explaining image display processing on the display LCD 220 in the digital still camera 100 according to the fourth embodiment. In step S501, it is determined whether the digital still camera 100 is set to a recording mode or a reproduction mode. The digital still camera 100 is operated by the operation member 407
It is determined that the recording mode has been set by a mode changeover switch (not shown) which is one of the above (step S501).
) And the luminance measurement is performed in step S502 irrespective of the input of the half-press operation signal.
It is determined whether or not the luminance measured based on the output signal from a is equal to or greater than a predetermined value (step S503). On the other hand, when it is determined that the digital still camera 100 is set to the playback mode (the playback mode side of step S501), the process proceeds to step S507. When it is determined that the luminance measured based on the output signal from the image sensor 214a is equal to or greater than a predetermined value (Y in step S503), the beam splitter 51
Imaging element 21 which is arranged on the transmission side of 9 and has a small amount of incident light
The image data based on the output of 4a is displayed on the display LCD 220 as a through image before photographing (step S5).
04), and the process proceeds to step S506.

In step S503, the image sensor 21
When it is determined that the luminance measured based on the output signal from 4a is less than the predetermined value (N in step S503), the luminance is determined based on the output of the imaging element 214b that is disposed on the reflection side of the beam splitter 519 and has a large incident light amount. Image data
It is displayed on the display LCD 220 as a through image before shooting (step S505). In step S506, when the above-described half-press operation signal is input, the image sensor 214a
Alternatively, a focus detection operation is performed based on the image data output from the 214b, and when a full-press operation signal is input, shooting is performed and recording processing on the memory card 424 is performed. In the display of a frozen image after photographing, as in the second embodiment, image data in which an image signal output from the image sensor 214a and an image signal output from the image sensor 214b are combined is displayed. When the recording process ends, the process according to this flowchart ends.

In step S507, when it is determined that the digital still camera 100 is in the reproduction mode (the reproduction mode side of step S501), the digital still camera 100 performs image processing based on the output signals from the image pickup devices 214a and 214b.
Image data recorded on the memory card 424 is displayed on the display LCD 220. In step S508,
A reproduction key input process for displaying another frame is performed. That is, when a key operation for displaying another frame is performed, the image data of the frame according to the key operation is read and displayed on the display LCD 220. If the reproduction operation is not performed, the processing according to this flowchart ends.

The features of the fourth embodiment will be summarized. (1) changing the amount of subject light incident on the two image sensors, displaying an image on the image display LCD 220 before shooting based on the output from one of the image sensors selected according to the brightness of the subject, The focus detection operation is performed in a time sharing manner.
As a result, it is possible to prevent the occurrence of overexposure and smear due to the high brightness of the subject in the through image display before shooting, the occurrence of black blur due to the low brightness of the subject, and the detection of the shift amount during focus detection. It will be easier. (2) In the recording mode, a through image before photographing is displayed on the display LCD 220 based on an image signal output from one of the two image sensors. This eliminates the need for a combining process such as when combining and displaying image signals from the camera, thereby enabling quick image display. Before shooting in the recording mode,
Since only the image sensor on the display side needs to be driven, power consumption can be reduced.

In the above description, the subject image detected on the image sensor 214b based on the light beam incident from the opening 508e and the subject image detected on the image sensor 214b based on the light beam incident from the opening 508f. An interval between a pair of subject images composed of images is detected. On the other hand, a subject image detected on the image sensor 214a based on the light beam incident from the opening 508e and the opening 508
The interval between a pair of subject images composed of subject images detected on the image sensor 214a may be detected based on the light beam incident from f.

Further, in the above description, the transmittance and the reflectance of the beam splitter 519 are set to 1: 4, but may be set to a larger ratio. For example, if the transmittance is set to 1 and the reflectance is set to 10, the possibility of occurrence of smear in the image sensor 214a can be suppressed to a very low level. In such a case,
The smear reduction processing described in the first embodiment is the same as that in FIG.
Need only be performed on the imaging element 214b side shown in FIG.

Further, in the above description, the ratio between the transmittance and the reflectance of the beam splitter 519 is made different so that the amount of light incident on the two imaging elements 214a and 214b is made different. When the difference between the reflectance and the reflectance is small, an ND filter or the like may be inserted in front of one of the image sensors for light reduction.

In the first to fourth embodiments, the digital still camera has been described. However, the smear reduction processing shown in FIGS. 11 and 12 may be performed in the form of software such as a CD-ROM or a floppy disk. It can also be stored as an image processing program in a storage medium and used when performing image processing on a personal computer. In this case, the image sensor 21
Based on the image signals output from 4a and 214b, both digitized image data were recorded on a large-capacity image data recording medium, respectively, and the recording medium was set on a personal computer to capture both image data. Above, the above-described image processing program performs the above-described smear reduction processing.

As described above, when image processing is performed on a personal computer, if image data recorded on the image data recording medium has already been subjected to smear reduction processing, the smear reduction processing is not performed again. Create a program in In this case, the smear reduction processing flag is also stored in the image data recording medium, and the processing for determining whether or not the image data has been subjected to the smear reduction processing when performing image processing on the personal computer is performed. Use as information to do. On the other hand, if the image data recorded on the image data recording medium has not been subjected to the smear reduction processing, a program is created so that the smear reduction processing is performed.
In this case, since the smear reduction processing flag is not set on the image data recording medium, the image data is identified as unsmear reduced unprocessed image data.

In the above description, a single-lens reflex digital still camera has been described. However, the present invention can be applied to a digital still camera whose lens cannot be replaced and a digital video camera which can also capture a moving image.

The correspondence between each component in the claims and each component in the embodiment of the invention will be described. The polarizing beam splitter 219 and the beam splitter 519 are optical means, and the image pickup device 214a is the first optical device.
The imaging device 214b serves as a second imaging device,
The CPU 439 corresponds to the detecting unit, the determining unit, and the correcting unit.

[0097]

As described in detail above, in the present invention,
The following effects are obtained. (1) According to the first and fourth aspects of the present invention, the first and second image pickup means respectively generate the first and second object images generated by optically separating the object image, and the output value is set to the A first pixel in the first imaging unit that is equal to or greater than a predetermined value of 1 and a second pixel in the second imaging unit corresponding to the pixel is detected.
It is determined whether the output value of the pixel is equal to or more than a predetermined value. Therefore, for example, when the smear has occurred, the output value of the second pixel does not exceed the second predetermined value, so that the occurrence of the smear can be detected. (2) According to the invention of claims 2 and 5, in addition to the configuration of claim 1, based on the determination result for the output value of the second pixel,
The output value of the first pixel is corrected so as to be smaller than the first predetermined value. Therefore, for example, when it is determined that the output value of the second pixel is not greater than or equal to the second predetermined value, it is determined that smear has occurred, and correction can be made so that the first pixel output value is reduced. As a result, in the image data obtained by the first imaging unit, there is an effect of preventing a portion corresponding to the first pixel from becoming too bright and whitening.

(3) According to the third aspect of the invention, in addition to the configuration of the first or second aspect, the first and second image pickup means are arranged so that the charge reading directions are opposite to each other. As a result, for example, the first and second
In the image data obtained by the image pickup means, the direction of the white streak appearing on the screen is reversed based on the charge reading direction, so that the smear can be reliably detected without adding a new circuit for detecting the occurrence of the smear. The effect is obtained.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration of a single-lens reflex digital still camera according to a first embodiment.

FIG. 2 is a view of the single-lens reflex digital still camera of FIG. 1 as viewed from the back.

FIG. 3 is a diagram illustrating an imaging optical system of the single-lens reflex digital still camera in FIG. 1;

FIG. 4 is a block diagram showing a circuit of the single-lens reflex digital still camera in FIG. 1;

FIG. 5 is a diagram illustrating the principle of a focus detection operation by pupil division.

FIG. 6 is a diagram illustrating a pixel group for reading a subject image for focus detection.

FIG. 7 is a diagram illustrating a relationship between a pixel group position and a total output of each pixel group.

FIG. 8 is a flowchart of a first half of a program started by a half-press switch.

FIG. 9 is a flowchart of the latter half of the program started by the half-press switch.

FIG. 10 is a diagram showing an arrangement of pixels on an image sensor;
(a) shows the arrangement of the image sensor 214a and (b) shows the image sensor 21.
4b.

FIG. 11 is a flowchart of a smear reduction process of the image sensor 214a.

FIG. 12 is a flowchart of a smear reduction process of an image sensor 214b.

13A and 13B are diagrams showing an image forming optical system including a field lens. FIG. 13A is a diagram in which a field lens is arranged on the object side from a primary image forming plane of a photographing lens, and FIG. It is arranged on the image side from the image plane.

FIG. 14 is a diagram showing a specification table of the imaging optical system according to the first embodiment of FIG.

FIG. 15 is a diagram illustrating various aberrations of the imaging optical system according to the first embodiment of FIG.

FIG. 16 is a diagram showing a specification table of the imaging optical system of FIG.

FIG. 17 is a diagram illustrating various aberrations of the imaging optical system of FIG.

FIG. 18 is a flowchart of image data display processing of the digital still camera according to the second embodiment.

FIG. 19 is a flowchart of image data display processing of the digital still camera according to the third embodiment.

FIG. 20 is a diagram illustrating an imaging optical system according to a fourth embodiment.

FIG. 21 is a flowchart of image data display processing of the digital still camera according to the fourth embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Photography lens, 3 ... Quick return mirror, 4 ... Focusing plate, 16 ... Primary imaging plane, 50 ... Finder device, 2
00: camera body, 208: shutter plate, 208a ...
Complete light-shielding portions, 208b to 208j opening, 209 field lens, 210, 211 mirror, 212 reduction optical system, 212a, 212b relay lens, 213
a, 213b: Protective glass, 214a, 214b: Image sensor, 215: Aperture plate, 215a to 215g: Opening,
219: polarization beam splitter, 220: display LCD,
223a, 223b: Optical low-pass filter, 308c to 308j: Polarizing plate, 407: Operating member, 424: Memory card, 433: DSP, 435:
Frame memory, 436 ... buffer memory, 439 ... C
PU, 508: shutter plate, 508a: complete light shielding part,
508b to 508h: opening, 512a, 512b: relay lens, 515: aperture plate, 515a to 515g: opening, 519: beam splitter

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Kiyoshige Shibazaki 3-2-2 Marunouchi, Chiyoda-ku, Tokyo Nikon Corporation F-term (reference) 2H054 AA01 5C022 AA13 AB03 AB12 AB28 AB51 AC02 AC03 AC12 AC42 AC53 AC54 AC56 AC69 AC74 5C065 AA03 BB02 BB08 BB11 BB24 CC01 DD02 DD18 DD20 EE01 EE02 EE05 EE06 EE12 EE14 EE18 EE20 FF02 GG13 GG17 GG18 GG23 GG30 GG31 GG32 GG44 GG50

Claims (5)

[Claims] An optical means for separating a subject image into a first subject image and a second subject image and forming them on first and second image forming surfaces, respectively; and the first and second image forming means First and second imaging means arranged on a plane to capture the first and second subject images, respectively, and a first pixel whose output value of the first imaging means is equal to or more than a first predetermined value And a determination unit that determines whether an output value of a second pixel of the second imaging unit corresponding to the first pixel is equal to or greater than a second predetermined value. And a digital camera. 2. The digital camera according to claim 1, wherein an output value of the first pixel is smaller than the first predetermined value based on a result of the determination by the determination unit.
A digital camera, comprising: a correction unit configured to correct an output value of the first pixel. 3. The digital camera according to claim 1, wherein said first and second image pickup means are charged based on the respective subject images projected on said both image pickup means. A digital camera, wherein, when reading out, the readout directions of the electric charges are opposite to each other in the two image pickup means. 4. The method according to claim 1, wherein the subject image is divided into first subject image data and second subject image data obtained by capturing the subject images on mutually conjugate planes and stored in a storage medium, respectively. An image signal processing storage medium storing an image processing program for reading out first and second subject image data and performing image processing, wherein a brightness value of the first subject image data is equal to or more than a first predetermined value. Detection processing for detecting the first data, and determination processing for determining whether the luminance value of the second data of the second subject image data corresponding to the first data is equal to or greater than a second predetermined value. A storage medium for processing an image signal, wherein a storage medium for storing an image signal is stored. 5. The image signal processing storage medium according to claim 4, wherein a luminance value of the first data is smaller than the first predetermined value based on a result of the determination. An image signal processing storage medium storing a program for performing a correction process for correcting the luminance value of the first data.