JP2010154310A - Compound-eye camera, and photographing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a compound-eye camera and a photographing method using the compound-eye camera in which, when creating one image by combining images photographed by using a plurality of imaging elements of different characteristics, deterioration in image quality caused by a difference in chip size between the imaging elements can be prevented. <P>SOLUTION: An imaging element 30-1 and a second imaging element 30-2 have the same size of pixels PIX, the chip size of the second imaging element 30-2 is greater than that of the first imaging element 30-1, and the second imaging element 30-2 is constituted of more pixels. A lens driving section 20-2 includes a lens switching function for bringing one of photographing lenses 18A-2 and 18B-2 selected in accordance with a mode in front of the second imaging element 30-2 and accommodating the other lens in a space between the first imaging element 30-1 and the second imaging element 30-2. During a compound-eye photographing mode, the photographing lens 18A-2 reduced in size for guiding subject light to an area inside the second imaging element 30-2 having the equal size and the equal number of pixels to an overall effective pixel region of the first imaging element 30-1 is brought to the front. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a compound eye camera and a photographing method, and more particularly to a compound eye camera having a plurality of different image sensors and a photographing method using the compound eye camera.

  In Patent Document 1, the number of effective pixels is larger than that of the first imaging unit having the first imaging optical system and the first solid-state imaging device, the second imaging optical system, and the first solid-state imaging device. A third imaging unit including a second imaging unit having a small second solid-state imaging device, a third imaging optical system, and a second solid-state imaging device having a smaller number of effective pixels than the first solid-state imaging device. An electronic trinocular camera device including an imaging unit is disclosed.

  In Patent Document 2, the number of effective pixels is higher than that of the first imaging unit having the first imaging optical system and the first solid-state imaging device, the second imaging optical system and the first solid-state imaging device. An electronic stereo camera including a second imaging unit having a small number of second solid-state imaging elements is disclosed.

Patent Document 3 discloses an imaging device that includes first and second CCDs for black and white and color, or having different element sizes, and that splits an optical signal from one optical lens and causes each CCD to receive light.
JP 2000-1112019 A JP 2000-102040 A JP 11-122536 A

  Conventionally, a multi-view camera including a plurality of image pickup optical systems and image pickup devices has been proposed, and it has been proposed to shoot and synthesize a plurality of images using such a multi-view camera. However, if there are differences in the characteristics of the plurality of image sensors (for example, saturation characteristics, sensitivity characteristics (characteristics indicating the relationship between light quantity and output), chip size, number of pixels, and pixel size), There is a risk that the image quality of the combined image will be deteriorated (for example, shading and color rotation will occur).

  Patent Documents 1 to 3 disclose a camera provided with a plurality of solid-state image sensors. When images obtained from solid-state image sensors having different numbers of pixels are synthesized, characteristics between the solid-state image sensors are disclosed. It is not disclosed how to prevent the deterioration of image quality due to the difference between the two.

  The present invention has been made in view of such circumstances, and when a single image is created by combining images captured using a plurality of image sensors having different characteristics, the difference in chip size between the image sensors. An object of the present invention is to provide a compound eye camera capable of preventing the deterioration of image quality caused by the above and a photographing method using the compound eye camera.

  In order to solve the above-described problem, a compound eye camera according to a first aspect of the present invention includes a first imaging unit including a first imaging element and an imaging lens, and the number of pixels as compared with the first imaging element. The second imaging element having the same pixel size and the pixel area of the second imaging element guides the subject light only to a partial area having the same number of pixels as the first imaging element. A second imaging unit comprising a partial irradiation imaging lens; and a full-irradiation imaging lens that guides the subject light over the entire pixel area of the second image sensor; the first imaging unit; In the case of a monocular imaging mode in which imaging is performed using only the second imaging means, imaging control means for controlling the second imaging means to capture an image and reading out an image signal, the second imaging Shooting lens for guiding the subject light to the element In the compound eye photographing mode in which photographing is performed using both the first photographing means and the second photographing means, a photographing lens that guides the subject light to the second imaging element. An imaging lens switching means for switching to the partial-illumination imaging lens, an image signal read from the first image sensor, and an image signal read from the second image sensor in the compound eye imaging mode And image synthesizing means for creating a synthesized image.

  According to the first aspect, when combining images captured using a plurality of imaging elements having different numbers of pixels, the pixel characteristics of the imaging elements are made the same, thereby obtaining saturation characteristics (amount of charge at saturation). ), Image quality deterioration caused by a difference in sensitivity characteristics (relationship between light quantity and output) can be eliminated, and the same size as the first image sensor in the pixel area of the second image sensor in the compound eye photographing mode. By using only signals from a part of the area, the difference in characteristics due to V shading, H shading and color shading can be eliminated.

  The compound-eye camera according to a second aspect of the present invention is the compound eye camera according to the first aspect, wherein the second image sensor transfers a signal charge accumulated in the pixels in a horizontal direction, An output amplifier connected to a horizontal transfer path, and the partial region is a region closest to the output amplifier in a pixel region of the second image sensor.

  According to the second aspect, by setting a partial region of the second image sensor to a region closest to the output amplifier, it is possible to prevent image quality degradation due to transfer residue on the horizontal transfer path.

  The compound-eye camera according to a third aspect of the present invention further includes drive control means for controlling driving of the second image sensor in the second aspect, and the second image sensor accumulates in the pixel. A vertical transfer path for transferring the signal charge in the vertical direction, and the drive control means is configured to perform the vertical transfer of the signal charge accumulated in the partial area in the compound eye photographing mode. The vertical transfer speed of the second image sensor is made faster than the vertical transfer of the signal charges accumulated in the partial area.

  According to the third aspect, it is possible to increase the speed at which the image signal of the second image sensor having a large number of pixels is vertically transferred in the compound eye photographing mode. The third aspect is effective for continuous shooting or moving image shooting.

  According to a fourth aspect of the present invention, there is provided a photographing method including a first photographing unit including a first imaging element and a photographing lens, a pixel number larger than that of the first imaging element, and a pixel size. A partially illuminating photographing lens that guides subject light only to a partial region of the same number of pixels as that of the first imaging device, out of the pixel region of the second imaging device, and the same second imaging device; A photographing method for photographing an image using a compound eye camera comprising: a second photographing unit including a whole surface irradiation photographing lens that guides the subject light over the entire pixel region of the second imaging element. In the monocular imaging mode in which imaging is performed using only the second imaging means, the first lens that switches the imaging lens that guides the subject light to the second imaging element to the imaging lens for whole surface irradiation. Before the switching step and the monocular mode A first imaging control step for controlling the second imaging means to take an image and reading out an image signal; and compound eye photography for taking an image using both the first imaging means and the second imaging means. A second lens switching step of switching a photographing lens that guides the subject light to the second imaging element to the partial irradiation photographing lens in the case of the mode, and the first lens in the case of the compound eye photographing mode. A second imaging control step of controlling the imaging means and the second imaging means to capture an image and reading an image signal; and an image read from the first image sensor in the compound eye imaging mode. An image synthesis step of synthesizing the signal and the image signal read from the second image sensor to create a synthesized image.

  The imaging method according to a fifth aspect of the present invention is the imaging method according to the fourth aspect, wherein the second imaging element includes a horizontal transfer path for transferring a signal charge accumulated in the pixel in a horizontal direction, An output amplifier connected to a horizontal transfer path, and the partial region is a region closest to the output amplifier in a pixel region of the second image sensor.

  In the imaging method according to the sixth aspect of the present invention, in addition to the configuration of the fifth aspect, in the compound eye imaging mode, the signal charge accumulated in the partial region of the second imaging element is A drive control step of increasing the vertical transfer speed of the second image sensor after the vertical transfer via the vertical transfer path of the second image sensor, compared to the vertical transfer of the signal charge accumulated in the partial region; In addition.

  According to the present invention, when combining images captured using a plurality of image sensors having different numbers of pixels, the pixel size of the image sensor is the same, so that saturation characteristics (charge amount at saturation), sensitivity Degradation of image quality due to a difference in characteristics (relationship between light quantity and output) can be eliminated, and a part of the pixel area of the second image sensor having the same size as the first image sensor in the compound-eye shooting mode By using only the signal from the area, it is possible to eliminate the difference in characteristics caused by V shading, H shading, and color shading. Furthermore, according to the present invention, by setting a partial area of the second image sensor to an area closest to the output amplifier, it is possible to prevent image quality deterioration due to transfer residue on the horizontal transfer path.

  Preferred embodiments of a compound eye camera and photographing method according to the present invention will be described below with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a block diagram showing a compound eye camera according to the first embodiment of the present invention. FIG. 2 is a plan view showing a first image sensor and a second image sensor of the compound eye camera according to the first embodiment of the present invention.

  As shown in FIG. 1, the compound-eye camera 10 includes a photographing lens 18-1, a lens driving unit 20-1, and a first photographing unit 12-1 including a first imaging element 30-1, and photographing lenses 18A-2 and 18B. -2, the 2nd imaging | photography part 12-2 containing the lens drive part 20-2 and the 2nd image pick-up element 30-2.

  The image sensors 30-1 and 30-2 are, for example, CCD (Charge Coupled Device) image sensors. It should be noted that as the imaging devices 30-1 and 30-2, other types of imaging devices (for example, CMOS (Complementary Metal Oxide Semiconductor) image sensors) can be used. As shown in FIG. 2, the first image sensor 30-1 and the second image sensor 30-2 have the same pixel (cell) PIX size, and the second image sensor 30-2 has the first image sensor 30-1. The chip size is larger and the number of pixels is larger. In the present embodiment, the number of pixels of the second image sensor 30-2 is an integral multiple of the number of pixels of the first image sensor 30-1. In one example, the first image sensor 30-1 has 6 million pixels, and the second image sensor 30-2 has 12 million pixels.

  As shown in FIG. 1, a photographing lens 18-1 is provided in front of the first image sensor 30-1. On the other hand, the second imaging element 30-2 is provided with two sets of photographing lenses 18A-2 and 18B-2. The photographic lens 18A-2 is configured to guide light from the subject only to a partial region of the image sensor 30-2 (region having the same pixel and size as the image sensor 30-1). The photographic lens 18A-2 can have the same configuration as the photographic lens 18-1 of the first photographing unit 12-1. On the other hand, the photographic lens 18B-2 is configured to guide light from the subject to the entire surface of the pixel region of the image sensor 30-2 (including the regions A10 and A12 in FIG. 3).

  The lens driving unit 20-2 extends one of the photographing lenses 18A-2 and 18B-2 selected according to the mode in front of the second image sensor 30-2 and the other as the first image sensor 30-1. It has a lens switching function for storing in the space between the second image sensor 30-2.

  For example, in the mode of photographing with a single eye using the second photographing unit 12-2 with high pixels and high image quality, as shown in FIG. The small imaging lens 18A-2 is extended forward of the element 30-2, and is accommodated in a space between the first imaging element 30-1 and the second imaging element 30-2. This makes it possible to capture a high-quality image using the entire surface of the image sensor 30-2 having a large chip size and a large number of pixels in the monocular imaging mode.

  On the other hand, in the case of a mode for photographing with a compound eye using both the first photographing unit 12-1 and the second photographing unit 12-2 (for example, in a 3D mode and a panoramic photographing mode described later, hereinafter referred to as a compound eye photographing mode) 1), as shown in FIG. 1B, the small photographic lens 18A-2 is extended forward of the second image sensor 30-2, and the photographic lens 18B-2 is connected to the first image sensor 30-1 and the second image sensor 30-1. It is housed in a space between the image sensor 30-2.

  FIG. 3 is a diagram for explaining a method of driving the first and second imaging elements during compound eye photography.

  As shown in FIG. 3, in the first photographing unit 12-1, light from the subject is guided to substantially the entire pixel area of the first image sensor 30-1 by the photographing lens 18-1.

  On the other hand, in the second photographing unit 12-2, light from the subject is guided only to a part of the effective pixel region A10 of the second imaging element 30-2 by the small photographing lens 18B-2 at the time of compound eye photographing. The effective pixel area A10 is set so that the number of effective pixels is the same as the pixel area of the first image sensor 30-1. The length (corresponding to the number of pixels) in the vertical direction (V direction) of the pixel region of the first image sensor 30-1 is a1, the length in the horizontal direction (H direction) is b1, and the second image sensor 30-2 The effective pixel area A10 is set so that a1 = a2 and b1 = b2, where the vertical length of the effective pixel area A10 is a2 and the horizontal length is b2.

  FIG. 4 is a graph showing the light amount-output characteristics of the first and second imaging elements. In FIG. 4, the horizontal axis represents the amount of incident light, and the vertical axis represents the level of the output signal of the image sensor (accumulated charge amount accumulated in one pixel). A curve L1 in FIG. 4 represents the light amount-output characteristic of the first image sensor 30-1, and a curve L2 represents the light amount-output characteristic of the second image sensor 30-2.

  In the present embodiment, since the pixel sizes of the first image sensor 30-1 and the second image sensor 30-2 are the same, as shown in FIG. 4A, the signal level in the saturation region of the characteristic curves L1, L2 Are equal. However, since the first image sensor 30-1 and the second image sensor 30-2 have different effective pixel numbers in the V direction and the H direction, as shown in FIG. 4A, V shading is performed in a linear region. Differences in characteristics due to H shading and color shading occur.

  Therefore, in the present embodiment, the effective pixel area A10 is set so that the number of effective pixels is the same as the pixel area of the first image sensor 30-1, thereby causing characteristics due to V shading, H shading, and color shading. To eliminate the difference. Furthermore, by setting the area closest to the output amplifier AMP2 connected to the horizontal transfer path HCCD2 of the second image sensor 30-2, the difference in characteristics caused by the transfer residue (residual charge) of the horizontal transfer path HCCD2 is eliminated. To do. As a result, as indicated by the symbol L ′ in FIG. 4B, the characteristic curves of both the image sensors 30-1 and 30-2 coincide with each other. It is possible to prevent deterioration in image quality when combined.

  In the compound eye photographing mode, unnecessary signal charges (image signals) accumulated in the region A12V adjacent in the vertical direction (V direction) to the effective pixel region A10 of the second image sensor 30-2 are transferred to the horizontal transfer path HCCD2. It is discarded without being transferred. Further, unnecessary image signals accumulated in the area A12H adjacent to the effective pixel area A10 in the horizontal direction (V direction) are once transferred by the horizontal transfer path HCCD2, and then discarded.

  FIG. 5 is a block diagram showing the main configuration of the compound eye camera according to the first embodiment of the present invention.

  As shown in FIG. 5, the compound eye camera 10 includes two first imaging units 12-1 and a second imaging unit 12-2. Two or more photographing units 12 may be provided.

  The compound-eye camera 10 synthesizes two pieces of image data photographed using the photographing units 12-1 and 12-2. For example, the compound eye camera 10 creates and displays an image for 3D display (stereoscopic display) by combining two pieces of parallax image data captured by the imaging units 12-1 and 12-2. The compound-eye camera 10 creates and displays a panoramic image by synthesizing image data for creating two panoramic images photographed by the photographing units 12-1 and 12-2. The compound-eye camera 10 stores the parallax image data and panoramic image creation image data in one recording image file (multi-picture file). In addition, the compound-eye camera 10 joins a plurality of frames of image data captured during bracket shooting, moving image shooting, or continuous shooting, and stores them in one recording image file (multi-picture file).

  A CPU (Central Processing Unit) 14 controls the operation of the compound eye camera 10 based on an input from the operation unit 16.

  The control bus 42 is a transmission path that transmits a command from the CPU 14 to each part of the compound eye camera 10. The data bus 44 is a transmission path for transmitting various data such as image signals.

  The main memory 48 stores programs executed by the CPU 14 and various data necessary for control, as well as various setting information relating to the operation of the compound eye camera 10 such as user setting information. The memory control unit 46 is an interface for reading data from the main memory 48, writing data to the main memory 48, and refreshing data in the main memory 48 according to a predetermined protocol.

  In the recording medium 58, the captured image file and various data are recorded. As the recording medium 58, for example, an SD memory card (registered trademark) or an xD picture card (registered trademark) can be used. The external memory control unit 56 is an interface for reading data from the recording medium 58 and writing data to the recording medium 58 according to a predetermined protocol.

  The operation unit 16 is an operation means for the user to perform various operation inputs, and includes a power switch for switching the power on and off, a mode dial for switching the operation mode of the compound eye camera 10, and a release. Includes buttons and zoom buttons.

  The mode dial is an operation means for performing an input to switch the operation mode of the compound eye camera 10, and in accordance with the setting position of the mode dial, a 2D mode for capturing a two-dimensional image (still image, moving image), a three-dimensional The operation mode is switched between a 3D mode for capturing an image (still image, moving image), a panorama shooting mode for capturing a panoramic image, and a playback mode for reproducing an image.

  The release button is composed of a two-stroke switch comprising so-called “half-press” and “full-press”. During still image shooting, when the release button is pressed halfway (S1 ON), shooting preparation processing (for example, automatic exposure adjustment processing (AE: Automatic Exposure), automatic focusing processing (AF: Auto Focus), automatic white balance adjustment) Processing (AWB: Automatic White Balance) is performed. When the release button is fully pressed (S2 is on), still image shooting / recording processing is performed. In addition, during movie shooting, movie shooting starts when the release button is fully pressed, and movie shooting ends when the release button is fully pressed again. Note that a release button for still image shooting and a release button for moving image shooting may be provided separately.

  The zoom button is an operation means for performing zooming operations of the photographing units 12-1 and 12-2, and includes a zoom tele button for instructing zooming to the telephoto side, and a zoom wide button for instructing zooming to the wide angle side. It has.

  The display unit 62 is a display device including a color liquid crystal panel (LCD), for example. The display unit 62 functions as a display unit that displays a captured image, and also functions as a user interface when performing settings related to various functions of the compound eye camera 10. The display unit 62 functions as an electronic viewfinder for confirming the angle of view in the shooting mode.

  The display unit 62 has a function of displaying a three-dimensional (3D) image that can be stereoscopically viewed by the user in the 3D mode. As a 3D image display method, for example, a light direction control system is employed. In the light direction control system, the image data for the left eye is displayed on the LCD, and the backlight panel irradiates the LCD with illumination light having directivity so as to reach the left eye of the user. A process of displaying image data for eyes on the LCD and irradiating the LCD with illumination light having directivity so as to reach the right eye of the user through the backlight panel (for example, at intervals of 1/60 seconds) )repeat. Thereby, the left-eye image and the right-eye image having parallax with each other are alternately observed by the left and right eyes of the user, so that the user can observe a stereoscopic image.

  The display controller 60 converts R, G, B color signals input via the data bus 44 into display signals.

  Next, the photographing function of the compound eye camera 10 will be described. In FIG. 5, the imaging units 12-1 and 12-2 are distinguished from each other by reference numerals 1 and 2. However, since the functions of the respective units are substantially the same, in the following description, reference numerals 1 and 2 are used. 2 will be omitted as appropriate.

  Each photographing unit 12 includes an imaging lens 18 and an imaging element 30. Between the photographic lens 18 and the image sensor 30, a diaphragm / mechanical shutter 22, an infrared cut filter 26, and an optical low-pass filter 28 are arranged.

  The imaging lens 18 includes a focus lens and a zoom lens. The focus lens and the zoom lens move back and forth along the optical axis of each photographing unit 12. The CPU 14 controls the operation of the lens driving unit 20 to adjust the position of the focus lens and perform focusing. Further, the CPU 14 controls the operation of the lens driving unit 20 to adjust the position of the zoom lens and perform zooming. In addition, the CPU 14 controls the amount of light incident on the image sensor 30 by adjusting the aperture (aperture value) of the diaphragm / mechanical shutter 22 by controlling the driving of the diaphragm drive unit 24. In addition, when reading data from the image sensor 30, the CPU 14 closes the aperture / mechanical shutter 22 to block light incident on the image sensor 30.

  The CPU 14 drives the imaging lenses 18-1 and 18-2 of the photographing units 12-1 and 12-2 in synchronization in the 3D mode and the panoramic photographing mode. In other words, the photographing units 12-1 and 12-2 are always set to the same focal length (zoom magnification) in the 3D mode and the panoramic photographing mode, and the diaphragm is adjusted so as to always have the same incident light amount (aperture value). . Further, in the 3D mode, focus adjustment is performed so that the same subject is always in focus.

  In the 3D mode, the CPU 14 controls a photographing unit drive mechanism (not shown) to obtain a relative position and a convergence angle (imaging lens) of the photographing units 12-1 and 12-2 so that parallax images corresponding to the left and right eyes can be obtained. The angle between the optical axes of 18-1 and 18-2) is adjusted. Further, the CPU 14 adjusts the relative positions and the convergence angles of the imaging units 12-1 and 12-2 so that a desired range can be imaged according to an input from the user in the panoramic imaging mode.

  The image sensor 30 is a CCD image sensor. A large number of light receiving elements (photodiodes) are two-dimensionally arranged on the light receiving surface of the image pickup element 30, and each photodiode has a predetermined arrangement of three color (for example, R, G, B) color filters. Is arranged. The pixel array is, for example, a Bayer array or a honeycomb array.

  When subject light is imaged on the light receiving surface of the image sensor 30 via the photographing unit 12, the subject light is converted into signal charges corresponding to the amount of incident light by the photodiode. The signal charge accumulated in each photodiode is a voltage signal (R, G, B image signal) corresponding to the amount of charge in accordance with a drive pulse given from the image sensor drive unit 32 in accordance with a command from the CPU 14. Are read sequentially. The image sensor 30 has an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.

  The analog signal processing unit 34 outputs a correlated double sampling circuit (CDS), R, G, B signals for removing reset noise (low frequency) included in the R, G, B signals output from the image sensor 30. An AGS circuit for amplifying and controlling to a certain level is included. Analog R, G, B signals output from the image sensor 30 are subjected to correlated double sampling processing and amplified by the analog signal processing unit 34. The amplification gains of the R, G, and B signals in the analog signal processing unit 34 correspond to shooting sensitivity (ISO sensitivity). The CPU 14 sets the photographing sensitivity by adjusting the amplification gain in accordance with the brightness of the subject. When an image is captured using two CCDs (in the 3D mode and the panorama shooting mode), the gain is set to the same value in the analog signal processing units 34-1 and 34-2.

  The analog R, G, B signals output from the analog signal processing unit 34 are converted into digital R, G, B signals by the A / D converter 36, and then the digital signal processing unit via the data bus 44. 50.

  The digital R, G, B signals generated as described above are subjected to predetermined processing (for example, synchronization processing (for example, spatial processing of color signals accompanying a color filter array of a single CCD) in the digital signal processing unit 50. The process of interpolating the shift and converting the color signal into a simultaneous expression), the white balance adjustment process, the gradation conversion (gamma correction) process and the contour correction process) are performed, and the luminance signal (Y signal) and the color difference signal (Cr , Cb signal), that is, Y / C signal.

  When a live view image (through image) is displayed on the display unit 62, the Y / C signal generated in the digital signal processing unit 50 is converted into R, G, B signals for each frame and then output to the display unit 62. Is done.

  Next, image capturing and recording processing will be described. When the CPU 14 detects that the release button is half-pressed in the still image shooting mode (S1 ON), the CPU 14 starts shooting preparation processing (AE processing and AF processing).

  The integrating unit 54 integrates digital R, G, B signals output from the photographing unit 12 in accordance with a command from the CPU 14 for each predetermined divided area, and outputs the integrated values of the R, G, B signals to the CPU 14. The CPU 14 calculates an exposure value (photographing EV value) based on the integrated values of the R, G, and B signals calculated by the integrating unit 54, and controls the aperture value and the shutter speed according to a predetermined program diagram.

  For example, the CPU 14 obtains a position where the high-frequency component of the G signal of the image signal is maximized as the in-focus position, and outputs a command to the lens driving unit 20 to move the focus lens to the in-focus position.

  When automatic white balance adjustment (AWB) is set, the integrating unit 54 calculates an average integrated value for each color of the R, G, and B signals for each predetermined divided area and outputs the average integrated value to the CPU 14. The CPU 14 determines the light source type based on the average integrated value of the R, G, and B signals calculated for each divided area, and controls the white balance gain for the R, G, and B signals.

  Further, the CPU 14 performs automatic light control by controlling the light emitting unit 38 and the light receiving unit 40 for flash.

  When the release button is fully pressed (S2 on) after the shooting preparation process is completed in response to the half-press of the release button (S1 on), the CPU 14 performs image shooting.

  In the 2D mode, a recording image is shot by a predetermined one shooting unit. Note that the user can select which imaging unit to use in the 2D mode. In the mode of shooting with a low number of pixels, shooting is performed by the first shooting unit 12-1, and in the mode of shooting with a high number of pixels (high image quality), shooting is performed by the second shooting unit 12-2. Is called.

  An image photographed by the designated photographing unit 12 in the 2D mode is compressed by the compression / decompression processing unit 52 and recorded on the recording medium 58 as an image file of a predetermined format. For example, JPEG (Joint Photographic Experts Group) is used for still images, and Moving Picture Experts Group (MPEG) 2 or MPEG4 is used for moving images. It is recorded as a compressed image file conforming to the H.264 standard.

  In the 3D mode and the panoramic shooting mode, images are shot in synchronization by the shooting units 12-1 and 12-2. In the 3D mode and the panorama shooting mode, the AF process and the AE process are performed based on the image signal acquired by one of the shooting units 12-1 and 12-2. The user may be able to select which imaging unit image signal is used for the AE process and the AF process.

  In the 3D mode and the panoramic shooting mode, the two viewpoint images shot by the shooting units 12-1 and 12-2 are respectively compressed by the compression / decompression processing unit 52 and stored in one multi-picture file. 58. The multi-picture file stores subject distance information, information about the distance between the photographing lenses of the photographing unit 12 and the angle of convergence, together with the compressed image data of two viewpoints.

  In the playback mode, a predetermined image file (for example, the last recorded image file) recorded on the recording medium 58 is read out and decompressed into an uncompressed Y / C signal by the compression / decompression processing unit 52. After being converted into R, G, B signals, it is output to the display unit 62. As a result, an image based on the image file recorded on the recording medium 58 is displayed on the display unit 62.

[Shooting process]
FIG. 6 is a flowchart showing a photographing process in the compound eye camera according to the first embodiment of the present invention.

  First, after the shooting preparation process is performed in response to half-pressing of the release button (S1 on), when the release button is fully pressed (S2 on), the image shooting process is started (step S10).

  In the monocular imaging mode (No in step S12, S14) and in the low pixel number mode for imaging with a low number of pixels (No in step S16), the first image sensor 30-1 is driven (step S18). And an image is image | photographed using the 1st imaging | photography part 12-1.

  Next, after the captured image is subjected to image processing including shading correction (step S24), the image is stored in an image file of a predetermined format and recorded on the recording medium 58 (step S26). In the case of the low-pixel number mode of the monocular imaging mode, in step S24, the output level (density level) of the image signal generated due to the characteristics of the first imaging element 30-1 and the influence of the aberration of the imaging lens 18-1 is uneven. Based on the shading correction data created based on the measured value of (shading), shading correction is performed by applying a gain to the output of the first image sensor 30-1.

  On the other hand, in the monocular imaging mode (No in step S12, S14) and in the high image quality mode for imaging with a high number of pixels (high image quality) (Yes in step S16), the second image sensor 30-2 is driven. At the same time (step S20), the photographing lens of the second photographing unit 12-2 is switched to the large photographing lens 18B-2 (step S22). And an image is image | photographed using the 2nd imaging | photography part 12-2.

  Next, after the captured image is subjected to image processing including shading correction (step S24), the image is stored in an image file of a predetermined format and recorded on the recording medium 58 (step S26). In the high image quality mode of the monocular imaging mode, in step S24, it occurs due to the influence of the characteristics of the entire pixel area (including A10 and A12) of the second imaging element 30-2 and the aberration of the large imaging lens 18B-2. Shading correction is performed by applying a gain to the output of the second image sensor 30-2 based on the shading correction data created based on the measured value of shading of the output level (density level) of the image signal. Is called.

  In the compound eye photographing mode (Yes in Step S12, S28), the photographing lens of the second photographing unit 12-2 is switched to the small photographing lens 18A-2 (Step S30). And an image is image | photographed using the 1st imaging | photography part 12-1 and the 2nd imaging | photography part 12-2 (step S32). In step S32, an image of the subject is formed only in a part of the effective pixel area A10 of the second image sensor 30-2 (an area having the same number of pixels as the pixel area of the first image sensor 30-1). Only the image signal accumulated in the area A10 is selectively read out, and the signal charge accumulated in the invalid pixel area A12 is discarded.

  Next, after the captured image is subjected to image processing including shading correction (step S24), the image is stored in an image file of a predetermined format and recorded on the recording medium 58 (step S26). In the compound eye photographing mode, in step S24, measurement of unevenness (shading) of the output level (density level) of the image signal caused by the characteristics of the first image sensor 30-1 and the aberration of the photographing lens 18-1. Based on the shading correction data created based on the value, a gain is applied to the output of the first image sensor 30-1, the characteristics of the effective pixel area A10 of the second image sensor 30-2, and the small photographic lens 18A- Gain to the output of the second image sensor 30-2 based on the shading correction data created based on the measured value of the unevenness (shading) of the output level (density level) of the image signal caused by the influence of the aberration of 2 The shading correction is performed by applying.

  According to this embodiment, when synthesizing images captured using a plurality of image sensors having different numbers of pixels, the pixel size of the image sensor is made the same, so that saturation characteristics (charge amount at saturation), Degradation of image quality due to a difference in sensitivity characteristics (relationship between light quantity and output) can be eliminated. In addition, according to the present embodiment, in the compound eye photographing mode, when photographing using the second imaging element 30-2 larger than the first imaging element 30-1, the same size as the first imaging element 30-1 is used. By using only the signal from the effective pixel area A10, the difference in characteristics due to V shading, H shading and color shading can be eliminated. Further, by setting the effective pixel area A10 at the time of compound eye photographing of the second image sensor 30-2 to an area closest to the output amplifier AMP2, it is possible to prevent deterioration in image quality due to the transfer residue of the horizontal transfer path HCCD2. it can.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the following description, the description of the same configuration as in the first embodiment is omitted.

  In the present embodiment, the speed at which the image signal of the second image sensor 30-2 having a large number of pixels is vertically transferred is increased in the compound eye photographing mode.

  FIG. 7 is a timing chart showing the flow of processing in the monocular imaging mode. FIG. 8 is a timing chart showing the flow of processing in the compound eye photographing mode.

  As shown in FIG. 7, since the second image sensor 30-2 has a larger number of pixels in the vertical direction (V direction), the time required for reading (V transfer) in the V direction than the first image sensor 30-1. Becomes longer.

  Therefore, in the compound eye photographing mode, as shown in FIG. 8, when the V transfer of the signal charge in the effective pixel region A10 is completed in the second image sensor 30-2 (the image signal of the invalid pixel region A12V in FIG. 3). Before reading starts), V high-speed transfer mode (a mode in which an image signal is transferred by an unnecessary charge sweeping pulse faster than a reading pulse at the time of V transfer, or a mode in which the charge of the second image sensor 30-2 is discarded to the substrate side ). Accordingly, when normal V transfer is performed (in the monocular imaging mode), the difference T1 between the time required for V transfer of the second image sensor 30-2 and the time required for V transfer of the first image sensor 30-1 is shorter. The V transfer in the second image sensor 30-2 can be completed at the time difference T2.

  FIG. 9 is a flowchart showing a photographing process in the compound eye camera according to the second embodiment of the present invention. Note that steps S40 to S52 in FIG. 9 are the same as steps S10 to S22 in FIG.

  First, after the shooting preparation process is performed in response to half-pressing of the release button (S1 on), when the release button is fully pressed (S2 on), the image shooting process is started (step S40).

  In the compound eye photographing mode (Yes in step S42, S58), the photographing lens of the second photographing unit 12-2 is switched to the small photographing lens 18A-2 (step S60).

  In the continuous shooting mode (Yes in step S62), after the driving mode of the second image sensor 30-2 is switched (step S64), the first photographing unit 12-1 and the second photographing unit 12-2 are switched. An image is photographed using this, and an image signal is read out (step S66). In the continuous shooting mode, as shown in FIG. 8, when the V transfer of the image signal in the effective pixel area A10 of the second image sensor 30-2 is completed, the speed is higher than the readout pulse during the V transfer. Unnecessary charge sweeping pulses are supplied to the second image sensor 30-2 to transfer the image signal.

  On the other hand, when it is not the continuous shooting mode (No in step S62), an image is shot using the first shooting unit 12-1 and the second shooting unit 12-2, and the image signal is read (step S66). .

  Next, the captured image is subjected to image processing including shading correction (step S54), and then the image is stored in an image file of a predetermined format and recorded on the recording medium 58 (step S56). In the compound eye photographing mode, in step S54, measurement of unevenness (shading) of the output level (density level) of the image signal caused by the characteristics of the first image sensor 30-1 and the aberration of the photographing lens 18-1. Based on the shading correction data created based on the values, gain is applied to the output of the first image sensor 30-1, the characteristics of the effective pixel area (A10) of the second image sensor 30-2, and the small photographic lens. The output of the second image sensor 30-2 based on the shading correction data created based on the measured value of the unevenness (shading) of the output level (density level) of the image signal generated due to the influence of the aberration of 18A-2, etc. By applying a gain to, shading correction is performed.

  According to the present embodiment, it is possible to increase the speed at which the image signal of the second image sensor 30-2 having a large number of pixels is vertically transferred in the compound eye photographing mode. This embodiment is effective for continuous shooting or moving image shooting.

1 is a block diagram showing a compound eye camera according to a first embodiment of the present invention. 1 is a plan view showing a first image sensor and a second image sensor of a compound eye camera according to a first embodiment of the present invention. The figure for demonstrating the drive method of the 1st and 2nd image pick-up element at the time of compound eye imaging | photography. The graph which shows the light quantity-output characteristic of a 1st and 2nd image sensor 1 is a block diagram showing the main configuration of a compound eye camera according to a first embodiment of the present invention. 6 is a flowchart showing imaging processing in the compound eye camera according to the first embodiment of the present invention. Timing chart showing the flow of processing in monocular shooting mode Timing chart showing the flow of processing in compound eye photography mode 7 is a flowchart showing photographing processing in a compound eye camera according to the second embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Compound eye camera, 12 ... Image pick-up part, 14 ... CPU, 16 ... Operation part, 18 ... Shooting lens, 20 ... Lens drive part, 22 ... Mechanical shutter combined with an aperture, 24 ... Diaphragm drive part, 26 ... Infrared cut filter, 28 DESCRIPTION OF SYMBOLS ... Optical low-pass filter, 30 ... Image pick-up element, 32 ... Image pick-up element drive part, 34 ... Analog signal processing part, 36 ... A / D converter, 38 ... Light emission part, 40 ... Light receiving part, 42 ... Control bus, 44 ... Data Bus ... 46 Memory control unit 48 Main memory 50 Digital signal processing unit 52 Compression / decompression processing unit 54 Integration unit 56 External memory control unit 58 Recording medium 60 Display control unit 62 ... Display section

Claims (6)

A first imaging means comprising a first imaging element and a photographic lens;
Of the second image sensor having the same number of pixels as the first image sensor and the same pixel size, and the pixel area of the second image sensor, the same number of pixels as the first image sensor. A second photographing unit comprising: a partial irradiation photographing lens that guides subject light only to a partial region; and a full irradiation photographing lens that guides the subject light over the entire pixel region of the second image sensor. When,
An imaging control unit that controls the first imaging unit and the second imaging unit to capture an image and read an image signal;
In the monocular imaging mode in which imaging is performed using only the second imaging unit, the imaging lens that guides the subject light to the second imaging element is switched to the imaging lens for whole surface irradiation, In the compound eye photographing mode in which photographing is performed using both the photographing means and the second photographing means, a photographing lens that guides the subject light to the second imaging element is used as the partial irradiation photographing lens. Photographing lens switching means for switching,
Image combining means for combining the image signal read from the first image sensor and the image signal read from the second image sensor in the compound eye photographing mode to create a composite image;
Compound eye camera equipped with. The second imaging device includes a horizontal transfer path for transferring the signal charge accumulated in the pixel in the horizontal direction, and an output amplifier connected to the horizontal transfer path,
The compound eye camera according to claim 1, wherein the partial region is a region closest to the output amplifier in a pixel region of the second image sensor. Drive control means for controlling the drive of the second image sensor;
The second image sensor includes a vertical transfer path for transferring signal charges accumulated in the pixels in a vertical direction,
When the vertical transfer of the signal charge accumulated in the partial area is completed in the compound eye photographing mode, the drive control unit is configured to perform the second control than the vertical transfer of the signal charge accumulated in the partial area. The compound-eye camera according to claim 2, wherein the vertical transfer speed of the imaging device is increased. A first imaging unit including a first imaging element and a photographic lens; a second imaging element having a larger number of pixels than the first imaging element and having the same pixel size; and the second imaging. A partially illuminating shooting lens that guides subject light only to a partial area of the same number of pixels as the first imaging element in the pixel area of the element; and the entire area of the pixel area of the second imaging element. A photographing method for photographing an image using a compound eye camera comprising a second photographing means comprising a whole-surface irradiation photographing lens for guiding subject light,
In a monocular imaging mode in which imaging is performed using only the second imaging means, a first lens switching that switches the imaging lens that guides the subject light to the second imaging element to the entire-surface irradiation imaging lens. Process,
A first imaging control step of controlling the second imaging means to capture an image and reading an image signal in the monocular imaging mode;
In a compound eye photographing mode in which photographing is performed using both the first photographing means and the second photographing means, a photographing lens for guiding the subject light to the second imaging element is used for the partial irradiation. A second lens switching step for switching to the taking lens;
A second imaging control step of controlling the first imaging unit and the second imaging unit to capture an image and read an image signal in the compound eye imaging mode;
An image combining step of combining the image signal read from the first image sensor and the image signal read from the second image sensor in the compound eye photographing mode to create a composite image;
A photographing method comprising: The second imaging device includes a horizontal transfer path for transferring the signal charge accumulated in the pixel in the horizontal direction, and an output amplifier connected to the horizontal transfer path,
The imaging method according to claim 4, wherein the partial area is an area closest to the output amplifier in a pixel area of the second image sensor.   During the compound eye photographing mode, the signal charge accumulated in the partial area of the second image sensor is vertically transferred through the vertical transfer path of the second image sensor and then accumulated in the partial area. The imaging method according to claim 5, further comprising a drive control step of increasing a vertical transfer speed of the second image sensor compared to a vertical transfer of signal charges.

Images