WO2011121840A1 - 立体撮像装置 - Google Patents
立体撮像装置 Download PDFInfo
- Publication number
- WO2011121840A1 WO2011121840A1 PCT/JP2010/070776 JP2010070776W WO2011121840A1 WO 2011121840 A1 WO2011121840 A1 WO 2011121840A1 JP 2010070776 W JP2010070776 W JP 2010070776W WO 2011121840 A1 WO2011121840 A1 WO 2011121840A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- imaging
- optical axis
- distortion correction
- image
- axis deviation
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/08—Stereoscopic photography by simultaneous recording
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B35/00—Stereoscopic photography
- G03B35/02—Stereoscopic photography by sequential recording
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/133—Equalising the characteristics of different image components, e.g. their average brightness or colour balance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/63—Control of cameras or camera modules by using electronic viewfinders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
Definitions
- the present invention relates to a stereoscopic imaging apparatus, and more particularly to a stereoscopic imaging apparatus that captures a plurality of images having parallax with each other.
- the stereoscopic imaging device captures the same subject from the left and right viewpoints using two imaging units arranged with parallax on the left and right, and acquires a left-eye image and a right-eye image, respectively.
- the left and right images acquired in this way are input to a 3D display capable of three-dimensional (3D) display and displayed so that they can be viewed separately by the left and right eyes so that they can be recognized as a stereoscopic image. Become.
- the imaging optical systems of the two imaging units usually have the same performance and characteristics, and are adjusted and incorporated with respect to the apparatus main body so that the optical axes of the respective imaging optical systems coincide with each other.
- storage means for storing the optical axis deviations of the two photographing optical systems for each zoom position, and the light corresponding to the zoom position of the photographing optical system at the time of photographing is provided.
- a stereoscopic imaging apparatus is proposed that reads out the axis deviation, converts the coordinates of one of the left and right images taken based on this optical axis deviation, and thereby matches the optical axis coordinates of the left and right images. (Patent Document 1).
- Patent Document 2 discloses a first lens barrel having a CCD for obtaining photographing information for the right eye, a second lens barrel having a CCD for obtaining photographing information for the left eye, and the like.
- a camera detection circuit for detecting the focal lengths of the first lens barrel and the second lens barrel, and the shift amounts of the optical axis centers of the first lens barrel and the second lens barrel at each focal length are stored in advance.
- a stereoscopic imaging device including a ROM composed of an EEPROM or the like and a CPU that controls an image cut-out area in at least one of the pair of left and right CCDs at each focal length based on an output from the ROM.
- Patent Document 3 an approximate expression for coordinate correction based on the lens center is set for the lens characteristics of the stereo camera, and the projection coordinates of the target image captured by the camera based on this approximate expression are set.
- An image processing apparatus for correction is disclosed.
- Patent Documents 1 and 2 the optical axis deviation due to individual differences between the two imaging optical systems is calculated based on the amount of optical axis deviation acquired in advance for each zoom position.
- a technique for correcting by converting coordinates or changing an image cutout range is disclosed, and Patent Document 3 discloses a technique for correcting image distortion due to lens distortion.
- the present invention has been made in view of such circumstances. Even if distortion correction with different correction accuracy is performed for each imaging mode for shooting a 3D moving image, a 3D still image, and the like, an image for stereoscopic viewing after distortion correction is performed. It is an object of the present invention to provide a stereoscopic imaging apparatus capable of obtaining a plurality of images that are easily viewed stereoscopically without any optical axis deviation.
- a stereoscopic imaging apparatus includes a plurality of imaging devices each having a photographing optical system and an image sensor that photoelectrically converts a subject image formed through the photographing optical system.
- a plurality of imaging units that capture a plurality of images having parallax with each other, and a first distortion correction formula that is set for each of a plurality of imaging modes and has different correction accuracy for each of the plurality of imaging modes.
- a distortion correction formula acquisition unit that acquires a distortion correction formula corresponding to the current imaging mode from among the storage unit, the distortion correction formula stored in the first storage unit, and the plurality of imaging units detected in advance The optical axis deviation amount of each imaging optical system, and the light after the distortion correction by the distortion correction formula set for each of the plurality of imaging modes
- a second storage unit that stores the shift amount; and an optical axis shift amount corresponding to the current imaging mode based on the optical axis shift amount of each imaging optical system stored in the second storage unit and the current imaging mode.
- the imaging control unit For the plurality of images acquired by the imaging control unit, the imaging control unit for acquiring a plurality of images from the plurality of imaging units according to the current imaging mode, The distortion correction unit that performs distortion correction based on the distortion correction formula acquired by the distortion correction formula acquisition unit corresponding to the current imaging mode, and the optical axis deviation amount acquisition unit acquired by the current imaging mode. And an image cutout unit that cuts out a stereoscopic display image from the plurality of images acquired by the imaging control unit based on the optical axis deviation amount. It is set to.
- the optical axis deviation amount acquisition unit acquires an optical axis deviation amount after distortion correction corresponding to the current imaging mode, and a plurality of images based on the acquired optical axis deviation amount. Therefore, even if distortion correction is performed using a distortion correction formula with different correction accuracy depending on the imaging mode, the optical axis is not affected by the distortion correction. Deviation correction can be performed.
- the zoom imaging apparatus further includes a zoom position detection unit that detects a current zoom position of the plurality of imaging optical systems
- the first storage unit includes: The distortion correction formula corresponding to the zoom position of the photographic optical system is stored, and the distortion correction formula acquisition unit obtains the first distortion correction formula corresponding to the current imaging mode and the current zoom position of the photographic optical system. It is obtained from the storage unit.
- the second storage unit is a light of each of the photographing optical systems according to a zoom position of the plurality of photographing optical systems.
- An axial deviation amount is stored, and the optical axis deviation amount acquisition unit acquires a corresponding optical axis deviation amount from the second storage unit according to a current imaging mode and a current zoom position of the imaging optical system. It is characterized by.
- the second storage unit uses each imaging mode and the imaging optical system as an optical axis shift amount of each imaging optical system.
- the optical axis deviation amount after distortion correction according to the distortion correction formula corresponding to the zoom position of each is stored according to each imaging mode and zoom position, and the optical axis deviation amount acquisition unit is configured to store the current imaging mode and current zoom position.
- a reading unit for reading out the corresponding optical axis deviation amount from the second storage unit.
- the second storage unit is an optical axis deviation amount before distortion correction as an optical axis deviation amount of each photographing optical system.
- the optical axis deviation amount acquisition unit acquires the optical axis deviation amount read based on the current zoom position from the second storage unit, and acquires the distortion correction expression.
- a calculating unit that calculates the amount of optical axis deviation after distortion correction by substituting into the distortion correction formula corresponding to the current imaging mode acquired by the unit.
- the second storage unit that stores the optical axis deviation amount after distortion correction by the distortion correction formula corresponding to each imaging mode and zoom position is read out, and in the case of the fifth aspect, the second storage unit that stores the optical axis deviation amount before distortion correction according to the zoom position.
- the read optical axis deviation amount is substituted into the distortion correction equation corresponding to the current imaging mode acquired by the distortion correction equation acquisition unit, and the optical axis deviation amount after distortion correction. Is calculated to obtain the amount of optical axis deviation after distortion correction.
- the second storage unit includes each imaging mode and the imaging optical system as an optical axis deviation amount of each imaging optical system.
- the information for calculating the amount of optical axis deviation after distortion correction by the distortion correction formula corresponding to the zoom position of each is stored for each imaging mode, and the optical axis deviation amount acquisition unit is in accordance with the current imaging mode. It has a calculating part which calculates the amount of optical axis deviation corresponding based on the information read from the 2nd storage part, and the present zoom position.
- Information for calculating the optical axis deviation after distortion correction corresponding to the zoom position includes, for example, the optical axis deviation after distortion correction at the two zoom positions of the wide end and the tele end, and these optical axis deviations.
- Information on the calculation formula of linear interpolation using the amount can be considered, and the optical axis deviation amount corresponding to the current imaging mode and zoom position can be calculated from this calculation formula and the current zoom position.
- a seventh aspect of the present invention is the stereoscopic imaging device according to the second aspect or the third aspect, further comprising a focus position detection unit that detects current focus positions of the plurality of imaging optical systems, and the second memory
- the unit stores the optical axis deviation amount after distortion correction by a distortion correction equation corresponding to each imaging mode and the zoom position of the imaging optical system according to each imaging mode, zoom position, and focus position, and the optical axis deviation amount
- the acquisition unit includes a reading unit that reads a corresponding optical axis deviation amount from the second storage unit in accordance with the current imaging mode, zoom position, and focus position.
- the seventh side surface has a second storage unit that stores the optical axis shift amount after distortion correction according to each imaging mode, zoom position, and focus position.
- the optical axis deviation amount corresponding to the current imaging mode, zoom position and focus position is read out.
- An eighth aspect of the present invention is the stereoscopic imaging device according to the second aspect or the third aspect, further comprising a focus position detection unit that detects a current focus position of the plurality of imaging optical systems, and the second memory.
- the unit stores information for calculating an optical axis deviation amount after distortion correction by a distortion correction equation corresponding to each imaging mode and the zoom position of the imaging optical system according to each imaging mode and zoom position,
- the optical axis deviation amount acquisition unit calculates a corresponding optical axis deviation amount based on the information read from the second storage unit according to the current imaging mode and the current zoom position and the current focus position. It is characterized by having.
- Information for calculating the amount of optical axis deviation after distortion correction by the distortion correction formula corresponding to each imaging mode and the zoom position of the imaging optical system includes, for example, distortion correction at two focus positions at the nearest and infinity.
- Information on the later optical axis misalignment amount and the calculation formula of linear interpolation using these optical axis misalignment amounts can be considered, and the current imaging mode, zoom position and focus position are supported from this formula and the current focus position.
- the amount of optical axis deviation to be calculated can be calculated.
- a ninth aspect of the present invention is the stereoscopic imaging device according to any one of the first to eighth aspects, further comprising a shading correction unit that performs shading correction of a plurality of images acquired by the imaging control unit,
- the image cutout unit is characterized in that the image cutout process is performed on the image subjected to the shading correction by the shading correction unit. Since the image cut-out process is performed after the brightness of the plurality of images is made uniform by shading correction, it is possible to obtain an image having no difference in brightness between the plurality of cut-out images.
- the image cutout unit performs an image cutout process on the image after distortion correction by the distortion correction unit. It is characterized by doing. Thereby, an image without distortion can be cut out from a plurality of images.
- the distortion correction unit performs distortion correction on the image cut out by the image cutout unit. It is characterized by.
- a twelfth aspect of the present invention is the stereoscopic imaging device according to any one of the first aspect to the eleventh aspect, wherein the plurality of imaging modes are an imaging mode during operation for displaying a live view image on a display unit, a still image It is characterized by being two or more imaging modes among an imaging mode, a moving image imaging mode, and a distortion-weighted imaging mode.
- a fish-eye imaging mode can be considered as the distortion-weighted imaging mode.
- the fish-eye imaging mode is also handled as a different imaging mode depending on whether it is a 3D moving image or a 3D still image.
- a continuous shooting mode for acquiring a plurality of time-series images from the plurality of imaging units during a preset number of images or a shooting instruction period is selected.
- an internal storage unit that temporarily stores an image being shot in the continuous shooting mode, and the shading correction unit stores a plurality of images stored in the internal storage unit after shooting in the continuous shooting mode is completed. It is characterized by reading and performing shading correction. By performing shading correction after the end of continuous shooting, a reduction in continuous shooting speed can be prevented.
- a mode selection unit that selects a shooting mode or a playback mode, and a shooting mode selected by the mode selection unit
- information indicating the imaging mode and information indicating the amount of optical axis deviation acquired by the optical axis deviation amount acquisition unit are associated with the plurality of acquired images.
- a recording section for recording on a recording medium, and the distortion correction section and the image cutout section are stored in association with the images together with the plurality of images from the recording medium in the playback mode selected by the mode selection section.
- the distortion correction formula and the optical axis corresponding to the read information for the plurality of read images.
- It is characterized by performing a clipping processing of the distortion correction and image based on the amount.
- the amount of processing during shooting can be reduced.
- high-definition 3D movies are shot and recorded at a high frame rate. can do.
- the recording unit records the image subjected to the distortion correction and the image cut-out process in the reproduction mode in the recording medium. It is said. Incidentally, the recording of the image after processing such as distortion correction may be overwritten on the image before processing or may be recorded separately.
- a parallax amount adjusting unit that adjusts a parallax amount between a plurality of images output from the plurality of imaging units.
- the image cutout unit performs image cutout processing in which the cutout position is further adjusted based on the amount of parallax adjusted by the parallax amount adjustment unit during the cutout processing of the image for stereoscopic display.
- the present invention even if distortion correction with different correction accuracy is performed for each imaging mode for capturing a 3D moving image, a 3D still image, and the like, there is no stereoscopic deviation between the stereoscopic images after distortion correction. A plurality of images easy to see can be obtained.
- the figure which shows the external appearance of the three-dimensional imaging device which concerns on this invention The figure which shows the external appearance of the three-dimensional imaging device which concerns on this invention 1 is a block diagram showing an embodiment of a stereoscopic imaging apparatus according to the present invention.
- the figure which shows an example of the table recorded on EEPROM at the time of the optical axis adjustment before the said shipment The figure which shows an example of the table recorded on EEPROM at the time of the optical axis adjustment before the said shipment
- the figure which shows an example of the table recorded on EEPROM at the time of the optical axis adjustment before the said shipment The figure which shows an example of the table recorded on EEPROM at the time of the optical axis adjustment before the said shipment
- the flowchart which shows embodiment which calculates the optical axis offset after distortion correction by calculation The figure which shows the table holding the optical axis deviation
- the flowchart which shows 2nd Embodiment of the image processing in FIG. A flowchart showing a first embodiment of image processing during continuous shooting.
- the flowchart which shows 3rd Embodiment of the image processing at the time of photography 1 is a flowchart showing a first embodiment of shooting processing of a stereoscopic imaging apparatus according to the present invention.
- FIG. 17 is a diagram used for explaining the image cut-out process of the second embodiment at the time of shooting / reproduction shown in FIG. 17 and FIG. The figure which shows a mode that the optical axis center of a right-and-left image shifts before and after distortion correction.
- FIG. 1 is a diagram illustrating an external appearance of a stereoscopic imaging apparatus according to the present invention
- FIG. 1A is a perspective view of the stereoscopic imaging apparatus as viewed from the front side
- FIG. 1B is a rear view.
- This stereoscopic imaging device (compound camera) 10 is a digital camera capable of recording and reproducing 2D / 3D still images and 2D / 3D moving images, and is a thin rectangular parallelepiped camera as shown in FIGS. 1A and 1B.
- a shutter button 11 and a zoom button 12 are disposed on the upper surface of the main body.
- a lens barrier 13 having a width substantially equal to the width in the left-right direction of the camera body is disposed so as to be movable in the up-down direction of the camera body.
- the front surfaces of the pair of left and right photographic optical systems 14-1 and 14-2 can be opened and closed simultaneously by moving in the vertical direction between the position indicated by and the position indicated by the solid line. Note that as the photographing optical systems 14-1 and 14-2, a zoom lens of a bending optical system is used.
- the camera power supply can be turned on / off in conjunction with the opening / closing operation of the lens front surface by the lens barrier 13.
- a 3D liquid crystal monitor 16 is disposed at the center of the back of the camera body.
- the liquid crystal monitor 16 can display a plurality of parallax images (right-eye image and left-eye image) as directional images each having a predetermined directivity by a parallax barrier.
- the 3D liquid crystal monitor 16 uses a lenticular lens, or can display a right eye image and a left eye image individually by wearing dedicated glasses such as polarized glasses or liquid crystal shutter glasses. Is applicable.
- the operation switch 18A is a switch for switching between still image shooting and moving image shooting
- the operation switch 18B is a parallax adjustment switch for adjusting the amount of parallax between the right-eye image and the left-eye image
- the operation switch 18C is 2D imaging. This is a changeover switch for switching between 3D imaging.
- the operation switch 18D is a seesaw key that functions as both a MENU / OK button and a playback button
- the operation switch 18E is a multifunction cross key
- the operation switch 18F is a DISP / BACK key.
- the MENU / OK button is an operation switch having both a function as a menu button for instructing to display a menu on the screen of the liquid crystal monitor 16 and a function as an OK button for instructing confirmation and execution of selection contents. It is.
- the playback button is a button for switching from the shooting mode to the playback mode.
- the cross key is an operation switch for inputting instructions in four directions, up, down, left, and right.
- a macro button, a flash button, a self-timer button, or the like is assigned to the menu key. When a menu is selected, the menu screen is displayed. Function as a switch (cursor moving operation unit) for selecting an item from the menu or instructing selection of various setting items from each menu.
- the left / right key of the cross key functions as a frame advance (forward / reverse feed) button in the playback mode.
- the DISP / BACK key is used for switching the display form of the liquid crystal monitor 16, canceling the instruction content on the menu screen, or returning to the previous operation state.
- 15 is a stereo microphone.
- FIG. 2 is a block diagram showing an embodiment of the stereoscopic imaging apparatus 10.
- the stereoscopic imaging apparatus 10 mainly includes a plurality of imaging units 20-1 and 20-2, a central processing unit (CPU) 32, the shutter button 11, the zoom button 12, and various operation switches described above. Including an operation unit 34, a display control unit 36, a liquid crystal monitor 16, a recording control unit 38, a compression / expansion processing unit 42, a digital signal processing unit 44, an AE (Automatic Exposure) detection unit 46, and an AF (Auto Focus): An automatic focus detection unit 48, an AWB (Automatic White Balance) detection unit 50, a VRAM 52, a RAM 54, a ROM 56, an EEPROM 58, and the like. Note that the imaging units 20-1 and 20-2 capture two parallax images, a left-eye image and a right-eye image, that have parallax with each other, but there may be three or more imaging units 20.
- the imaging units 20-1 and 20-2 capture two parallax images, a left-eye image and a right-eye image, that have par
- the imaging unit 20-1 that captures an image for the left eye includes an imaging optical system 14-1 (FIG. 1A) including a prism (not shown), a focus lens and a zoom lens 21, an optical unit including a diaphragm 22 and a mechanical shutter 23.
- the imaging unit 20-2 that captures the image for the right eye has the same configuration as the imaging unit 20-1 that captures the image for the left eye, and thus the description of the specific configuration is omitted.
- the CPU 32 controls the overall operation of the camera according to a predetermined control program based on the input from the operation unit 34.
- the ROM 56 stores a control program executed by the CPU 32 and various data necessary for the control.
- the EEPROM 58 stores various information indicating adjustment results at the time of adjustment before product shipment, for example, pixel defect information of the CCD 24, Correction parameters, tables, and the like used for image processing are stored. The details of various information stored here will be described later.
- the VRAM 52 is a memory for temporarily storing image data for display displayed on the liquid crystal monitor 16, and the RAM 54 includes a calculation work area for the CPU 32 and a temporary storage area for image data.
- the focus lens and zoom lens 21 included in the photographing optical system are driven by the lens driving unit 28 and moved back and forth along the optical axis.
- the CPU 32 controls the driving of the lens driving unit 28 to control the position of the focus lens so as to adjust the focus so that the subject is in focus, and in response to a zoom command from the zoom button 12 in the operation unit 34. Control the zoom position of the zoom lens to change the zoom magnification.
- the diaphragm 22 is configured by an iris diaphragm, for example, and is driven by the diaphragm driving unit 29 to operate.
- the CPU 32 controls the aperture amount (aperture value) of the aperture 22 via the aperture drive unit 29 and controls the amount of light incident on the CCD 24.
- the mechanical shutter 23 determines the exposure time in the CCD 24 by opening and closing the optical path, and prevents smear from occurring by preventing unnecessary light from entering the CCD 24 when the image signal is read from the CCD 24.
- the CPU 32 outputs a shutter close signal synchronized with the exposure end time corresponding to the shutter speed to the shutter control unit 30 to control the mechanical shutter 23.
- the CCD 24 is composed of a two-dimensional color CCD solid-state imaging device. A large number of photodiodes are two-dimensionally arranged on the light receiving surface of the CCD 24, and color filters are arranged in a predetermined arrangement on each photodiode.
- the optical image of the subject imaged on the CCD light receiving surface via the optical unit having the above configuration is converted into a signal charge corresponding to the amount of incident light by the photodiode.
- the signal charge accumulated in each photodiode is sequentially read out from the CCD 24 as a voltage signal (image signal) corresponding to the signal charge based on a drive pulse given from the CCD control unit 31 according to a command from the CPU 32.
- the CCD 24 has an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.
- the electronic shutter controls the charge accumulation start time corresponding to the shutter speed, and the exposure end time (charge accumulation end time) is controlled by closing the mechanical shutter 23.
- the CCD 24 is used as the image pickup device, but an image pickup device having another configuration such as a CMOS sensor may be used.
- the analog signals R, G, and B read from the CCD 24 are subjected to correlated double sampling (CDS) and amplification by the analog signal processing unit 25, and then the R, G, and B analog signals are output by the A / D converter 26. Converted to a digital signal.
- CDS correlated double sampling
- the image input controller 27 has a built-in line buffer having a predetermined capacity, and temporarily stores R, G, B image signals (CCDRAW data) A / D converted by the A / D converter 26 and then a bus 60. And stored in the RAM 54.
- the CPU 32 controls the imaging unit 20-2 that captures the image for the right eye in the same manner as the imaging unit 20-1 that captures the image for the left eye in the 3D imaging mode.
- the AE detection unit 46 calculates subject brightness necessary for AE control based on an image signal captured when the shutter button 11 is half-pressed, and outputs a signal indicating the subject brightness (shooting EV value) to the CPU 32.
- the CPU 32 sets the shutter speed (exposure time), aperture value, and imaging sensitivity in the plurality of imaging units 20-1 and 20-2 according to a predetermined program diagram based on the input imaging EV value.
- the AF detection unit 48 integrates the absolute value of the high frequency component of the image signal in the AF area captured when the shutter button 11 is half-pressed, and outputs this integrated value (AF evaluation value) to the CPU 32.
- the CPU 32 moves the focus lens from the closest position to the infinity side, searches for a focus position where the AF evaluation value detected by the AF detection unit 48 is maximum, and moves the focus lens to the focus position. Adjust the focus on the subject (main subject).
- so-called hill climbing control is performed in which the focus lens is moved so that the AF evaluation value always takes the maximum value.
- the AWB detection unit 50 automatically obtains the light source type (the color temperature of the object scene) based on the R, G, and B image signals acquired at the time of the main imaging, and R, G, The corresponding white balance gain is read out from the table storing the B white balance gain (white balance correction value).
- the digital signal processing unit 44 interpolates a spatial shift of color signals such as R, G, and B accompanying a white balance correction circuit, a gradation conversion processing circuit (for example, a gamma correction circuit), and a color filter array of a single-plate CCD.
- the image processing is performed on the R, G, and B image signals (CCDRAW data) stored in the RAM 54, including a synchronization circuit for aligning the position of each color signal, a contour correction circuit, a luminance / color difference signal generation circuit, and the like. .
- the R, G, and B CCDRAW data are multiplied by the white balance gain detected by the AWB detection unit 50 in the digital signal processing unit 44 and subjected to white balance correction, and thereafter, a gradation conversion process (for example, After predetermined processing such as gamma correction is performed, the signal is converted into a YC signal including a luminance signal (Y signal) and a color difference signal (Cr, Cb signal).
- Y signal a luminance signal
- Cr, Cb signal color difference signal
- the YC signal processed by the digital signal processing unit 44 is stored in the RAM 54.
- the digital signal processing unit 44 cuts out an image of a predetermined cutout area from each of the distortion correction circuit and the left and right viewpoint images for correcting the lens distortion correction of the imaging optical systems of the plurality of imaging units 20-1 and 20-2.
- the image pickup processing circuit includes an image cutout processing circuit that corrects an optical axis shift of the imaging optical system of the plurality of imaging units 20-1 and 20-2. Details of processing contents of the distortion correction circuit and the image cut-out processing circuit will be described later.
- the compression / decompression processing unit 42 compresses the YC signal stored in the RAM 54 in accordance with a command from the CPU 32 during recording on the memory card 40, and decompresses the compressed compressed data recorded on the memory card 40. To YC signal.
- the recording control unit 38 converts the compressed data compressed by the compression / decompression processing unit 42 into an image file of a predetermined format (for example, a 3D still image is an MP (multi-picture) format image file, a 3D video is motion JPEG, H .264, MPEG4, MPEG4-MVC video files) or recorded on the memory card 40, or image files are read from the memory card 40.
- the liquid crystal monitor 16 is used as an image display unit for displaying captured images, and is used as a GUI (graphical user interface) at various settings.
- the liquid crystal monitor 16 is used as an electronic viewfinder that displays a live view image (hereinafter referred to as “through image”) for confirming the angle of view in the shooting mode.
- through image a live view image
- the display control unit 36 alternately displays the left-eye image and the right-eye image held in the VRAM 52 pixel by pixel.
- the parallax barrier provided in the liquid crystal monitor 16 the left and right images alternately arranged pixel by pixel are visually recognized separately by the left and right eyes of the user observing from a predetermined distance. This enables stereoscopic viewing.
- the stereoscopic imaging apparatus 10 also has a function of recording and reproducing audio information (audio data) acquired by the stereo microphone 15 shown in FIG. 1A.
- FIG. 3 is a flow chart showing a first embodiment of processing at the time of optical axis adjustment before shipment according to the present invention.
- the stereoscopic imaging device 10 to be adjusted and the adjustment chart for adjusting the optical axis are set so as to have a predetermined positional relationship, and each imaging optical system of the stereoscopic imaging device 10 is set.
- the focus position is adjusted to focus on the adjustment chart (step S10).
- the adjustment chart is installed at the position of the convergence point where the optical axes of the photographing optical systems intersect.
- the adjustment chart is photographed to obtain left and right images (steps S14 and S16).
- the distortion correction corresponding to the current imaging mode N is performed on the acquired left and right images (step S18).
- corresponding point detection is performed to detect corresponding feature points in the left and right images (step S20).
- a corresponding point detection method for example, a block matching method can be applied. That is, the degree of coincidence between a block of a predetermined block size extracted from an arbitrary pixel from the left image and the block of the right image is evaluated, and the reference of the block of the right image when the degree of coincidence between the blocks is maximized.
- Let a pixel be a pixel of the right image corresponding to an arbitrary pixel of the left image.
- a function for evaluating the degree of coincidence between blocks in the block matching method for example, there is a function that uses a sum of squares (SSD) of luminance differences of pixels in each block (SSD block matching method).
- a deviation amount of the corresponding point (when a plurality of corresponding points are detected, an average of deviation amounts of the plurality of corresponding points) is detected, and the detected deviation amount is calculated.
- the optical axis deviation amounts of the two photographing optical systems are stored in the EEPROM 58 in association with the imaging mode N (steps S22 and S24).
- the imaging optical system is a single focus lens
- the optical axis deviation amount is acquired in advance as described above.
- the imaging optical system is a zoom lens as in the present embodiment
- the zoom lens is used. The amount of optical axis deviation is acquired for each zoom position.
- the zoom lens is moved to each zoom position, and the zoom position is described above for each zoom position.
- the optical axis deviation amount is stored in the EEPROM 58 for each zoom position.
- the distortion correction in step S18 uses a distortion correction formula corresponding to each zoom position.
- FIG. 4A to FIG. 4C are examples of tables indicating the optical axis deviation amount for each imaging mode stored in the EEPROM 58.
- Each table has an optical axis shift amount (vertical (V) direction) with respect to the left and right images for each zoom position. Is stored).
- V vertical
- the optical axis deviation in the V direction between the images becomes a problem when performing stereoscopic viewing. Therefore, only the optical axis deviation amount in the V direction is stored. You may make it memorize
- the distortion correction formula for each imaging mode and each zoom position is also stored in the EEPROM 58.
- the same general distortion correction formula is stored for each zoom position, and only the coefficients of each term are stored differently for each zoom position, and different distortion correction formulas are stored for each zoom position. It can be considered that
- 3D moving image shooting mode a shooting mode for shooting a 3D moving image
- 3D still image shooting mode a shooting mode for shooting a 3D still image
- step S40 it is determined by the operation switches 18A and 18C whether the 3D moving image shooting mode is set as the shooting mode or the 3D still image shooting mode is set.
- the CPU 32 determines whether there is a moving image shooting instruction (the shutter button 11 is fully pressed (switch S2 is turned on)) (step S42). If the switch S2 is OFF, a 3D through image is displayed on the liquid crystal monitor 16 (step S44).
- a distortion correction equation (second order polynomial) for the through image is applied to the left and right images sequentially acquired in time series from the left and right imaging units 20-1 and 20-2.
- the optical axis deviation amounts in the left and right images corresponding to the zoom position of the current zoom lens are read out from the through image table shown in FIG. 4A, and these optical axis deviation amounts are canceled out.
- a 3D live view image is cut out from the distortion-corrected image and output to the liquid crystal monitor 16.
- step S46 the CPU 32 starts shooting a 3D moving image
- image processing such as distortion correction and image cut-out processing and recording are performed on the left and right images sequentially acquired in time series from the left and right imaging units 20-1 and 20-2. Processing is performed (step S48).
- FIG. 6 is a flowchart showing a first embodiment of image processing in step S48 of FIG.
- the distortion correction circuit in the digital signal processing unit 44 is a distortion correction formula (fourth order polynomial) for moving images, and the zoom of the current zoom lens.
- the distortion correction of the acquired left and right images is performed using a distortion correction formula corresponding to the position (step S484).
- the amount of optical axis deviation corresponding to the current zoom position (Z-pos) is read from the moving image table of FIG. 4C, and the light from the left and right images subjected to the distortion correction based on the read amount of optical axis deviation.
- An image cutout process is performed by moving the cutout range by the axis deviation amount (step S486). Thereby, the cut-out image becomes an image in which the optical axis deviation is corrected.
- the left and right images cut out as described above are each compressed in a predetermined compression format and then recorded in a moving image file created on the memory card 40 (step S488).
- the moving image recording process is performed every time image processing of a prescribed number of frames for one second (60 frames when the frame rate is 60 frames / second) is completed. Will be added sequentially.
- audio data acquired by the stereo microphone 15 is compressed every second and recorded in a moving image file.
- step S50 it is determined whether or not the switch S2 is turned on again (step S50). If the switch S2 is turned off, the process proceeds to step S48 to continue the processing of the moving image. When the switch S2 is turned on, The shooting of the 3D video is terminated.
- step S40 when it is determined in step S40 that the mode is the 3D still image shooting mode, the CPU 32 determines whether or not there is a still image shooting instruction (switch S2 is ON) (step S52). When the switch S2 is OFF, a 3D through image is displayed on the liquid crystal monitor 16 (step S54). The display of the 3D through image on the liquid crystal monitor 16 is performed in the same manner as in step S44 described above.
- step S56 When the switch S2 is turned on, the CPU 32 takes a 3D still image (step S56). Needless to say, when the two-stroke shutter button is half-pressed before it is fully pressed, the switch S1 is turned on, thereby performing preparatory shooting preparation operations such as AE processing and AF processing.
- step S58 The left and right images acquired from the left and right imaging units 20-1 and 20-2 at the time of the main photographing are subjected to image processing such as distortion correction and image clipping processing and recording processing, similarly to step S48 (step S58).
- the distortion correction in step S58 is performed using a distortion correction formula of a sixth-order polynomial with high correction accuracy, and the image clipping process corresponds to the current zoom position from the still image table of FIG. 4B. This is performed based on the read optical axis deviation amount.
- FIG. 7 is a flowchart showing a second embodiment of the processing at the time of optical axis adjustment according to the present invention.
- the same step number is attached
- the optical axis deviation amount is detected and stored in the EEPROM 58 for each imaging mode of the through image, moving image, and still image (see FIGS. 4A to 4C).
- the corresponding points of the left and right images are detected without performing distortion correction on the left and right images acquired in step S ⁇ b> 16, and the optical axis deviation amount of the left and right images is detected.
- the detected optical axis deviation amount is stored in the EEPROM 58 (step S24). Note that storing the optical axis deviation amount in the EEPROM 58 for each zoom position of the zoom lens is the same as in the embodiment shown in FIG.
- the optical axis deviation amounts of the left and right images before distortion correction are detected and recorded, and the optical axis deviation amount is not detected and stored for each imaging mode.
- step S60 and S62 information on the current imaging mode and information on the zoom position of the zoom lens are acquired.
- the optical axis shift amount corresponding to the zoom position is acquired (step S64). Further, a distortion correction formula (calculation formula) corresponding to the current imaging mode and zoom position is acquired from the EEPROM 58 based on the acquired current imaging mode and current zoom position.
- the optical axis deviation amount after distortion correction is calculated by substituting the optical axis deviation amount obtained in step S64 for the obtained calculation formula (steps S66 and S68).
- the calculated optical axis deviation amount is used for image clipping processing in steps S44, S48, and S58 shown in FIG. 5 and step S486 shown in FIG.
- the optical axis deviation amount before distortion correction is held in the EEPROM 58, and the value held in the EEPROM 58 is applied to the calculation formula for distortion correction at the time of actual image cutout processing, so that the optical axis deviation after distortion correction is obtained.
- the memory capacity of the EEPROM 58 can be reduced, and the cutout position can be easily changed even when the firmware or the like is changed (including a change in distortion correction). .
- the optical axis deviation amount table stores the optical axis deviation amount for each zoom position, but at least two zoom positions (for example, the wide end and the tele end). Only the optical axis deviation amount is stored, and the optical axis deviation amount at a zoom position other than the stored zoom position is obtained by linearly interpolating the stored optical axis deviation amounts of at least two zoom positions according to the current zoom position. May be calculated.
- the optical axis center is shifted depending on the zoom position of the zoom lens, but the optical axis center is also shifted depending on the focus position of the focus lens.
- the optical axis deviation amount for each focus position of the focus lens is acquired by inspection before shipment and stored in the EEPROM 58.
- the amount of optical axis deviation is detected for each focus position of the focus lens (the closest focus position (F1) to the infinity focus position (Fn)).
- an adjustment chart is arranged at a subject distance corresponding to a certain focus position, and the optical axis deviation amount at each zoom position is detected while changing the zoom position of the zoom lens.
- This optical axis deviation amount may be detected based on the left and right images after distortion correction as described in the first embodiment of FIG. 3, or as in the second embodiment shown in FIG. You may make it detect based on the image of right and left before distortion correction.
- the detection of the optical axis deviation amount is performed while sequentially moving the position of the adjustment chart to the position corresponding to the focus position, thereby obtaining the optical axis deviation amounts at all zoom positions and focus positions.
- FIG. 9 shows a table of optical axis deviation amounts acquired corresponding to the zoom position (Z1 to Zn) and all the focus positions (F1 to Fn).
- the table shown in FIG. 9 is produced for every imaging mode.
- the table is selected according to the current imaging mode. Is obtained by reading out the corresponding optical axis deviation based on the current zoom position and focus position.
- the distortion correction formula corresponding to the current imaging mode and zoom position corresponds to the current zoom position and focus position. Then, the read optical axis deviation amount is substituted to calculate the optical axis deviation amount after distortion correction.
- the table shown in FIG. 9 stores the optical axis deviation amounts corresponding to all zoom positions and focus positions, but not limited to this, two focus positions (closest focus positions) as shown in FIG. (Near) and infinity focus position (Inf)) are detected and stored for the optical axis misalignment amount, and the optical axis misalignment amount at the intermediate focus position is the intermediate focus position. May be calculated by linear interpolation.
- the optical axis deviation amounts at the three focus positions of close, intermediate, and infinity are detected and stored, and from close to intermediate, Alternatively, it is preferable to calculate the amount of optical axis deviation by linear interpolation from the middle to infinity.
- FIG. 11 is a flowchart showing a second embodiment of image processing in step S48 of FIG. In addition, the same step number is attached
- step S ⁇ b> 483 for performing shading correction is added before step S ⁇ b> 484 for performing distortion correction. This is different from the first embodiment.
- step S483 the left and right images acquired in step S482 are obtained by a table for each left and right image in which shading correction values are stored according to the view angle position, or by a calculation formula according to the view angle position.
- the luminance of each pixel is corrected (shading correction) using the shading correction value.
- FIG. 12 is a flowchart showing a first embodiment of image processing during continuous shooting.
- step S70 When the continuous shooting mode is set and an instruction for continuous shooting (switch S2 is ON) is input (step S70), the CPU 32 performs shooting for one frame of continuous shooting (step S72), and the left and right images are taken. The image with the full angle of view is temporarily stored in the RAM 54 (step S74).
- step S76 ON / OFF of the switch S2 is determined (step S76). If it is ON, the process proceeds to step S72. If it is OFF, the process proceeds to step S78. That is, when the switch S2 is turned on, continuous shooting is performed during that time, and left and right full angle images are stored in the RAM 54.
- step S78 When the switch S2 is turned OFF, the time-series left and right images stored in the RAM 54 are read one frame at a time, and the distortion correction formula corresponding to the zoom position of the zoom lens and the continuous shooting mode during continuous shooting is used. Distortion correction is performed (step S78).
- the zoom position of the zoom lens at the time of continuous shooting and the optical axis shift amount corresponding to the continuous shooting mode are read from the EEPROM 58 or calculated, and the full angle of view corrected for distortion based on the optical axis shift amount.
- the image for correcting the optical axis deviation is cut out from the image (step S80).
- the image cut out as described above is recorded in the memory card 40 after being subjected to compression processing or the like (step S82).
- FIG. 13 is a flowchart illustrating a second embodiment of image processing during continuous shooting.
- the same step number is attached
- step S77 is added between steps S76 and S78.
- the second embodiment of the image processing during continuous shooting shown in FIG. 13 is different from the first embodiment in that the processing of step S77 is added between steps S76 and S78.
- step S77 the left and right images stored in the RAM 54 are subjected to shading correction according to the angle of view, respectively, so that the brightness of the left and right images is uniform. As a result, it is possible to obtain an image having no difference in brightness between the left and right images on which the subsequent distortion correction and the image clipping process are performed.
- FIG. 14 is a flowchart illustrating a third embodiment of image processing during continuous shooting.
- the same step number is attached
- step S76 when the switch S2 is turned off (step S76), the time-series left and right images stored in the RAM 54 are read out and reproduced in the shooting order (step S90).
- N 3D images are shot by continuous shooting
- the left and right images after this processing are displayed as 3D still images on the liquid crystal monitor 16 (step S92).
- the user determines whether or not to save the image in the memory card 40 while viewing the 3D still image displayed on the liquid crystal monitor 16 (step S94). In the case of “Yes” (for example, when the MENU / OK button is turned on), the 3D still image displayed on the liquid crystal monitor 16 is stored in the memory card 40.
- step S90 the next left and right images are read from the RAM 54, and the same as above. Processing is performed.
- continuous shooting is performed while the switch S2 is ON.
- the present invention is not limited to this, and when the switch S2 is turned ON, continuous shooting is performed for a preset number of images. You can do it.
- the shooting mode at the time of shooting for example, 3D moving image shooting mode, 3D still image shooting mode
- zoom position of the zoom lens for example, 3D moving image shooting mode, 3D still image shooting mode
- optical axis shift The amount is written in the tag of the image file recorded in the memory card 40 (step S100).
- the optical axis deviation amount is read from the EEPROM 58 or calculated depending on the imaging mode, zoom position, and the like.
- the captured left and right full-angle images are stored in the image file (step S102).
- tag information is read out together with the left and right images from the playback target image file stored in the memory card 40 (step S110). ).
- a distortion correction formula specified by the imaging mode and zoom position included in the tag information is acquired, and distortion correction is performed on the read left and right images using the distortion correction formula (step S112).
- an image for correcting the optical axis deviation is cut out from the left and right images after distortion correction based on the optical axis deviation amount included in the tag information (step S114).
- the left and right images cut out in this way are displayed so that the cut-out center is the center of the screen of the liquid crystal monitor 16 (step S116), thereby being displayed as a 3D image that is easy to view stereoscopically without optical axis deviation.
- the optical axis deviation amount is recorded as tag information.
- the coordinates of the optical axis center or the diagonal coordinates of the cutout range may be recorded.
- any information may be used as long as it is image cutout information that can correct the optical axis deviation.
- an image that has been subjected to distortion correction and image clipping processing during 3D playback can be recorded in the memory card 40.
- the original image file may be deleted, or both image files may coexist.
- FIG. 17 and 18 are flowcharts showing a second embodiment of the photographing / reproducing process of the stereoscopic imaging apparatus 10 according to the present invention.
- the stereoscopic imaging apparatus 10 when the stereoscopic imaging apparatus 10 is set to the shooting mode and shooting is started, various correction processes are performed on the captured left and right full-angle images (moving images or still images) (step S120).
- the image processing here is image processing other than the image cutout processing for correcting the optical axis shift of the left and right images, and includes white balance correction, gamma correction, shading correction, distortion correction, and the like.
- the left and right images that have undergone image processing are compressed in a predetermined compression format and then stored in the memory card 40 (step S122).
- the left and right images are read from the playback target image file stored in the memory card 40 (step S130).
- corresponding point detection for detecting corresponding feature points of the left and right images is performed, and information for correcting the optical axis shift of the left and right images is acquired (step S132).
- Corresponding point detection can be performed by, for example, a block matching method.
- a corresponding pixel on the other image is obtained by corresponding point detection.
- a region where the corresponding point can be detected and a region where the corresponding point cannot be detected can be obtained between the left and right images. Then, by detecting each of the regions surrounded by the four outermost sides of the region where the corresponding points of the left and right images can be detected, information for correcting the optical axis shift of the left and right images can be acquired.
- the center of the region surrounded by the four sides is a cut-out region where a stereoscopic image is cut out from pixels at all angles of view, and the center of the region surrounded by the four sides is the cut-out center.
- the left and right images cut out from the left and right full-view angle images by the cut-out areas obtained as described above are displayed so that the cut-out center is the screen center of the liquid crystal monitor 16 (step S134).
- the image is displayed as a 3D image that is easy to be viewed stereoscopically without optical axis deviation.
- cutout regions (cutout start point or cutout center and a region specified by the cutout size) indicated by dotted lines obtained by corresponding point detection are cut out and cut out, respectively.
- the displayed image (the left and right overlapping portions) is displayed on the liquid crystal monitor 16 as a stereoscopic image.
- the cut-out centers of the left and right images are displayed in a consistent manner, so that the optical axis shift (shift in the V direction) of the left and right images is corrected.
- a pattern such as a frame is arranged in a portion where the left and right images do not overlap.
- the MTF modulation transfer function
- the stereoscopic imaging apparatus 10 can display a 3D through image on the liquid crystal monitor 16, and the user operates the parallax amount adjustment switch 18B (FIG. 1B) while viewing the 3D through image.
- the parallax amount of the 3D image moving image or still image
- the parallax amount of the 3D image can be adjusted.
- the parallax amount (parallax adjustment value) of the 3D image can be increased or decreased by operating the parallax amount adjustment switch 18B in the + direction or the ⁇ direction.
- the plurality of imaging modes for performing distortion correction with different correction accuracy are not limited to this embodiment, and include an imaging mode in which distortion correction is not performed and an imaging mode in which distortion is emphasized, such as a fish-eye imaging mode. Also good.
- the image clipping process is performed on the distortion-corrected image.
- the image clipping process for correcting the optical axis deviation is performed on the image.
- distortion correction may be performed.
- the image cut-out process in this case is performed in consideration of the optical axis shift caused by the subsequent distortion correction.
- SYMBOLS 10 Stereo imaging device, 11 ... Shutter button, 12 ... Zoom button, 14-1, 14-2 ... Shooting optical system, 16 ... Liquid crystal monitor, 20-1, 20-2 ... Imaging part, 21 ... Focus lens and zoom Lens, 24 ... CCD, 25 ... Analog signal processing unit, 32 ... Central processing unit (CPU), 34 ... Operation unit, 44 ... Digital signal processing unit, 54 ... RAM, 56 ... ROM, 58 ... EEPROM
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Studio Devices (AREA)
Abstract
Description
図1は本発明に係る立体撮像装置の外観を示す図であり、図1Aは立体撮像装置を前面側から見た斜視図であり、図1Bは背面図である。
図2は上記立体撮像装置10の実施形態を示すブロック図である。
次に、製品出荷前の調整時にEEPROM58に記憶させる光軸調整用の情報について説明する。
次に、本発明に係る立体撮像装置10の3D動画又は3D静止画の撮影動作について、図5に示すフローチャートを参照しながら説明する。
図6は図5のステップS48等における画像処理の第1の実施形態を示すフローチャートである。
図7は本発明に係る光軸調整時の処理の第2の実施形態を示すフローチャートである。尚、図3に示した第1の実施形態と共通する部分には同一のステップ番号を付し、その詳細な説明は省略する。
立体撮像装置10により取得される左右の画像は、ズームレンズのズーム位置により光軸中心がずれるが、フォーカスレンズのフォーカス位置によっても光軸中心がずれる。
図11は図5のステップS48等における画像処理の第2の実施形態を示すフローチャートである。尚、図6の第1の実施形態と共通する部分には同一のステップ番号を付し、その詳細な説明は省略する。
連写撮影される各画像は静止画として鑑賞されるため、3D静止画と同様な画像処理が要求される。即ち、ディストーション補正は、補正精度の高いディストーション補正式による補正を実施する必要がある。
図13は連写撮影時の画像処理の第2の実施形態を示すフローチャートである。尚、図12に示した連写撮影時の画像処理の第1の実施形態と共通する部分には同一のステップ番号を付し、その詳細な説明は省略する。
図14は連写撮影時の画像処理の第3の実施形態を示すフローチャートである。尚、図12に示した連写撮影時の画像処理の第1の実施形態と共通する部分には同一のステップ番号を付し、その詳細な説明は省略する。
図15及び図16は本発明に係る立体撮像装置10の撮影/再生処理の第1の実施形態を示すフローチャートである。
図17及び図18は本発明に係る立体撮像装置10の撮影/再生処理の第2の実施形態を示すフローチャートである。
画像の切り出しが行われた左右の画像の切り出し中心は、ほぼ同じ被写体が存在するため、左右の画像の切り出し中心部分のMTF(modulation transfer function)測定を行い、左右の画像の解像度の差を計算する。そして、左右の画像に対する画質設定(輪郭強調、ガンマ補正)等を変更することにより、左右の画像の解像度を均一にすることができる。
Claims (16)
- 撮影光学系と該撮影光学系を介して結像される被写体像をそれぞれ光電変換する撮像素子とを有する複数の撮像部であって、互いに視差を有する複数の画像を撮像する複数の撮像部と、
複数の撮像モードごとに設定され、前記複数の撮像モードごとに補正精度の異なるディストーション補正式を記憶する第1の記憶部と、
前記第1の記憶部に記憶されたディストーション補正式のうちから、現在の撮像モードに対応するディストーション補正式を取得するディストーション補正式取得部と、
予め検出された前記複数の撮像部の各撮影光学系の光軸ずれ量であって、前記複数の撮像モードごとに設定されるディストーション補正式によりディストーション補正された後の光軸ずれ量を記憶する第2の記憶部と、
前記第2の記憶部に記憶された各撮影光学系の光軸ずれ量と現在の撮像モードとに基づき現在の撮像モードに対応する光軸ずれ量を取得する光軸ずれ量取得部と、
現在の撮像モードに応じて前記複数の撮像部から複数の画像を取得する撮像制御部と、
前記撮像制御部により取得された複数の画像に対して、前記ディストーション補正式取得部により現在の撮像モードに対応して取得されたディストーション補正式に基づいてディストーション補正を行うディストーション補正部と、
前記光軸ずれ量取得部により現在の撮像モードに対応して取得された光軸ずれ量に基づいて、前記撮像制御部により取得された複数の画像に対して立体表示用の画像の切り出し処理を行う画像切出し部と、
を備えたことを特徴とする立体撮像装置。 - 前記複数の撮影光学系の現在のズーム位置を検出するズーム位置検出部を更に有し、
前記第1の記憶部は、前記撮影光学系のズーム位置に応じた前記ディストーション補正式を記憶し、
前記ディストーション補正式取得部は、現在の撮像モード及び前記撮影光学系の現在のズーム位置に対応するディストーション補正式を前記1の記憶部から取得することを特徴とする請求項1に記載の立体撮像装置。 - 前記第2の記憶部は、前記複数の撮影光学系のズーム位置に応じた前記各撮影光学系の光軸ずれ量を記憶し、
前記光軸ずれ量取得部は、現在の撮像モード及び前記撮影光学系の現在のズーム位置に応じて前記第2の記憶部から対応する光軸ずれ量を取得することを特徴とする請求項1又は2に記載の立体撮像装置。 - 前記第2の記憶部は、各撮影光学系の光軸ずれ量として、各撮像モード及び前記撮影光学系のズーム位置に対応するディストーション補正式によるディストーション補正後の光軸ずれ量を、各撮像モード及びズーム位置に応じて記憶し、
前記光軸ずれ量取得部は、現在の撮像モード及び現在のズーム位置に応じて前記第2の記憶部から対応する光軸ずれ量を読み出す読出部を有することを特徴とする請求項2又は3に記載の立体撮像装置。 - 前記第2の記憶部は、各撮影光学系の光軸ずれ量として、ディストーション補正前の光軸ずれ量を前記撮影光学系のズーム位置に応じて記憶し、
前記光軸ずれ量取得部は、前記第2の記憶部から現在のズーム位置に基づいて読み出した光軸ずれ量を、前記ディストーション補正式取得部により取得された現在の撮像モードに対応するディストーション補正式に代入してディストーション補正後の光軸ずれ量を算出する算出部を有することを特徴とする請求項2又は3に記載の立体撮像装置。 - 前記第2の記憶部は、各撮影光学系の光軸ずれ量として、各撮像モード及び前記撮影光学系のズーム位置に対応するディストーション補正式によるディストーション補正後の光軸ずれ量を算出するための情報を、各撮像モード毎に記憶し、
前記光軸ずれ量取得部は、現在の撮像モードに応じて前記第2の記憶部から読み出した情報と現在のズーム位置とに基づいて対応する光軸ずれ量を算出する算出部を有することを特徴とする請求項2又は3に記載の立体撮像装置。 - 前記複数の撮影光学系の現在のフォーカス位置を検出するフォーカス位置検出部を更に備え、
前記第2の記憶部は、各撮像モード及び前記撮影光学系のズーム位置に対応するディストーション補正式によるディストーション補正後の光軸ずれ量を各撮像モード、ズーム位置及びフォーカス位置に応じて記憶し、
前記光軸ずれ量取得部は、現在の撮像モード、ズーム位置及びフォーカス位置に応じて前記第2の記憶部から対応する光軸ずれ量を読み出す読出部を有することを特徴とする請求項2又は3に記載の立体撮像装置。 - 前記複数の撮影光学系の現在のフォーカス位置を検出するフォーカス位置検出部を更に備え、
前記第2の記憶部は、各撮像モード及び前記撮影光学系のズーム位置に対応するディストーション補正式によるディストーション補正後の光軸ずれ量を算出するための情報を、各撮像モード及びズーム位置に応じて記憶し、
前記光軸ずれ量取得部は、現在の撮像モード及び現在のズーム位置に応じて前記第2の記憶部から読み出した情報と現在のフォーカス位置とに基づいて対応する光軸ずれ量を算出する算出部と、を有することを特徴とする請求項2又は3に記載の立体撮像装置。 - 前記撮像制御部により取得された複数の画像のシェーディング補正を行うシェーディング補正部を更に備え、
前記画像切出し部は、前記シェーディング補正部によるシェーディング補正された画像に対して前記画像の切り出し処理を行うことを特徴とする請求項1から8のいずれかに記載の立体撮像装置。 - 前記画像切出し部は、前記ディストーション補正部によるディストーション補正後の画像に対して画像の切り出し処理を行うことを特徴とする請求項1から9のいずれかに記載の立体撮像装置。
- 前記ディストーション補正部は、前記画像切出し部により切り出し処理された画像に対してディストーション補正を行うことを特徴とする請求項1から9のいずれかに記載の立体撮像装置。
- 前記複数の撮像モードは、ライブビュー画像を表示部に表示させる動作時の撮像モード、静止画撮像モード、動画撮像モード、及びディストーション強調撮像モードのうちの2以上の撮像モードであることを特徴とする請求項1から11のいずれかに記載の立体撮像装置。
- 予め設定した枚数又は撮影指示期間中、前記複数の撮像部から時系列の複数の画像を取得する連写モードを選択する部と、
前記連写モードにより撮影中の画像を一時記憶する内部記憶部と、を備え、
前記シェーディング補正部は、前記連写モードによる撮影終了後に前記内部記憶部に記憶された複数の画像を読み出してシェーディング補正を行うことを特徴とする請求項10に記載の立体撮像装置。 - 撮影モード又は再生モードを選択するモード選択部と、
前記モード選択部により選択された撮影モード時に前記撮像制御部により取得された複数の画像とともに、前記撮像モードを示す情報及び前記光軸ずれ量取得部により取得された光軸ずれ量を示す情報を、前記取得された複数の画像に関連付けて記録媒体に記録する記録部と、を備え、
前記ディストーション補正部及び画像切出し部は、前記モード選択部により選択された再生モード時に前記記録媒体から前記複数の画像とともに、該画像に関連付けて記憶された情報を読み出し、該読み出した複数の画像に対して、前記読み出された情報に対応する前記ディストーション補正式及び光軸ずれ量に基づいてディストーション補正及び画像の切り出し処理を行うことを特徴とする請求項1から13のいずれかに記載の立体撮像装置。 - 前記記録部は、前記再生モード時に前記ディストーション補正及び画像の切り出し処理された画像を、前記記録媒体に記録することを特徴とする請求項14に記載の立体撮像装置。
- 前記複数の撮像部から出力される複数の画像間の視差量を調整する視差量調整部を備え、
前記画像切出し部は、前記立体表示用の画像の切り出し処理時に前記視差量調整部により調整された視差量に基づいて更に切り出し位置を調整した画像の切り出し処理を行うことを特徴とする請求項1から15のいずれかに記載の立体撮像装置。
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011527139A JP4875225B2 (ja) | 2010-03-31 | 2010-11-22 | 立体撮像装置 |
CN201080003915.8A CN102318331B (zh) | 2010-03-31 | 2010-11-22 | 立体图像拾取装置 |
BRPI1006035A BRPI1006035A2 (pt) | 2010-03-31 | 2010-11-22 | "aparelho de coletar imagem estereiscópica ". |
US13/142,656 US8363091B2 (en) | 2010-03-31 | 2010-11-22 | Stereoscopic image pick-up apparatus |
EP10838383.7A EP2458842B1 (en) | 2010-03-31 | 2010-11-22 | 3d-image capturing device |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010081051 | 2010-03-31 | ||
JP2010-081051 | 2010-03-31 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011121840A1 true WO2011121840A1 (ja) | 2011-10-06 |
Family
ID=44711614
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2010/070776 WO2011121840A1 (ja) | 2010-03-31 | 2010-11-22 | 立体撮像装置 |
Country Status (6)
Country | Link |
---|---|
US (1) | US8363091B2 (ja) |
EP (1) | EP2458842B1 (ja) |
JP (1) | JP4875225B2 (ja) |
CN (1) | CN102318331B (ja) |
BR (1) | BRPI1006035A2 (ja) |
WO (1) | WO2011121840A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013152522A (ja) * | 2012-01-24 | 2013-08-08 | Fujitsu Semiconductor Ltd | 画像の光学歪み補正装置,画像の光学歪み補正方法,および,画像の光学歪み補正装置を有する画像生成装置 |
JP2014142917A (ja) * | 2012-12-28 | 2014-08-07 | Ricoh Co Ltd | 画像処理装置 |
US9661303B2 (en) | 2011-12-14 | 2017-05-23 | Sony Corporation | Multi-camera stereoscopic imaging apparatus for reducing the effects of aberration |
TWI625051B (zh) * | 2017-03-21 | 2018-05-21 | 奇景光電股份有限公司 | 深度感測裝置 |
US11843894B2 (en) | 2021-04-26 | 2023-12-12 | Canon Kabushiki Kaisha | Electronic apparatus, method of controlling the same, and storage medium |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080031907A1 (en) * | 2002-10-25 | 2008-02-07 | Foamix Ltd. | Cosmetic and pharmaceutical foam |
WO2012032778A1 (ja) * | 2010-09-08 | 2012-03-15 | パナソニック株式会社 | 立体画像処理装置、立体撮像装置、立体撮像方法およびプログラム |
JP5758138B2 (ja) * | 2011-02-01 | 2015-08-05 | シャープ株式会社 | 撮像装置、データ処理方法、およびプログラム |
WO2013031227A1 (ja) * | 2011-09-01 | 2013-03-07 | パナソニック株式会社 | 撮像装置およびプログラム |
US9160917B2 (en) | 2013-07-09 | 2015-10-13 | Htc Corporation | Handheld electronic apparatus and image capturing apparatus and focusing method thereof |
JP5320524B1 (ja) * | 2012-01-20 | 2013-10-23 | パナソニック株式会社 | ステレオ撮影装置 |
US9894269B2 (en) | 2012-10-31 | 2018-02-13 | Atheer, Inc. | Method and apparatus for background subtraction using focus differences |
JP6010870B2 (ja) * | 2013-12-24 | 2016-10-19 | カシオ計算機株式会社 | 画像補正装置、及び画像補正方法、プログラム |
US20150215530A1 (en) * | 2014-01-27 | 2015-07-30 | Microsoft Corporation | Universal capture |
US9804392B2 (en) | 2014-11-20 | 2017-10-31 | Atheer, Inc. | Method and apparatus for delivering and controlling multi-feed data |
JP6494328B2 (ja) * | 2015-03-02 | 2019-04-03 | キヤノン株式会社 | 画像処理装置、撮像装置、画像処理方法、画像処理プログラム、および、記憶媒体 |
US10917543B2 (en) | 2017-04-24 | 2021-02-09 | Alcon Inc. | Stereoscopic visualization camera and integrated robotics platform |
US11083537B2 (en) | 2017-04-24 | 2021-08-10 | Alcon Inc. | Stereoscopic camera with fluorescence visualization |
US10299880B2 (en) * | 2017-04-24 | 2019-05-28 | Truevision Systems, Inc. | Stereoscopic visualization camera and platform |
US10762658B2 (en) * | 2017-10-24 | 2020-09-01 | Altek Corporation | Method and image pick-up apparatus for calculating coordinates of object being captured using fisheye images |
CN108270952B (zh) * | 2017-11-21 | 2020-08-07 | 深圳市芯舞时代科技有限公司 | 一种双目摄像机影像色差的校正方法及系统 |
JP7350510B2 (ja) | 2019-05-14 | 2023-09-26 | キヤノン株式会社 | 電子機器、電子機器の制御方法、プログラム、及び、記憶媒体 |
JP2022027107A (ja) * | 2020-07-31 | 2022-02-10 | セイコーエプソン株式会社 | 画像表示方法、画像表示装置及び表示制御プログラム |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08317424A (ja) | 1995-05-19 | 1996-11-29 | Olympus Optical Co Ltd | 立体撮影装置 |
JPH11355813A (ja) * | 1998-06-04 | 1999-12-24 | Honda Motor Co Ltd | カメラの内部パラメータ決定装置 |
JP2004007304A (ja) * | 2002-06-03 | 2004-01-08 | Fuji Photo Film Co Ltd | デジタル撮影装置 |
JP2004126905A (ja) | 2002-10-02 | 2004-04-22 | Honda Motor Co Ltd | 画像処理装置 |
WO2006064770A1 (ja) * | 2004-12-16 | 2006-06-22 | Matsushita Electric Industrial Co., Ltd. | 撮像装置 |
JP2006162991A (ja) | 2004-12-07 | 2006-06-22 | Fuji Photo Film Co Ltd | 立体画像撮影装置 |
JP2007282245A (ja) * | 2006-04-10 | 2007-10-25 | Sony Taiwan Ltd | 画像合成装置及び画像合成方法 |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0795623A (ja) | 1993-06-25 | 1995-04-07 | Sanyo Electric Co Ltd | 立体撮像装置 |
JPH08336165A (ja) | 1995-06-09 | 1996-12-17 | Canon Inc | 複眼撮像装置 |
JP4012710B2 (ja) * | 2001-02-14 | 2007-11-21 | 株式会社リコー | 画像入力装置 |
US20040001138A1 (en) | 2002-06-27 | 2004-01-01 | Weerashinghe W.A. Chaminda P. | Stereoscopic panoramic video generation system |
JP4046276B2 (ja) | 2002-10-24 | 2008-02-13 | 富士フイルム株式会社 | デジタルカメラ |
CN101841728B (zh) * | 2003-04-17 | 2012-08-08 | 夏普株式会社 | 三维图像处理装置 |
EP1635138A4 (en) | 2003-05-29 | 2011-05-04 | Olympus Corp | STEREO OPTICAL MODULE AND STEREO CAMERA |
US7596286B2 (en) | 2003-08-06 | 2009-09-29 | Sony Corporation | Image processing apparatus, image processing system, imaging apparatus and image processing method |
JP3743828B2 (ja) * | 2003-10-14 | 2006-02-08 | カシオ計算機株式会社 | 電子カメラ |
JP4104571B2 (ja) * | 2004-03-29 | 2008-06-18 | 三洋電機株式会社 | 歪曲補正装置及びこの歪曲補正装置を備えた撮像装置 |
WO2006062325A1 (en) | 2004-12-06 | 2006-06-15 | Electronics And Telecommunications Research Institute | Apparatus for correcting image distortion of stereo-camera and method thereof |
JP2006267768A (ja) * | 2005-03-25 | 2006-10-05 | Fuji Photo Film Co Ltd | 撮影装置および投光モジュール |
US7920200B2 (en) * | 2005-06-07 | 2011-04-05 | Olympus Corporation | Image pickup device with two cylindrical lenses |
JP5247702B2 (ja) * | 2006-09-15 | 2013-07-24 | デジタルオプティクス・コーポレイション・ヨーロッパ・リミテッド | 改良された画像品質を伴う画像システム及び関連する方法 |
JP4406937B2 (ja) * | 2006-12-01 | 2010-02-03 | 富士フイルム株式会社 | 撮影装置 |
JP4714174B2 (ja) * | 2007-03-27 | 2011-06-29 | 富士フイルム株式会社 | 撮像装置 |
JP4714176B2 (ja) * | 2007-03-29 | 2011-06-29 | 富士フイルム株式会社 | 立体撮影装置及び光軸調節方法 |
CN101715064A (zh) * | 2008-10-06 | 2010-05-26 | 鸿富锦精密工业(深圳)有限公司 | 影像撷取装置及其影像拍摄方法 |
JP4995854B2 (ja) * | 2009-03-11 | 2012-08-08 | 富士フイルム株式会社 | 撮像装置、画像補正方法および画像補正プログラム |
-
2010
- 2010-11-22 WO PCT/JP2010/070776 patent/WO2011121840A1/ja active Application Filing
- 2010-11-22 CN CN201080003915.8A patent/CN102318331B/zh not_active Expired - Fee Related
- 2010-11-22 JP JP2011527139A patent/JP4875225B2/ja not_active Expired - Fee Related
- 2010-11-22 BR BRPI1006035A patent/BRPI1006035A2/pt not_active IP Right Cessation
- 2010-11-22 EP EP10838383.7A patent/EP2458842B1/en not_active Not-in-force
- 2010-11-22 US US13/142,656 patent/US8363091B2/en not_active Expired - Fee Related
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08317424A (ja) | 1995-05-19 | 1996-11-29 | Olympus Optical Co Ltd | 立体撮影装置 |
JPH11355813A (ja) * | 1998-06-04 | 1999-12-24 | Honda Motor Co Ltd | カメラの内部パラメータ決定装置 |
JP2004007304A (ja) * | 2002-06-03 | 2004-01-08 | Fuji Photo Film Co Ltd | デジタル撮影装置 |
JP2004126905A (ja) | 2002-10-02 | 2004-04-22 | Honda Motor Co Ltd | 画像処理装置 |
JP2006162991A (ja) | 2004-12-07 | 2006-06-22 | Fuji Photo Film Co Ltd | 立体画像撮影装置 |
WO2006064770A1 (ja) * | 2004-12-16 | 2006-06-22 | Matsushita Electric Industrial Co., Ltd. | 撮像装置 |
JP2007282245A (ja) * | 2006-04-10 | 2007-10-25 | Sony Taiwan Ltd | 画像合成装置及び画像合成方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP2458842A4 |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9661303B2 (en) | 2011-12-14 | 2017-05-23 | Sony Corporation | Multi-camera stereoscopic imaging apparatus for reducing the effects of aberration |
CN103163717B (zh) * | 2011-12-14 | 2017-06-30 | 索尼公司 | 立体摄像设备 |
JP2013152522A (ja) * | 2012-01-24 | 2013-08-08 | Fujitsu Semiconductor Ltd | 画像の光学歪み補正装置,画像の光学歪み補正方法,および,画像の光学歪み補正装置を有する画像生成装置 |
JP2014142917A (ja) * | 2012-12-28 | 2014-08-07 | Ricoh Co Ltd | 画像処理装置 |
TWI625051B (zh) * | 2017-03-21 | 2018-05-21 | 奇景光電股份有限公司 | 深度感測裝置 |
US11843894B2 (en) | 2021-04-26 | 2023-12-12 | Canon Kabushiki Kaisha | Electronic apparatus, method of controlling the same, and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US8363091B2 (en) | 2013-01-29 |
EP2458842A4 (en) | 2012-11-28 |
BRPI1006035A2 (pt) | 2016-12-06 |
US20110279653A1 (en) | 2011-11-17 |
CN102318331B (zh) | 2014-07-30 |
EP2458842A1 (en) | 2012-05-30 |
JPWO2011121840A1 (ja) | 2013-07-04 |
EP2458842B1 (en) | 2013-12-25 |
CN102318331A (zh) | 2012-01-11 |
JP4875225B2 (ja) | 2012-02-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4875225B2 (ja) | 立体撮像装置 | |
JP4897940B2 (ja) | 立体撮像装置 | |
JP5166650B2 (ja) | 立体撮像装置、画像再生装置及び編集ソフトウエア | |
US9560341B2 (en) | Stereoscopic image reproduction device and method, stereoscopic image capturing device, and stereoscopic display device | |
US8736671B2 (en) | Stereoscopic image reproduction device and method, stereoscopic image capturing device, and stereoscopic display device | |
JP5722975B2 (ja) | 撮像装置、撮像装置用シェーディング補正方法及び撮像装置用プログラム | |
WO2012132797A1 (ja) | 撮像装置及び撮像方法 | |
JP5426262B2 (ja) | 複眼撮像装置 | |
US9310672B2 (en) | Stereoscopic image capturing device and method of controlling thereof | |
JP5466773B2 (ja) | 立体動画再生装置、立体動画再生プログラムならびにその記録媒体、立体ディスプレイ装置、立体撮像装置及び立体動画再生方法 | |
JP2010237582A (ja) | 立体撮像装置および立体撮像方法 | |
JP2010103949A (ja) | 撮影装置、方法およびプログラム | |
JP2010200024A (ja) | 立体画像表示装置および立体画像表示方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080003915.8 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011527139 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13142656 Country of ref document: US Ref document number: 4621/CHENP/2011 Country of ref document: IN Ref document number: 2010838383 Country of ref document: EP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10838383 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01A Ref document number: PI1006035 Country of ref document: BR |
|
REG | Reference to national code |
Ref country code: BR Ref legal event code: B01E Ref document number: PI1006035 Country of ref document: BR Free format text: REGULARIZE A PROCURACAO, UMA VEZ QUE O DOCUMENTO APRESENTADO NAO CONCEDE AO PROCURADOR PODERES DE REPRESENTACAO JUDICIAL PARA RECEBER CITACOES EM NOME DO OUTORGANTE, CONFORME O DISPOSTO NO ART. 217 DA LEI NO 9.279/96. |
|
ENP | Entry into the national phase |
Ref document number: PI1006035 Country of ref document: BR Kind code of ref document: A2 Effective date: 20110630 |