WO2013031392A1

WO2013031392A1 - 3d imaging device

Info

Publication number: WO2013031392A1
Application number: PCT/JP2012/067786
Authority: WO
Inventors: 克俊井澤; 林　淳司; 智行河合; 沢地　洋一
Original assignee: 富士フイルム株式会社
Priority date: 2011-08-30
Filing date: 2012-07-12
Publication date: 2013-03-07

Abstract

A 3D imaging device (10) pertaining to one embodiment of the present invention is provided with a first and second imaging unit (11-1, 11-2) that are both monocular 3D imaging units. The 3D imaging device (10) is further provided with a cross-point control unit (40), and the cross-point control unit controls an image generation unit in a manner so that the cross-point of the 3D images displayed at an image display unit (30) does not change before and after switching when switching from imaging by means of a multi-eye 3D imaging function to a monocular 3D imaging function, and when switching from imaging by means of the monocular 3D imaging function to imaging by means of the multi-eye 3D imaging function. As a result, it is possible to suppress large changes in pop-out state resulting from changes in the cross-point, and it is possible to display an image with a natural feeling when switching imaging modes.

Description

Stereo imaging device

The present invention relates to a stereoscopic imaging apparatus, and more particularly to a stereoscopic imaging apparatus provided with a plurality of optical systems.

Recently, stereoscopic display devices have been developed for displaying a stereoscopic image. For example, in the stereoscopic display device described in Patent Document 1, the cross point of a stereoscopic image is adjusted according to conditions such as the size of the display device and the distance between the viewer and the display device. In addition, stereoscopic cameras capable of acquiring stereoscopic image data have been used. For example, in the stereoscopic camera described in Patent Document 2, stereoscopic image data is acquired using two (left and right) lenses and one image sensor having different sensitivity depending on the incident angle. The imaging device described in Document 3 includes a plurality of optical systems, and can switch between monocular imaging and compound-eye imaging so that a stereoscopic image can be acquired.

WO2004 / 082297 JP 2007-279512 A JP 2011-030123 A

However, the technique described in Patent Document 1 adjusts the cross point according to the conditions at the time of image reproduction. When switching and displaying a stereoscopic image shot in a different shooting mode, the cross point is shifted and the user is informed. It will give an unnatural feeling. Further, in the techniques described in Patent Document 2 and Patent Document 3, the settable shooting modes are limited, and images cannot be acquired in shooting modes arbitrarily selected from various shooting modes.

The present invention has been made based on such circumstances, and an object thereof is to provide a stereoscopic imaging apparatus capable of displaying an image with a natural feeling when the shooting mode is switched. It is another object of the present invention to provide a stereoscopic imaging apparatus that can acquire various images according to a user's request.

In order to achieve the above object, a stereoscopic imaging apparatus according to the first aspect of the present invention is a plurality of imaging units that capture an image of a subject, and a light beam that has passed through different areas of a single imaging optical system is a pixel group. A plurality of imaging units including at least one monocular stereoscopic imaging unit having an imaging device including a plurality of pixel groups that perform photoelectric conversion for each, an image generation unit that generates a stereoscopic image of a subject from imaging signals of the plurality of imaging units, A stereoscopic imaging device including a crosspoint control unit that controls a crosspoint of a stereoscopic image, wherein the image generation unit configures a stereoscopic image from a plurality of viewpoint images obtained by photographing with the monocular stereoscopic imaging unit. A composite image that forms a stereoscopic image from an imaging function, a viewpoint image obtained by at least one monocular stereoscopic imaging unit, and a viewpoint image obtained by an imaging unit other than the monocular stereoscopic imaging unit among a plurality of imaging units. The cross-point control unit when switching from shooting with the compound eye stereoscopic imaging function to shooting with the monocular stereoscopic imaging function, and when switching from shooting with the anterior eye stereoscopic imaging function to shooting with the compound eye stereoscopic imaging function In addition, the image generation unit is controlled so that the cross point of the displayed stereoscopic image does not change before and after the switching.

In a stereoscopic image obtained by the monocular stereoscopic imaging function, a focused point in the image becomes a cross point, whereas in a stereoscopic image obtained by the compound-eye stereoscopic imaging function, the in-focus position in the image is crossed. It is different from the point. Therefore, when the monocular stereoscopic imaging and the compound eye stereoscopic imaging are switched to each other, the cross point changes before and after the switching, so that the viewer may feel uncomfortable that the subject jumps forward or retracts backward. Such a sense of incongruity is particularly felt when the pop-out state of the main subject that is often in focus changes. Therefore, in the stereoscopic imaging device according to the first aspect of the present invention, a cross-point control unit is provided to switch from imaging with the compound eye stereoscopic imaging function to imaging with the monocular stereoscopic imaging function, and from imaging with the monocular stereoscopic imaging function to compound eye stereoscopic imaging. When switching to shooting by function, the image generation unit is controlled so that the cross point of the stereoscopic image displayed on the image display unit does not change before and after the switching. As a result, a large change in the pop-out state due to a change in cross point can be suppressed, and an image can be displayed with a natural feeling when the shooting mode is switched.

In the present invention, the “cross point” means a point where parallax is zero in a stereoscopic image. During stereoscopic image playback, a subject that is in front of the cross point appears to protrude (front) from the screen, and a subject that is behind the cross point appears to be retracted (back) from the screen.

As shown in the second aspect of the present invention, in the stereoscopic imaging device according to the first aspect, the crosspoint control unit performs imaging with the compound-eye stereoscopic imaging function while imaging with the compound-eye stereoscopic imaging function continues. The first in-focus area in one viewpoint image among the viewpoint images constituting the stereoscopic image obtained by the above is detected, and the detected first in the other viewpoint images among the viewpoint images constituting the stereoscopic image is detected. A first corresponding area corresponding to the in-focus area is detected, and based on the detected first corresponding area, a first misregistration amount between one viewpoint image and another viewpoint image is calculated. By moving one viewpoint image or another viewpoint image by the calculated first positional deviation amount, the cross point of the stereoscopic image obtained by imaging with the compound eye stereoscopic imaging function can be captured with the monocular stereoscopic imaging function. Of stereoscopic images obtained by Controls the image generating unit to match the first focus area is loss point, it may be.

In the second aspect, while the imaging with the compound eye stereoscopic imaging function is continued so that the cross point of the stereoscopic image does not shift when the compound eye stereoscopic imaging function is switched to the monocular imaging function, The cross point of the stereoscopic image obtained by photographing is shifted in advance so as to coincide with the first in-focus area that is the cross point of the stereoscopic image obtained by photographing with the monocular stereoscopic imaging function. As a result, the cross point does not change suddenly when actually switching from the compound-eye stereoscopic imaging function to the monocular stereoscopic imaging function, and an image can be displayed with a natural feeling.

As shown in the third aspect of the present invention, in the stereoscopic imaging device according to the first or second aspect, the crosspoint control unit is configured to provide a monocular stereoscopic imaging function while shooting with the monocular stereoscopic imaging function is continued. When one of the viewpoint images constituting the stereoscopic image obtained by shooting at the point of view, the second in-focus area which is a cross point of the stereoscopic image is detected, and shooting with the compound eye stereoscopic imaging function is performed. A second correspondence corresponding to the detected second in-focus area in another viewpoint image obtained by an imaging unit other than the imaging unit used for shooting with the monocular stereoscopic imaging function. A region is detected, a second positional deviation amount between one viewpoint image and another viewpoint image is calculated based on the detected second corresponding region, and the already stored positional deviation amount is calculated as the second position. Update and store the displacement amount, monocular 3D photography When the function is switched to the compound eye stereoscopic imaging function, a compound image is formed by constructing a stereoscopic image from one viewpoint image and an image obtained by shifting the other viewpoint image by the second positional shift amount stored. The image generation unit is controlled so that the cross point of the stereoscopic image obtained by shooting with the stereoscopic imaging function matches the second in-focus area that is the cross point of the stereoscopic image obtained by shooting with the monocular stereoscopic imaging function. You may do it.

In the third aspect, the second positional shift amount with respect to the viewpoint image is calculated and updated in advance while shooting with the monocular stereoscopic imaging function is continued, and switching from the monocular stereoscopic imaging function to the compound eye stereoscopic imaging function is performed. When there is, a stereoscopic image is formed after shifting the viewpoint image newly obtained by switching to the compound eye stereoscopic imaging function by shifting the second positional deviation amount. As a result, when the monocular stereoscopic imaging function is actually switched to the compound eye stereoscopic imaging function, the cross point does not suddenly change greatly, and an image can be displayed with a natural feeling.

As shown in the fourth aspect of the present invention, in the stereoscopic imaging device according to any one of the first to third aspects, an imaging function automatic switching unit that automatically switches between a monocular stereoscopic imaging function and a compound-eye stereoscopic imaging function. In addition, the imaging function automatic switching unit operates the monocular stereoscopic imaging function while the in-focus position is closer than a predetermined distance, and switches to the compound eye stereoscopic imaging function when the in-focus position becomes farther than the predetermined distance, and the cross point The control unit is operated to operate the compound eye stereoscopic imaging function while the in-focus position is closer than a predetermined distance, and the user is instructed to switch to the monocular stereoscopic imaging function when the in-focus position is closer than the predetermined distance. In such a case, it may be switched to the monocular stereoscopic imaging function.

When comparing single-eye stereoscopic imaging and compound-eye stereoscopic imaging, it is generally possible to use either the single-eye stereoscopic imaging function or the compound-eye stereoscopic imaging function for long-distance shooting, but for short-distance shooting, monocular stereoscopic imaging with a short baseline length is a compound-eye stereoscopic imaging. Since it is more advantageous than imaging, photographing with a monocular stereoscopic imaging function is preferable. Therefore, as described above, the switching between the monocular stereoscopic imaging function and the compound eye stereoscopic imaging function is performed in consideration of the shooting distance and the change request by the user, and the switching of the monocular / compound eye stereoscopic imaging function is described above. It is preferable to control the cross point. By performing such control, even when the monocular stereoscopic imaging function and the compound eye stereoscopic imaging function are automatically switched, shooting is performed without giving the user the impression that the cross point has suddenly changed due to the switching. it is possible, it is possible to display the captured three-dimensional image in a natural feeling.

As shown in the fifth aspect of the present invention, in the stereoscopic imaging device according to any one of the first to fourth aspects, the image generation unit is an imaging element included in the monocular stereoscopic imaging unit during the operation of the compound eye stereoscopic imaging function. The pixel signal of each pixel group which comprises a some pixel group may be added for every pixel position of this, and the pixel signal addition process which makes the said addition result the pixel signal in each pixel position may be performed. By adding a plurality of pixel signals in this manner, the amount of noise in the generated image can be reduced, and various images can be acquired according to the user's request.

In order to achieve the above object, a stereoscopic imaging apparatus according to a sixth aspect of the present invention is a plurality of imaging units that capture an image of a subject, and a light beam that has passed through different areas of a single imaging optical system is grouped into a pixel group. A plurality of imaging units including at least one monocular stereoscopic imaging unit having an imaging element including a plurality of pixel groups that perform photoelectric conversion, an image generation unit that generates a stereoscopic image of a subject from imaging signals of the plurality of imaging units, and a user An imaging mode setting unit that sets an imaging mode based on an instruction input, and the imaging mode setting unit obtains the number of imaging units to be used for imaging among a plurality of imaging units and acquires the imaging mode setting unit. Based on a user instruction input including the number of viewpoint images, two-dimensional imaging mode and monocular stereoscopic imaging mode using at least one monocular stereoscopic imaging unit, and at least one monocular stereoscopic imaging unit and a plurality of imaging units Without even sets an imaging mode from among the imaging modes including two-dimensional imaging mode and the compound-eye stereoscopic imaging mode, using the image pickup unit other than the one monocular stereoscopic imaging unit.

In the stereoscopic imaging device according to the sixth aspect, the size of the parallax, the number of viewpoints, and the like differ by changing the number of imaging units used for shooting and the number of viewpoint images to be acquired (number of viewpoints) among the plurality of imaging units. Various imaging modes can be set. For example, the two-dimensional imaging mode or the stereoscopic imaging mode can be selected, and the monocular stereoscopic imaging mode or the compound eye stereoscopic imaging mode can be selected also in the stereoscopic imaging mode. As described above, the stereoscopic imaging device according to the second aspect can acquire various images according to the user's request.

Note that in the stereoscopic imaging device according to the sixth aspect, as in the stereoscopic imaging devices according to the first to fifth aspects, crosspoint control and automatic switching of the monocular / compound eye stereoscopic imaging function may be performed. .

As shown in the seventh aspect of the present invention, in the stereoscopic imaging device according to the sixth aspect, the image generation unit is based on a user instruction input, and the two-dimensional imaging mode using at least one monocular stereoscopic imaging unit, In the compound-eye stereoscopic imaging mode using at least one monocular stereoscopic imaging unit and an imaging unit other than at least one monocular stereoscopic imaging unit among the plurality of imaging units, each of the imaging elements included in at least one monocular stereoscopic imaging unit For each pixel position, pixel signals of each pixel group constituting a plurality of pixel groups may be added, and pixel signal addition processing may be performed using the added result as a pixel signal at each pixel position. In this way, the amount of noise of an image generated by adding a plurality of pixel signals can be reduced, and more various images can be acquired according to the user's request.

As shown in the eighth aspect of the present invention, in the stereoscopic imaging device according to any one of the first to seventh aspects, the number of the plurality of imaging units is 2, and other than at least one monocular stereoscopic imaging unit. imaging unit also may be a monocular stereoscopic imaging unit.

As shown in the ninth aspect of the present invention, the stereoscopic imaging apparatus according to any of the first to eighth aspects may further include a stereoscopic image display unit that displays the generated stereoscopic image.

As described above, according to the stereoscopic imaging device of the present invention, it is possible to display an image with a natural feeling when the shooting mode is switched, and it is possible to acquire various images according to a user's request.

FIG. 1 is a block diagram showing a configuration of a stereoscopic imaging apparatus 10 according to the first embodiment of the present invention. Figure 2 is an image view showing an appearance of a stereoscopic imaging device 10. FIG. 3 is a diagram illustrating a configuration of an imaging element used in the monocular stereoscopic imaging unit. 4 is a diagram showing the main / sub-pixels of the image sensor shown in FIG. 3 one by one. FIG. 5A is a diagram showing a configuration of a normal CCD. Figure 5B is a diagram showing an example of a configuration of a monocular 3D sensor. FIG. 5C is a diagram illustrating another example of the configuration of the monocular 3D sensor. FIG. 6 is a block diagram illustrating a main part of the stereoscopic imaging apparatus according to the first embodiment. FIG. 7 is a flowchart showing the cross point control at the time of switching from the compound eye stereoscopic imaging function to the monocular stereoscopic imaging function. FIG. 8A is a conceptual diagram illustrating a relationship between a cross point and a focal point during compound eye stereoscopic imaging. FIG. 8B is a conceptual diagram illustrating a relationship between a cross point and a focal point during monocular stereoscopic imaging. FIG. 9 is another conceptual diagram showing the cross point control at the time of switching from the compound eye stereoscopic imaging function to the monocular stereoscopic imaging function. FIG. 10 is a flowchart showing the cross point control at the time of switching from the monocular stereoscopic imaging function to the compound eye stereoscopic imaging function. FIG. 11 is a conceptual diagram showing a relationship between a cross point and a focal point during monocular / compound eye stereoscopic imaging. FIG. 12 is a flowchart showing a process for automatically switching between the monocular / compound-eye stereoscopic imaging function. FIG. 13 is a table showing shooting modes that can be set in the stereoscopic imaging apparatus according to the first embodiment. FIG. 14 is a conceptual diagram illustrating a procedure for setting a shooting mode in the stereoscopic imaging apparatus according to the first embodiment. FIG. 15 is a conceptual diagram illustrating pixel signal addition processing in the stereoscopic imaging apparatus according to the first embodiment. FIG. 16 is a flowchart illustrating a procedure for selecting a shooting mode in consideration of pixel signal addition processing. FIG. 17 is a block diagram illustrating a main part of a stereoscopic imaging apparatus according to the second embodiment of the present invention. FIG. 18 is a table showing shooting modes that can be set in the stereoscopic imaging apparatus according to the second embodiment.

Hereinafter, embodiments for carrying out an imaging apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
[Overall configuration of imaging device]
FIG. 1 is a block diagram illustrating an embodiment of a stereoscopic imaging apparatus 10 according to the present invention, and FIG. 2 is an image diagram illustrating an external appearance of the stereoscopic imaging apparatus 10.

The stereoscopic imaging apparatus 10 displays a captured image on a liquid crystal monitor (LCD) 30 or records it on a memory card 54 (hereinafter also referred to as “media”). The overall operation of the apparatus is a central processing unit (CPU) 40. It is controlled by

The stereoscopic imaging device 10 is provided with operation units 38 such as a shutter button, a mode dial, a playback button, a MENU / OK key, a cross key, and a BACK key. A signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the stereoscopic imaging device 10 based on the input signal. For example, lens driving control, aperture driving control, photographing operation control, image processing control, image processing Data recording / reproduction control, display control of the liquid crystal monitor 30 for stereoscopic display, and the like are performed.

The shutter button is an operation button for inputting an instruction to start shooting, and is configured by a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. The mode dial is a selection means for selecting a 2D shooting mode, a 3D shooting mode, an auto shooting mode, a manual shooting mode, a scene position such as a person, a landscape, a night view, a macro mode, a moving image mode, and a parallax priority shooting mode according to the present invention. is there.

The playback button is a button for switching to a playback mode in which a still image or a moving image of a stereoscopic image (3D image) or a planar image (2D image) that has been recorded is displayed on the liquid crystal monitor 30. The MENU / OK key is an operation key having both a function as a menu button for instructing to display a menu on the screen of the liquid crystal monitor 30 and a function as an OK button for instructing confirmation and execution of the selection contents. It is. The cross key is an operation unit for inputting instructions in four directions, up, down, left, and right, and functions as a button (cursor moving operation means) for selecting an item from the menu screen or instructing selection of various setting items from each menu. To do. The up / down key of the cross key functions as a zoom switch for shooting or a playback zoom switch in playback mode, and the left / right key functions as a frame advance (forward / reverse feed) button in playback mode. . The BACK key is used to delete a desired object such as a selection item, cancel an instruction content, or return to the previous operation state.

In the photographing mode, the image light indicating the subject is a phase difference image sensor via the photographing lenses 12 (12-1, 12-2) including the focus lens and the zoom lens, and the aperture 14 (14-1, 14-2). Is imaged on the light receiving surface of a solid-state imaging device 16 (16-1, 16-2, hereinafter referred to as “monocular 3D sensor”). The photographing lenses 12 (12-1, 12-2) are driven by a lens driving unit 36 (36-1, 36-2) controlled by the CPU 40, and focus control, zoom control, and the like are performed. The diaphragm 14 (14-1, 14-2) is composed of, for example, five diaphragm blades, and is driven by a diaphragm driver 34 (34-1, 34-2) controlled by the CPU 40. For example, the diaphragm value F1 Aperture control is performed in 6 steps in increments of 1AV from .4 to F11.

The CPU 40 controls the diaphragm 14 (14-1, 14-2) via the diaphragm driving unit 34 (34-1, 34-2) and the CCD control unit 32 (32-1, 32-2). The charge accumulation time (shutter speed) in the monocular 3D sensor 16, the readout control of the image signal from the monocular 3D sensor 16, and the like are performed.

<Configuration Example of Monocular 3D Sensor>
FIG. 3 is a diagram illustrating a configuration example of the monocular 3D sensor 16.

As shown in FIGS. 3A to 3C, the monocular 3D sensor 16 includes odd-line pixels (main pixels) and even-line pixels (sub-pixels) arranged in a matrix. In addition, the image signals for the two surfaces photoelectrically converted by these main and sub-pixels can be read independently.

As shown in FIG. 2, among the pixels provided with R (red), G (green), and B (blue) color filters on the odd lines (1, 3, 5,...) Of the monocular 3D sensor 16, GRGR ... And BGBG... Pixel array lines are provided alternately. On the other hand, the pixels on the even lines (2, 4, 6,...) Are arranged in the GRGR. And BGBG... Pixel array lines are alternately provided, and the pixels are arranged so as to be shifted in the line direction by a half pitch with respect to the even-numbered pixels.

FIG. 4 is a diagram showing the main lens PDa and the sub-pixel PDb of the photographing lens 12 (shooting optical system), the diaphragm 14, and the monocular 3D sensor 16, and FIGS. 5A to 5C are main parts of FIG. It is an enlarged view.

As shown in FIG. 5A, the light beam passing through the exit pupil enters the normal CCD pixel (photodiode PD) via the microlens L without being restricted.

On the other hand, the monocular 3D sensor 16 shown in FIG. 5B includes a microlens L that collects the light beam that has passed through the photographing lens 12, and a photodiode PD that receives the light beam that has passed through the microlens L (the main pixel PDa and the subpixel). PDb) and a light shielding member 16A that partially shields the light receiving surface of the photodiode PD. In this example, the right half or the left half of the light receiving surfaces of the main pixel PDa and the subpixel PDb is shielded by the light shielding member 16A. That is, the light shielding member 16A functions as a pupil division member.

The monocular 3D sensor 16 having the above-described configuration is configured such that the main pixel PDa and the sub-pixel PDb have different regions (right half and left half) where the light beam is limited by the light shielding member 16A. For example, as shown in FIG. 5C, the microlens L and the photodiode PD (PDa, PDb) are relatively shifted in the left-right direction (pupil division direction) without providing the light shielding member 16A. The light beam incident on the photodiode PD may be limited by disposing the optical axis Ic of the photodiode PD and the optical axes Pc of the photodiodes PDa and PDb. Further, by providing one microlens for two pixels (main pixel and subpixel), the light flux incident on each pixel may be limited.

Returning to FIG. 1, the signal charge accumulated in the monocular 3D sensor 16 (16-1, 16-2) is read out as a voltage signal corresponding to the signal charge based on the readout signal applied from the CCD controller 32. The voltage signal read from the monocular 3D sensor 16 (16-1, 16-2) is applied to the analog signal processing unit 18 (18-1, 18-2), where R, G, The B signal is sampled and held, amplified by a gain designated by the CPU 40 (corresponding to ISO sensitivity), and then added to the A / D converter 20 (20-1, 20-2). The A / D converter 20 (20-1, 20-2) sequentially converts the input R, G, B signals into digital R, G, B signals and converts them into image input controllers 22 (22-1, 22-22). Output to 2).

A first photographing lens 12-1, a first diaphragm 14-1, a first monocular 3D sensor 16-1, a first analog signal processing unit 18-1, a first A / D converter 20-1, The first image input unit 22-1, the first CCD control unit 32-1, the first aperture driving unit 34-1 and the first lens driving unit 36-1 are used to form the first imaging unit 11-. 1 is configured. The second photographing lens 12-2, the second diaphragm 14-2, the second monocular 3D sensor 16-2, the second analog signal processing unit 18-2, and the second A / D converter 20- 2, the second image input controller 22-2, the second CCD control unit 32-2, the second diaphragm driving unit 34-2, and the second lens driving unit 36-2 11-2 is configured.

The digital signal processing unit 24 performs gain control processing including gamma correction processing, gamma correction processing, synchronization processing, YC processing for digital image signals input via the image input controller 22, including offset processing, white balance correction, and sensitivity correction. Then, predetermined signal processing such as sharpness correction is performed.

The EEPROM 46 stores a camera control program, defect information of the monocular 3D sensor 16, various parameters and tables used for image processing, a program diagram, a plurality of parallax priority program diagrams according to the present invention, and the like. It is a non-volatile memory.

Here, as shown in FIGS. 3B and 3C, the main image data read from the odd-line main pixels of the monocular 3D sensor 16 is processed as the left viewpoint image data and read from the even-line sub-pixels. The sub image data to be processed is processed as right viewpoint image data.

The left viewpoint image data and right viewpoint image data (3D image data) processed by the digital signal processing unit 24 are input to the VRAM 50. The VRAM 50 includes an A area and a B area each storing 3D image data representing a 3D image for one frame. In the

VRAM

50, 3D image data representing a 3D image for one frame is rewritten alternately in the A area and the B area. The written 3D image data is read from an area other than the area in which the 3D image data is rewritten in the A area and the B area of the VRAM 50. The 3D image data read from the VRAM 50 is encoded by the video encoder 28 and is output to the stereoscopic display liquid crystal monitor 30 provided on the back of the camera, whereby the 3D subject image is displayed on the display screen of the liquid crystal monitor 30. Is displayed.

The liquid crystal monitor (LCD) 30 is a stereoscopic display unit that can display a stereoscopic image (a left viewpoint image and a right viewpoint image) as a directional image having a predetermined directivity by a parallax barrier, but is not limited thereto. Alternatively, the left viewpoint image and the right viewpoint image may be viewed separately by using a lenticular lens or by wearing dedicated glasses such as polarized glasses or liquid crystal shutter glasses. In this embodiment, the case where the stereoscopic imaging apparatus 10 includes the liquid crystal monitor 30 capable of displaying a stereoscopic image has been described. However, the stereoscopic imaging apparatus 10 does not include the liquid crystal monitor 30 and is recorded on the memory card 54. A stereoscopic image may be viewed on another stereoscopic image display device using the image data.

In addition, when the shutter button of the operation unit 38 is pressed (half-pressed) in the first stage, the monocular 3D sensor 16 starts the AF operation and the AE operation, and the focus in the photographing lens 12 is set via the lens driving unit 36. Control is performed so that the lens comes to the in-focus position. The image data output from the A / D converter 20 when the shutter button is half-pressed is taken into the AE detection unit 44.

The AE detection unit 44 integrates the G signals of the entire screen or integrates the G signals that are weighted differently in the central portion and the peripheral portion of the screen, and outputs the integrated value to the CPU 40. The CPU 40 calculates the brightness of the subject (shooting EV value) from the integrated value input from the AE detection unit 44, and the aperture value of the diaphragm 14 and the electronic shutter (shutter speed) of the monocular 3D sensor 16 based on the shooting EV value. Is determined according to a predetermined program diagram, the aperture 14 is controlled via the aperture drive unit 34 based on the determined aperture value, and the monocular 3D sensor is controlled via the CCD control unit 32 based on the determined shutter speed. The charge accumulation time at 16 is controlled.

The AF processing unit 42 is a part that performs contrast AF processing. In the contrast AF process, a high frequency component of image data in a predetermined focus area is extracted from at least one of the left viewpoint image data and the right viewpoint image data, and the in-focus state is determined by integrating the high frequency component. An AF evaluation value is calculated. AF control is performed by controlling the focus lens in the photographic lens 12 so that the AF evaluation value is maximized. Note that phase difference AF processing may be performed. In this case, the phase difference between the image data corresponding to the main pixel and the sub-pixel in the predetermined focus area of the left viewpoint image data and the right viewpoint image data is detected. The defocus amount is obtained based on the information indicating the phase difference. AF control is performed by controlling the focus lens in the taking lens 12 so that the defocus amount becomes zero.

When the AE operation and the AF operation are completed and the shutter button is pressed in the second stage (full press), the left corresponding to the main pixel and the sub-pixel output from the A / D converter 20 in response to the press. Two pieces of image data of the viewpoint image (main image) and the right viewpoint image (sub-image) are input from the image input controller 22 to the memory (SDRAM) 48 and temporarily stored.

The two pieces of image data temporarily stored in the memory 48 are appropriately read out by the digital signal processing unit 24, where predetermined signals such as luminance data and color difference data generation processing (YC processing) are performed. Processing is performed. The YC processed image data (YC data) is stored in the memory 48 again. Subsequently, the two pieces of YC data are respectively output to the compression / decompression processing unit 26 and subjected to predetermined compression processing such as JPEG (joint photographic experts group), and then stored in the memory 48 again.

A multi-picture file (MP file: a file in a format in which a plurality of images are connected) is generated from two pieces of YC data (compressed data) stored in the memory 48, and the MP file is generated by the media controller 52. It is read and recorded in the memory card 54.

The stereoscopic imaging device 10 having the configuration shown in FIG. 1 includes a first imaging unit 11-1 and a second imaging unit 11-2 that image a subject, and a first imaging unit 11-1 and a second imaging unit. A CPU 40 is provided as a control unit for controlling 11-2. Both the first imaging unit 11-1 and the second imaging unit 11-1 are a plurality of units that photoelectrically convert light beams that have passed through different areas of the exit pupil of the photographing lens 12 (12-1, 12-2). An image sensor (monocular 3D sensors 16-1 and 16-2) including a pixel group is included. The stereoscopic imaging device 10 uses a viewpoint image (image information) obtained by the first imaging unit 11-1 and a viewpoint image (image information) obtained by the second imaging unit 11-2 as a stereoscopic image on the liquid crystal monitor. And a plurality of viewpoint images (a plurality of viewpoint images obtained by one imaging unit having a plurality of pixel groups among the first imaging unit 11-1 and the second imaging unit 11-2). The image information) on the liquid crystal monitor 30 as a stereoscopic image. In the stereoscopic imaging apparatus 10 according to the first embodiment, as illustrated in the image diagram of FIG. 2, the imaging unit on the right side (left side as viewed from the user) is the “first imaging unit”. The imaging unit on the left side when viewed from the right side (right side when viewed from the user) is described as a “second imaging unit”.

[Cross-point control when switching between monocular / compound-eye stereoscopic imaging functions]
Next, the cross point control at the time of switching the monocular / compound eye stereoscopic imaging function in the stereoscopic imaging apparatus 10 according to the first embodiment will be described. 8A and 8B are conceptual diagrams showing the relationship between the cross point and the focus during monocular / compound eye stereoscopic imaging. In the present invention, the cross point means a point where parallax is zero in a stereoscopic image.

In general, at the time of compound eye stereoscopic imaging, the left and right image data can be electronically shifted to freely set the cross point. Therefore, as shown in the example of FIG. 8A, the cross point (object B) and the focal point (object A ) Is often different. On the other hand, at the time of monocular three-dimensional imaging, since a three-dimensional effect is generated by dividing an image blur circle to the left and right, as shown in the example of FIG. It automatically becomes a crosspoint (object A). Therefore, when switching between monocular stereoscopic imaging and compound eye stereoscopic imaging, it may appear to the viewer that the cross point has suddenly shifted. From such a viewpoint, the stereoscopic imaging apparatus 10 according to the first embodiment can perform the following crosspoint control. Such cross-point control is particularly effective when shooting continuously, that is, when shooting moving images and acquiring so-called through images. Note that whether or not to perform crosspoint control may be determined by a user input via the operation unit 38.

<When switching from the compound-eye stereoscopic imaging function to the monocular stereoscopic imaging function>
FIG. 7 is a flowchart showing the cross point control at the time of switching from the compound eye stereoscopic imaging function to the monocular stereoscopic imaging function, and FIG. 9 is a conceptual diagram showing the state of the cross point control. When the compound eye stereoscopic photographing mode (compound eye 3D mode) is set by the operation of the operation unit 38 in S102, left and right viewpoint images are acquired in S104. Here, the left viewpoint image (L in FIG. 9A) is acquired by the left channel of the first imaging unit 11-1, and the right viewpoint image (FIG. 9) is acquired by the right channel of the second imaging unit 11-2. R) of (a) shall be acquired. Here, it is assumed that the contrast AF process is performed using the left viewpoint image L acquired in the left channel of the first imaging unit 11-1. In the example of FIG. 9A, the cross point is object B and the focal point is object A, as in FIG. 8A.

In S106, for detecting a focused area in the left viewpoint image L. Here, as shown in FIG. 9B, it is assumed that focusing is performed in region S1 (the contrast is maximum). In S108, a region corresponding to the region S1 is detected in the right viewpoint image R. In the example of FIG. 9, this area is the area S2 of (c) of FIG. This corresponding area detection can be performed by various methods such as a correlation method and template matching. However, in order to reliably detect, it is preferable to set the area corresponding to the AF area in the left viewpoint image L as a detection target. .

In S110, an amount (shift amount ΔX) of moving the right viewpoint image R so that the cross point becomes object A is calculated. In S112, the right viewpoint image R is moved by ΔX and (the right viewpoint of FIG. 9D) The stereoscopic image composed of the image R ′), the left viewpoint image L and the right viewpoint image R ′ is displayed on the liquid crystal monitor 30. The three-dimensional image may be recorded on the memory card 54.

Through the processing up to S112, the cross point of the stereoscopic image obtained by the compound eye stereoscopic imaging is object A, which matches the in-focus area. Such processing is performed while the compound eye stereoscopic photography is continued, and the state where the cross point and the in-focus point coincide with each other is maintained until switching from the compound eye stereoscopic photography to the monocular stereoscopic photography. Such processing may be performed at predetermined time intervals, for example, 100 msec intervals.

When switching from compound eye stereo photography to monocular stereo photography (YES in S116), monocular stereo photography using a monocular stereo imaging unit is performed. Here, it is assumed that monocular stereoscopic imaging is performed using the first imaging unit 11-1 which is a monocular stereoscopic imaging unit. In the state in which the compound eye stereoscopic shooting is switched to the monocular stereoscopic shooting, the state up to the cross point is maintained by the processing up to S114, and as described above, the monocular stereoscopic shooting matches the cross point. Since the focal points coincide with each other, even if switching from compound eye stereoscopic photography to monocular stereoscopic photography, the viewer does not feel that the cross point has changed suddenly. Therefore, an image can be displayed with a natural feeling when the shooting mode is switched.

<When switching from monocular stereo imaging function to compound eye stereo imaging function>
Next, cross point control at the time of switching from the monocular stereoscopic imaging function to the compound eye stereoscopic imaging function will be described. FIG. 10 is a flowchart showing the procedure of such crosspoint control, and FIG. 11 is a conceptual diagram showing the state of the crosspoint control. When the single-eye stereoscopic photographing mode (monocular 3D mode) is set by operating the operation unit 38 in S202, left and right viewpoint images are acquired in S204. Here, it is assumed that the left viewpoint image is acquired by the left channel of the first imaging unit 11-1, and the right viewpoint image is acquired by the right channel of the first imaging unit 11-1. In this case, since monocular stereoscopic shooting is performed, the stereoscopic image is composed of two viewpoint images with small parallax, as shown in FIG. In addition, since it is monocular stereoscopic photography, the cross point and the focal point coincide (object A in FIG. 11A). In S206, the viewpoint image obtained in this way is displayed on the liquid crystal monitor 30 as a stereoscopic image.

In S208, the in-focus area is detected in the image acquired by the first imaging unit 11-1. As described above, since the cross point coincides with the focal point at the time of monocular stereoscopic photography, as shown in (b) and (c) of FIG. By detecting as a cross point, the in-focus area can be detected. The focus area may be detected by one of the left and right viewpoint images. In order to perform processing such as corresponding point detection described below, a viewpoint image by the second imaging unit 11-2 is also acquired in S210 ((d) in FIG. 11).

Next, an area corresponding to the detected in-focus area is detected in the viewpoint image acquired by the second imaging unit 11-2 (S212). The corresponding area can be detected by an algorithm such as a correlation method as described above. Then, based on the detected corresponding area, a deviation amount between the viewpoint image acquired by the first imaging unit 11-1 and the viewpoint image acquired by the second imaging unit 11-2 is calculated ((e) in FIG. 11). ) Updates the stored value and stores it (S214). This shift amount is an image shift amount for causing the cross point and the in-focus area to coincide with each other in the stereoscopic image obtained when switching to the compound eye stereoscopic shooting. Such processing is performed while monocular stereoscopic photography continues, and the calculation, update, and recording of the deviation amount are continued until switching from monocular stereoscopic photography to compound eye stereoscopic photography. Such processing may be performed at predetermined time intervals, for example, 100 msec intervals.

When switching from single-eye stereoscopic photography to compound-eye stereoscopic photography (YES in S216), the viewpoint image acquired by the second imaging unit 11-2 in S218 is shifted by the above deviation amount ((f) in FIG. 11), The stereoscopic image is displayed on the liquid crystal monitor 30 together with the viewpoint image acquired by the first imaging unit 11-1 (S220). As described above, the viewpoint image acquired by the second imaging unit 11-2 is shifted by the amount of shift, so that the stereoscopic image obtained when switching to compound eye stereoscopic shooting is aligned with the cross point as in monocular stereoscopic shooting. The focal area matches. Therefore, even when switching from single-eye stereoscopic photography to compound-eye stereoscopic photography, the viewer does not feel that the cross point has changed suddenly, and an image can be displayed with a natural feeling when the photographing mode is switched.

[Automatic crosspoint adjustment]
The cross point control at the time of switching to monocular / compound eye stereoscopic photography has been described above. Next, automatic activation of cross point adjustment will be described.

When comparing single-eye stereoscopic imaging and compound-eye stereoscopic imaging, it is generally possible to use either the single-eye stereoscopic imaging function or the compound-eye stereoscopic imaging function for long-distance shooting, but for short-distance shooting, monocular stereoscopic imaging with a short baseline length is a compound-eye stereoscopic imaging. Since it is more advantageous than imaging, photographing with a monocular stereoscopic imaging function is preferable. Therefore, the cross-point adjustment function only needs to be activated when switching to long-distance shooting for short-distance shooting, but for long-distance shooting, it is not only when switching to short-distance shooting but also for monocular / compound-eye stereoscopic shooting by the user. It is necessary to always activate it (so that the crosspoint is adjusted continuously) so that it can be handled even when there is a change request.

Specifically, as shown in the flowchart of the automatic activation process for crosspoint adjustment shown in FIG. 12, first, the user selects whether to automatically switch between monocular / compound-eye stereoscopic photography (S302). / The user is allowed to select any one of the compound eye stereoscopic photographing (S304). If YES in step S302, the process proceeds to step S306, where the in-focus position is detected, and if close to a predetermined threshold (for example, 70 cm) (YES in step S306), monocular stereoscopic shooting is performed (S308). This determination is continuously performed at a predetermined time interval (for example, 100 msec) (S310). If the in-focus position is farther than the predetermined threshold value and short-distance shooting is no longer possible (NO in S312), compound eye stereoscopic shooting is performed (S314), and automatic cross-point adjustment is performed in preparation for changing to short-distance shooting. The function is activated to match the cross point with the in-focus position (S316).

If NO in S306, that is, if the in-focus position is the same as or farther than the predetermined threshold value, the compound eye stereoscopic shooting is performed, and the crosspoint automatic adjustment function is activated in preparation for a change to short-distance shooting. And the in-focus position are matched (S316).

After the state of the compound eye stereoscopic photographing / cross point automatic adjustment function is activated in S316, it is determined whether or not to switch to monocular stereoscopic photographing at a predetermined time interval (S318) (S320). If the in-focus distance is shorter than the predetermined threshold value in S320, or if a change request to monocular stereoscopic photography is made by a user instruction via the operation unit 38, the process returns to S308 and monocular stereoscopic photography is performed. If none of the conditions is satisfied in S320, the process returns to S316 to continue the state of the compound-eye stereoscopic photographing and the automatic activation of the crosspoint function.

With the above processing, even when monocular / compound-eye stereoscopic shooting is automatically switched, shooting can be performed without giving the user an impression that the cross point has suddenly shifted, and an image can be displayed with a natural feeling. it can.

[Shooting mode settings]
As described above, in the stereoscopic imaging device 10 according to the first embodiment, both the first imaging unit 11-1 and the second imaging unit 11-2 have a monocular stereoscopic imaging function. Therefore, if left and right viewpoint images are acquired by both of the two imaging units, a total of four viewpoints can be obtained. If the number of viewpoints is large, an improvement in performance can be expected when taking the corresponding points and measuring the amount of parallax.

When it is not necessary to acquire four viewpoints, the first imaging unit 11-1 acquires the left viewpoint image, and the second imaging unit 11-2 acquires the right viewpoint image. Alternatively, the left viewpoint image may be acquired by the second imaging unit 11-2, and the right viewpoint image may be acquired by the first imaging unit 11-1. In this case the latter is obtained parallax slightly smaller. Further, in both the first imaging unit 11-1 and the second imaging unit 11-2, if the pixel value is obtained by performing pixel addition as shown by the dotted line in FIG. 15, two viewpoint images with reduced noise are obtained. It is possible to obtain.

If the first imaging unit 11-1 or the second imaging unit 11-2 alone separates light incident from the left and light incident from the right, two (left and right) viewpoint images with small parallax can be obtained. Can do. A stereoscopic image composed and displayed from a viewpoint image with a small parallax has less presence than a stereoscopic image with a large parallax, but “is less tiring” and “when displayed on a 3D (three-dimensional) TV” , It has an advantage of being viewed as a normal 2D (two-dimensional) image by a person who is not wearing glasses (not a double image).

Of course, when a single imaging unit is used, only one viewpoint image may be obtained without separating the left and right incident light, that is, a two-dimensional image may be obtained.

As described above, the number of viewpoints, the amount of parallax, the amount of noise, and the like differ depending on the combination of the number of imaging units used for shooting, the number of viewpoint images to be acquired, or the presence or absence of pixel addition. The image according to the user's request can be acquired. The table shown in FIG. 13 summarizes such shooting modes that can be set by the stereoscopic imaging apparatus 10.

FIG. 14 is a diagram showing an example of a specific procedure for setting the shooting mode shown in FIG. In the stereoscopic imaging apparatus 10, the interface shown in FIG. 14 can be displayed on the liquid crystal monitor 30, and the shooting mode can be set by a user instruction input via the operation unit 38.

The stereoscopic imaging apparatus 10 first displays the screen shown in FIG. 14A on the liquid crystal monitor 30 and prompts the user to input the number of viewpoints (1, 2, or 4). Here, in the case of four viewpoints, it is determined by the shooting mode [1] shown in FIG. 13 ((e) of FIG. 14). In the case of one viewpoint or two viewpoints, the stereoscopic imaging apparatus 10 prompts the user for further input. That is, which one of the first and second imaging units 11-1 and 11-2 is used for one viewpoint (FIG. 14B), two viewpoint images are obtained by monocular stereoscopic imaging for two viewpoints. Or whether to acquire two viewpoint images by compound eye stereoscopic photography ((c) of FIG. 14). In the case of one viewpoint, depending on whether the first stereoscopic imaging unit 11-1 is used (shooting from the left viewpoint) or the second stereoscopic imaging unit 11-2 is used (shooting from the right viewpoint), the shooting mode [7] or [8] In the case of two viewpoints, in the case of monocular stereoscopic imaging, the imaging mode [7] or [8] is set depending on whether the first stereoscopic imaging unit 11-1 or the second stereoscopic imaging unit 11-2 is used.

In the case of compound eye stereoscopic imaging with two viewpoints (that is, using the first and second optical systems), the stereoscopic imaging device 10 prompts the input of the amount of parallax as shown in FIG. Then, one of the shooting modes [2] to [4] is set according to the input amount of parallax ((e) in FIG. 14).

Note that the shooting mode setting is not limited to the above example. For example, the first imaging unit 11-1 may be selected unconditionally without selecting the first / second imaging unit at the time of one viewpoint. This is because it is expected that the use of the imaging unit opposite to the side where the shutter button is located is relatively less likely to have a finger (a user's finger covers the lens).

[Automatic selection of monocular / compound eye photography mode]
FIG. 15 is a conceptual diagram regarding addition of pixel signals. The amount of noise can be reduced by adding the pixel signals of the two pixels of the image sensor 16 as indicated by the dotted line in FIG. On the other hand, the high-frequency signal may become dull and the resolution may be lowered. Therefore, in the stereoscopic imaging device 10, during stereoscopic imaging from two viewpoints, according to the brightness of the screen, the contrast of the signal, etc., “the viewpoint image is acquired by compound eye stereoscopic imaging, and the result of adding the pixel signals of two pixels is captured for each imaging. The image can be automatically selected from the following: “Make a viewpoint image of a part” and “Shoot with monocular three-dimensional imaging and do not add pixel signals”.

FIG. 16 is a flowchart showing an example of such an automatic selection process of the stereoscopic shooting mode. When the process is started (S400), it is determined in S402 whether the luminance (Bv value) of the entire screen is equal to or greater than a predetermined threshold value. If it is equal to or greater than the predetermined threshold (YES in S402), the process proceeds to S406, and it is determined whether or not the response of the contrast extraction filter is equal to or greater than the predetermined threshold. If YES in S406, monocular stereoscopic shooting is performed with the shooting mode set to [5] or [6] (S408), and if NO, compound eye stereoscopic shooting is performed with the shooting mode set to [3] (S404). On the other hand, when the luminance (Bv value) of the entire screen does not reach the predetermined threshold value (NO in S402), compound eye stereoscopic shooting is performed with the shooting mode set to [3] (S404). By selecting the stereoscopic shooting mode in this way, it is possible to perform shooting according to the user's request regarding the amount of noise and the resolution of the image.

<Second Embodiment>
In the first embodiment described above, the case where both the first imaging unit and the second imaging unit stage are monocular stereoscopic imaging units has been described, but the embodiment of the present invention is limited to such a mode. is not. In the stereoscopic imaging device of the present invention, at least one of the plurality of imaging units may be a monocular stereoscopic imaging unit, and the types of other imaging units are not particularly limited. For example, a normal imaging unit (an imaging unit that is not a monocular stereoscopic imaging unit) that does not include an imaging device that can separate the left and right incident light may be used.

FIG. 17 is a block diagram illustrating a main part of the stereoscopic imaging apparatus 10 ′ according to the second embodiment. In the stereoscopic imaging apparatus 10 ′, the first imaging unit 11-1 is a monocular stereoscopic imaging unit, and the second imaging unit 11-2 ′ is an imaging unit having a normal sensor 17. Since the configuration other than this is the same as that of the stereoscopic imaging apparatus 10 according to the first embodiment, the same reference numerals as those of the stereoscopic imaging apparatus 10 are used, and detailed description thereof is omitted.

In such a stereoscopic imaging apparatus 10 ′, similarly to the stereoscopic imaging apparatus 10 according to the first embodiment, monocular stereoscopic imaging / compound eye stereoscopic imaging is possible, and the cross point at the time of switching between monocular / compound stereoscopic imaging described above. Automatic activation of control and cross point adjustment can be performed. However, in the stereoscopic imaging device 10 ′, the second imaging unit is a normal imaging unit, and thus the shooting modes that can be set are different from those in the stereoscopic imaging device 10 according to the first embodiment. The shooting modes that can be set by the stereoscopic imaging device 10 'are shown in the table of FIG. The shooting mode can be set according to the number of viewpoints, presence / absence of pixel addition, and the like as in the case of the stereoscopic imaging apparatus 10.

As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

The order of execution of each process such as operation, procedure, step, and stage in the apparatus, system, program, and method shown in the claims, the description, and the drawings is as follows. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if it is described using “first”, “next”, etc. for the sake of convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10,10 '... Three-dimensional imaging device, 11-1 ... 1st imaging part, 11-2, 11-2' ... 2nd imaging part, 16, 16-1, 16-2 ... Solid-state image sensor (monocular 3D sensor), 17 ... normal sensor, 30 ... liquid crystal monitor, 38 ... operation part

Claims

At least one monocular three-dimensional imaging unit having a plurality of imaging units for imaging a subject, the imaging unit including a plurality of pixel groups that photoelectrically convert, for each pixel group, light beams that have passed through different regions of a single photographing optical system. A plurality of imaging units including two,
An image generation unit that generates a stereoscopic image of the subject from imaging signals of the plurality of imaging units;
A cross point control unit for controlling a cross point of the stereoscopic image;
A stereoscopic imaging device comprising:
The image generation unit
A monocular stereoscopic imaging function for constructing the stereoscopic image from a plurality of viewpoint images obtained by photographing by the monocular stereoscopic imaging unit;
Compound eye stereoscopic imaging that constitutes the stereoscopic image from the viewpoint image obtained by the at least one monocular stereoscopic imaging unit and the viewpoint image obtained by an imaging unit other than the monocular stereoscopic imaging unit among the plurality of imaging units. And having a function
The cross point control unit is configured to switch the stereoscopic image when switching from shooting by the compound eye stereoscopic imaging function to shooting by the monocular stereoscopic imaging function and when switching from shooting by the monocular stereoscopic imaging function to shooting by the compound eye stereoscopic imaging function. Controlling the image generation unit so that the cross point of the change does not change before and after the switching,
Stereo imaging device.
While the cross point control unit continues shooting with the compound eye stereoscopic imaging function,
Detecting a first in-focus area in one viewpoint image among viewpoint images constituting a stereoscopic image obtained by photographing with the compound-eye stereoscopic imaging function;
Detecting a first corresponding area corresponding to the detected first in-focus area in another viewpoint image among the viewpoint images constituting the stereoscopic image;
Based on the detected first corresponding region, a first positional deviation amount between the one viewpoint image and the other viewpoint image is calculated,
Moving the one viewpoint image or the other viewpoint image by the calculated first positional deviation amount;
By
The image so that a cross point of a stereoscopic image obtained by photographing with the compound eye stereoscopic imaging function matches the first in-focus area which is a cross point of a stereoscopic image obtained by photographing with the monocular stereoscopic imaging function. Control the generator,
The stereoscopic imaging apparatus according to claim 1.
While the crosspoint control unit continues shooting with the monocular stereoscopic imaging function,
Detecting a second in-focus area which is a cross point of the stereoscopic image in one of the viewpoint images constituting the stereoscopic image obtained by photographing with the monocular stereoscopic imaging function;
In the other viewpoint image obtained by the imaging unit that is used when shooting with the compound-eye stereoscopic imaging function and that is other than the imaging unit that is used when shooting with the monocular stereoscopic imaging function, the detected first Detecting a second corresponding area corresponding to the in-focus area of 2;
Based on the detected second corresponding region, a second positional deviation amount between the one viewpoint image and the other viewpoint image is calculated,
Update and store the already stored misregistration amount with the second misregistration amount;
When switching from the monocular stereoscopic imaging function to the compound eye stereoscopic imaging function, a stereoscopic image is generated from the one viewpoint image and an image obtained by shifting the other viewpoint image from the stored second misalignment amount. By composing the image,
The image so that a cross point of a stereoscopic image obtained by photographing with the compound eye stereoscopic imaging function coincides with the second in-focus area which is a cross point of a stereoscopic image obtained by photographing with the monocular stereoscopic imaging function. Control the generator,
The stereoscopic imaging apparatus according to claim 1 or 2.
Further comprising an imaging function automatic switching unit that automatically switches between the monocular stereoscopic imaging function and the compound eye stereoscopic imaging function, the imaging function automatic switching unit,
While the in-focus position is closer than a predetermined distance, operate the monocular stereoscopic imaging function, and when the in-focus position is farther than a predetermined distance, switch to the compound eye stereoscopic imaging function and operate the crosspoint control unit,
While the in-focus position is closer than a predetermined distance, the compound eye stereoscopic imaging function is operated, and when the in-focus position is closer than the predetermined distance or when the user gives an instruction to switch to the monocular stereoscopic imaging function, Switch to monocular 3D imaging function,
The three-dimensional imaging device according to claim 1.
The image generation unit, during the operation of the compound eye stereoscopic imaging function,
A pixel signal that adds the pixel signals of each pixel group constituting the plurality of pixel groups for each pixel position of the imaging element included in the monocular three-dimensional imaging unit, and uses the added result as a pixel signal at each pixel position Perform addition processing,
The three-dimensional imaging device according to claim 1.
At least one monocular three-dimensional imaging unit having a plurality of imaging units for imaging a subject, the imaging unit including a plurality of pixel groups that photoelectrically convert, for each pixel group, light beams that have passed through different regions of a single photographing optical system. A plurality of imaging units including two,
An image generation unit that generates a stereoscopic image of the subject from imaging signals of the plurality of imaging units;
An imaging mode setting unit for setting an imaging mode based on a user's instruction input;
A stereoscopic imaging device comprising:
The imaging mode setting unit is based on the user's instruction input including the number of imaging units to be used for imaging among the plurality of imaging units and the number of viewpoint images to be acquired.
Two-dimensional imaging mode and monocular stereoscopic imaging mode using the at least one monocular stereoscopic imaging unit, an imaging unit other than the at least one monocular stereoscopic imaging unit and the at least one monocular stereoscopic imaging unit among the plurality of imaging units 2D imaging mode and compound eye stereoscopic imaging mode using
Set one imaging mode from among imaging modes including
Stereo imaging device.
The image generation unit, based on a user instruction input, a two-dimensional imaging mode using the at least one monocular stereoscopic imaging unit, and the at least one of the at least one monocular stereoscopic imaging unit and the plurality of imaging units. In a compound eye stereoscopic imaging mode using an imaging unit other than one monocular stereoscopic imaging unit,
For each pixel position of the image sensor included in the at least one monocular three-dimensional imaging unit, pixel signals of each pixel group constituting the plurality of pixel groups are added, and the result of the addition is referred to as a pixel signal at each pixel position. Perform pixel signal addition processing,
The stereoscopic imaging apparatus according to claim 6.
The stereoscopic imaging device according to any one of claims 1 to 7, wherein the number of the plurality of imaging units is 2, and an imaging unit other than the at least one monocular stereoscopic imaging unit is also a monocular stereoscopic imaging unit.
The stereoscopic imaging device according to any one of claims 1 to 8, further comprising a stereoscopic image display unit that displays the generated stereoscopic image.