WO2013136832A1 - Stereoscopic image display control device and stereoscopic image display control method - Google Patents

Abstract

In the present invention, in the case where a parallax image that cannot be stereoscopically displayed in the current state thereof is selected, the selected image is rotated in an appropriate direction and stereoscopically displayed, and a sense of discomfort is therefore not imparted upon the user. Furthermore, in the present invention, a direction in which it becomes possible for the selected parallax image to be stereoscopically viewed is displayed by means of information such as a diagram. The user perceives this information and is able to understand that the image will be stereoscopically displayed if made to rotate.

Description

Stereoscopic image display control apparatus and stereoscopic image display control method

The present invention relates to a stereoscopic image display control device and a stereoscopic image display control method, and in particular, based on four parallax images obtained by forming, on an image sensor, subject images that have passed through different regions in four directions of a photographing lens. The present invention relates to a technology for stereoscopic display.

Patent Document 1 discloses an imaging apparatus that arranges a single microlens for a plurality of captured images and generates a method of right and left parallax images and upper and lower parallax images.

Further, Patent Document 2 discloses a display device that can simultaneously generate both a left and right parallax image and an upper and lower parallax image.

In Patent Document 3, pixels of a stereoscopic image display device that expresses an image in three dimensions are divided and formed into pairs of odd-numbered pixels and even-numbered pixels, and the light rays are horizontally placed in front of the odd-numbered pixels and even-numbered pixels. And a stereoscopic image display device provided with polarizing means for passing only one of the vertical components.

Other conventional techniques related to the present invention include the following Patent Documents 4 to 8.

JP 2009-165115 A JP 2006-285247 A JP-A-6-258733 JP 2010-109414 A JP 2010-175643 A JP 2005-130313 A JP 2008-141514 A JP 2009-165115 A

A parallax image in the vertical and horizontal directions can be obtained using a Bayer array type imaging device having a configuration of one microlens for four photodiodes as in Patent Document 1.

When a parallax image in the vertical and horizontal directions is acquired, the parallax image can be rotated and displayed so as to obtain an appropriate stereoscopic view according to the parallax direction of the viewer and the orientation of the display device. For example, when a stereoscopic display device having a display parallax (a direction in which binocular parallax occurs on the display surface) in the left-right direction stereoscopically displays a vertical parallax image, if the parallax image is rotated by + 90 ° or −90 °, The pupil division direction matches the display parallax direction, and stereoscopic viewing is possible. Note that three-dimensional display methods include a parallax barrier method, a lenticular method, and a liquid crystal shutter method.

However, when only a vertical parallax image or a horizontal parallax image is acquired, stereoscopic display becomes impossible when the image or display is rotated. For example, when binocular parallax is generated by placing the display of the display device in the horizontal direction (horizontal placement), the parallax image in the left-right direction is stereoscopically displayed horizontally. However, if the display is rotated by 90 ° (vertically placed), the left-right parallax image cannot be recognized as a stereoscopic display. In addition, since the vertical parallax image has not been acquired, the vertical parallax image cannot be stereoscopically displayed on the vertical display.

Suppose that an image set consisting only of left and right parallax images and an image set consisting of vertical parallax images are acquired and mixed on the same recording medium. When the display device stereoscopically displays these images sequentially by frame advance, the parallax image is not recognized as a stereoscopic display unless the parallax direction of the parallax image matches the binocular parallax direction of the display device viewed from the observer. On the other hand, when the parallax direction of the parallax image matches the binocular parallax direction of the display device viewed from the observer, it is recognized as stereoscopic display. If the observer cannot recognize the stereoscopic effect by the frame during frame advance, the observer feels uncomfortable and tires the eyes of the observer.

The present invention has been made in view of such circumstances, and an object thereof is to appropriately display a parallax image input from the outside corresponding to the parallax direction of a display device.

The present invention relates to an input unit capable of inputting a set of images having parallax from the outside, a selection unit for selecting a desired set of images from the set of images input to the input unit, and a set of images having parallax. A determination unit that determines whether or not a specific display parallax direction of the display unit capable of stereoscopic display matches an image parallax direction that is a parallax direction of a selected image set that is a set of images selected by the selection unit; In response to determining that the display parallax direction and the image parallax direction do not match, the determination unit rotates each image of the selected image set by an angle at which the display parallax direction and the image parallax direction match, and There is provided a stereoscopic image display control device including a display control unit that controls display.

When the display parallax direction is the horizontal direction and the image parallax direction is the vertical direction, the display control unit controls to rotate each image of the selected image set by 90 ° and display the image on the display unit in a stereoscopic manner.

The display control unit includes a setting unit that sets whether to rotate the selected image set. When the setting unit is set to rotate the image, the display control unit selects the selected image set by an angle at which the display parallax direction matches the image parallax direction. These images are rotated so as to be displayed in a stereoscopic manner on the display unit.

The display control unit displays information indicating the rotation angle of the image or the rotation direction of the display unit on the display unit so that the display parallax direction and the image parallax direction match when the setting unit is configured not to rotate the image. Control.

The present invention provides a display unit capable of stereoscopically displaying a set of images having parallax in a specific display parallax direction, an input unit capable of inputting a set of images having parallax from the outside, and a set of images input to the input unit A selection unit that selects a desired set of images from the image, and determines whether the display parallax direction and the image parallax direction that is the parallax direction of the selected image set that is a set of images selected by the selection unit match And the rotation angle of the image or the rotation of the display unit so that the display parallax direction and the image parallax direction match in accordance with the determination that the display parallax direction and the image parallax direction do not match. There is provided a stereoscopic image display control device including a display control unit that controls to display information indicating a direction on a display unit.

Information is a figure and / or a character indicating the rotation angle of the image or the rotation direction of the display unit.

The display parallax direction is the horizontal direction, and the input unit can input a set of images having parallax in a single horizontal or vertical direction and a set of images having parallax in a plurality of horizontal and vertical directions It is.

When the selected image set has parallax in the horizontal and vertical directions, the display control unit stereoscopically displays the set of images having parallax in the horizontal direction on the display unit, and stereoscopically displays the set of images having parallax in the vertical direction. Control is performed so that information indicating that the information can be displayed is displayed on the display unit.

When the selected image set has parallax in the horizontal and vertical directions, the display control unit stereoscopically displays the set of images having parallax in the horizontal direction on the display unit, and displays the set of images having parallax in the vertical direction. To display on the display unit a representative image showing a state of rotation so as to match.

When the selected image set has parallax in the horizontal and vertical directions, the display control unit converts the set of images having parallax in the vertical direction by rotating the set of rotated images by an angle that matches the display parallax direction and the parallax in the horizontal direction. Control is performed to stereoscopically display the set of images on the display unit sequentially or simultaneously.

The light beam from the subject is divided into four light beams by pupil division in the vertical and horizontal directions, and the four light beams are imaged on four photoelectric conversion element groups corresponding to the color filters of a specific arrangement, respectively, and subjected to photoelectric conversion. An imaging unit that outputs a set of four viewpoint image signals having color components corresponding to each color filter, and a set of four viewpoint image signals having color components corresponding to each color filter output by the imaging unit By combining the two viewpoint image signals obtained from the photoelectric conversion elements adjacent in the vertical direction or the two viewpoint image signals obtained from the photoelectric conversion elements adjacent in the horizontal direction, there is a parallax in the horizontal direction. A composition unit that creates a set of images or a set of images having parallax in the vertical direction, and the input unit is configured to create a set of images having parallax in the horizontal direction created by the synthesis unit or in the vertical direction. Inputting a set of images having the difference.

The combining unit includes two viewpoint image signals obtained from photoelectric conversion elements adjacent in the vertical direction and a horizontal direction among a set of four viewpoint image signals having color components corresponding to the color filters output from the imaging unit. By combining the two viewpoint image signals obtained from the photoelectric conversion elements adjacent to each other, a set of images having parallax in the horizontal direction and a set of images having parallax in the vertical direction are created. A set of images having parallax in the horizontal direction and a set of images having parallax in the vertical direction are input.

When the selected image set has parallax in the horizontal and vertical directions, the display control unit selects a set of images having parallax that matches the display parallax direction at the time of output of the imaging unit, and stereoscopically displays the selected set of images Control to do.

A posture information detection unit that detects posture information at the time of outputting a set of four viewpoint image signals of the imaging unit, and the combining unit is a photoelectric conversion element adjacent in the vertical direction based on the posture information detected by the posture information detection unit The left viewpoint image signal set and the right viewpoint image signal set obtained from the above are discriminated, the discriminated left viewpoint image signal set is synthesized, and the discriminated right viewpoint image signal set is synthesized Thus, a set of images having parallax in the horizontal direction is created.

A display parallax direction acquisition unit that acquires a specific display parallax direction of the display unit is provided, and the determination unit is a parallax of a selected image set that is a set of the display parallax direction acquired by the display parallax direction acquisition unit and the image selected by the selection unit It is determined whether or not the image parallax direction, which is the same direction, matches.

According to the present invention, a stereoscopic image display control device inputs a set of images having parallax from the outside, a step of selecting a desired set of images from the set of input images, Determining whether or not a specific display parallax direction of the display unit capable of stereoscopically displaying the set matches an image parallax direction that is a parallax direction of a selected image set that is a set of selected images, and a display parallax direction And a step of controlling each image of the selected image set to be stereoscopically displayed by rotating an angle at which the display parallax direction and the image parallax direction match in response to determining that the image parallax direction and the image parallax direction do not match. A stereoscopic image display control method to be executed is provided.

According to the present invention, a stereoscopic image display control device inputs a set of images having parallax from the outside, a step of selecting a desired set of images from the set of input images, Determining whether or not a specific display parallax direction of a display unit capable of stereoscopically displaying the set matches an image parallax direction that is a parallax direction of a selected image set that is a set of selected images; The rotation angle of the image such that the display parallax direction matches the image parallax direction in response to determining that the specific display parallax direction and the image parallax direction of the display unit capable of stereoscopically displaying the set of images are inconsistent Alternatively, a method of controlling to display information indicating the rotation direction of the display unit is provided.

In the present invention, when a parallax image that cannot be stereoscopically displayed as it is is selected, the selected image is rotated in an appropriate direction and stereoscopically displayed, so that the user does not feel uncomfortable. In the present invention, the direction in which the selected parallax image can be viewed stereoscopically is displayed as information such as a figure. If the user recognizes this information and rotates the image, the user can understand that the image is displayed three-dimensionally.

Block diagram of monocular stereoscopic imaging device The figure which shows the structural example of the light-receiving surface of CCD Compositing process flowchart The figure which shows the parallax image synthesize | combined corresponding to the arrangement | positioning direction of a monocular three-dimensional imaging device. The figure which shows the example of the frame of a parallax image Flow chart of rotation process Diagram showing an example of rotation of a composite image Notification process flowchart The figure which shows the example of a display of the information explaining the rotation direction in which a three-dimensional display is attained Flow chart of rotation processing that can be selected on / off Flowchart of notification processing according to the fourth embodiment The figure which shows the example of a display of the information which shows direction of the display apparatus in which a stereoscopic vision is possible The figure which shows the example of a display of the information which shows the direction which can be displayed three-dimensionally The figure which shows the example of a display of the information which shows that a plane display is possible Flowchart of notification processing according to the fifth embodiment The figure which shows the example of a display of the icon which shows that the stereo image of an up-down direction can be displayed The figure which shows the example of a display of the thumbnail of the image which has the parallax of an up-down direction The figure which shows the example which displays sequentially each of the stereoscopic image which rotated the image set Image1-1 which has parallax in the left-right direction, and the image set Image1-2 which has binocular parallax in the vertical direction by 90 degrees The figure which shows the example which each displays the image set Image1-1 which has binocular parallax in a horizontal direction, and each of the three-dimensional image Image1-2 which rotated the set of images which have binocular parallax in a vertical direction 90 degree | times. Block diagram of a stereoscopic display device of a seventh embodiment Block diagram of the stereoscopic display device of the eighth embodiment Flowchart of composition processing according to the eighth embodiment

<First Embodiment>
FIG. 1 is a block diagram showing an embodiment of a monocular stereoscopic imaging device 1.

This monocular stereoscopic imaging device 1 records a captured image on a recording medium 57, and the operation of the entire device is centrally controlled by a central processing unit (CPU) 40. Further, the operation power supply of each block of the apparatus is supplied from the power supply unit 60.

The monocular three-dimensional imaging device 1 is provided with operation units 38 such as a shutter operation unit, a mode operation unit, a reproduction operation unit, a MENU / OK operation unit, a cross operation unit, and a BACK operation unit. A signal from the operation unit 38 is input to the CPU 40, and the CPU 40 controls each circuit of the monocular stereoscopic imaging device 1 based on the input signal. For example, lens drive control, aperture drive control, shooting operation control (shutter operation unit half) Control of processing from pressing to image recording completion), image processing control, image data recording / playback control, display control of the display device 30 for stereoscopic display, power on / off, and the like.
The shutter operation unit is a means for inputting an instruction to start imaging, and can be constituted by a two-stage stroke type switch composed of so-called “half press” and “full press”. In the monocular stereoscopic imaging device 1, when the shutter operation unit is half-pressed, an S1 ON signal is output, and when the shutter button is further pressed halfway down, an S2 ON signal is output and an S1 ON signal is output. Shooting preparation processing such as automatic focus adjustment (AF processing) and automatic exposure control (AE processing) is executed, and when the S2 ON signal is output, shooting processing is executed.

The shutter operation unit is not limited to a two-stroke type switch composed of half-pressing and full-pressing, and may output an S1 on signal and an S2 on signal in one operation, A switch may be provided to output an S1 on signal and an S2 on signal.

Further, in the form in which the operation instruction is performed using the touch panel or the like, the operation instruction may be output by touching an area corresponding to the operation instruction displayed on the screen of the touch panel as the operation unit. In the invention, the form of the operation means is not limited to this as long as it instructs the shooting preparation process and the shooting process. Further, the shooting preparation process and the shooting process may be executed continuously by an operation instruction to one operation means.

The mode operation unit is a means for selecting one of an auto shooting mode for shooting a still image, a manual shooting mode, a scene position such as a person, a landscape, a night view, and a moving image mode for shooting a moving image.

The reproduction operation unit is means for switching to a reproduction mode in which a still image or a moving image of a stereoscopic image (3D image) or a planar image (2D image) that has been recorded is displayed on the display device 30. The MENU / OK operation unit is a means having both a function for instructing to display one or a plurality of menus on the screen of the display device 30 and a function for instructing confirmation and execution of selection contents. The cross operation unit is an operation unit that inputs instructions in four directions, up, down, left, and right, and functions as a cursor movement operation unit that selects an item from a menu screen or instructs selection of various setting items from each menu. The up / down operation unit of the cross operation unit functions as a zoom operation unit during shooting or a playback zoom operation unit during playback mode, and the left / right operation unit performs frame advance (forward / reverse feed) during playback mode. ) Functions as an operation unit. The BACK operation unit is used to delete a desired object such as a selection item, cancel an instruction content, or return to the previous operation state.

In the shooting mode, the image light indicating the subject corresponds to four types of viewpoints, top, bottom, left, and right, of a parallax image obtained by pupil division in the vertical and horizontal directions through the imaging lens 12, the microlens ML, and a diaphragm (not shown). An image is formed on the light receiving surface of a solid-state imaging device (hereinafter referred to as “CCD”) 16 which is a phase difference image sensor that can be acquired by the pixels 16a to 16d. The imaging lens 12 is driven by a motor driver 36 controlled by the CPU 40, and performs focus control, zoom (focal length) control, and the like. The diaphragm is composed of, for example, five diaphragm blades and is driven by a diaphragm driving unit (not shown) controlled by the CPU 40. For example, a diaphragm value (F value) F2.8 to F11 is controlled in five steps in increments of 1AV. Is done.

Further, the CPU 40 performs charge accumulation time (shutter speed) in the CCD 16 and reading control of an image signal from the CCD 16 through the timing generator 37.

The signal charge accumulated in the CCD 16 is read out as a voltage signal corresponding to the signal charge based on the readout signal applied from the timing generator 37. From the voltage signals read from the pixels 16a to 16d of the CCD 16, R, G, and B signals for each pixel are sampled and held, amplified, and then applied to the A / D converters 20a to 20d, respectively. The A / D converter 20 converts R, G, and B signals that are sequentially input into digital R, G, and B signals and outputs them to the image input controller 22.

The image signal processing unit 24 performs predetermined processing such as offset control, gain control processing including white balance correction and sensitivity correction, gamma correction processing, YC processing, etc., on the digital image signal input via the image input controller 22. Perform signal processing.

The top / bottom / left / right viewpoint image data (3D image data) processed by the image signal processing unit 24 is input to the RAM 50. The RAM 50 includes an A area and a B area each storing 3D image data representing a 3D image for one frame. In the RAM 50, 3D image data representing a 3D image for one frame is rewritten alternately in the A area and the B area. The written 3D image data is read from an area other than the area where the 3D image data is rewritten in the A area and the B area of the RAM 50. The 3D image data read from the RAM 50 is converted into an image signal for stereoscopic display by the 3D image signal processing unit 34, encoded by the video encoder 28, and provided for the stereoscopic display provided on the back of the camera. This is output to the display device 30, whereby a 3D subject image is displayed on the display screen of the display device 30.

The display device 30 can display a stereoscopic image (left viewpoint image and right viewpoint image) as a directional image having directivity in a specific display parallax direction, here the horizontal direction, by a parallax barrier. It is. However, the stereoscopic image display method adopted by the display device 30 is not limited to this, and each viewpoint image can be viewed individually by using a lenticular lens or wearing special glasses such as polarized glasses or liquid crystal shutter glasses. You can do it.

When the first stage pressing (half-pressing) of the shutter operation unit of the operation unit 38 is performed, the CPU 40 starts the AF operation and the AE operation, and the focus lens in the imaging lens 12 is brought into the in-focus position via the motor driver 36. Control to come to. The image data output from the A / D converter 20 when the shutter operation unit is half-pressed is taken into the AE / AWB detection circuit 44.

The AE / AWB detection circuit 44 integrates the G signals of the entire screen or integrates the G signals that are weighted differently in the central portion and the peripheral portion of the screen, and outputs the integrated value to the CPU 40. The CPU 40 calculates the brightness of the subject (shooting Ev value) from the integrated value input from the AE / AWB detection circuit 44, and the aperture value of the aperture for obtaining an appropriate exposure based on this shooting Ev value and the electronic shutter of the CCD 16 ( (Shutter speed) is determined according to the program diagram, the aperture is controlled via the aperture drive unit based on the determined aperture value, and the charge is accumulated in the CCD 16 via the timing generator 37 based on the determined shutter speed. Control the time. Note that the brightness of the subject may be calculated based on an external photometric sensor.

The AF processing unit 42 is a part that performs contrast AF processing or phase difference AF processing. When performing the contrast AF process, the high-frequency component of the image data in the focus area is extracted from at least one of the left-viewpoint image data and the right-viewpoint image data, and the high-frequency component is integrated to focus. An AF evaluation value indicating the state is calculated. The AF control is performed by controlling the focus lens in the imaging lens 12 so that the AF evaluation value is maximized. Further, when performing phase difference AF processing, the phase difference between the image data corresponding to the main pixel and the sub pixel in the focus area in the left viewpoint image data and the right viewpoint image data is detected, and this phase difference is indicated. Based on the information, a defocus amount (a shift amount in a state where the in-focus position is shifted with respect to the target subject) is obtained. AF control is performed by controlling the focus lens in the imaging lens 12 so that the defocus amount becomes zero.

When the AE operation and the AF operation are completed and the shutter operation unit is depressed in the second stage (full depression), the vertical and horizontal viewpoint images corresponding to the pixels output from the A / D converter 20 in response to the depression. Is input from the image input controller 22 to the RAM 50 and temporarily stored.

The image data temporarily stored in the RAM 50 is appropriately read out by the image signal processing unit 24, where signal processing including luminance data and color difference data generation processing (YC processing) is performed. The YC processed image data (YC data) is stored in the RAM 50 again. Subsequently, the YC data is respectively output to the image signal processing unit 24 and subjected to compression processing such as JPEG (joint photographic photographic experts group), and then stored in the memory 48 again.

A multi-picture file (MP file: a file in which a plurality of images are connected) is generated from YC data (compressed data) of the upper, lower, left, and right viewpoint images stored in the RAM 50, and the MP file is a media recording control unit. 52 and is recorded on the recording medium 57.

The speaker 53 emits sound according to the control of the sound input / output processing unit 54. The contents of the voice are stored in the ROM 55. The microphone 56 acquires sound in conjunction with image recording and converts it into an analog sound signal. The analog audio signal is converted into compressed digital audio data via the audio input / output processing unit 54 and recorded in the RAM 50 or the recording medium 57.

The parallax correction unit 63 deconvolves the small area with a restoration filter corresponding to the small area covering each viewpoint image, and restores the corresponding small area of the viewpoint image. Specifically, the parallax correction unit 63 obtains a defocus amount corresponding to each of the small areas based on the phase difference calculated for each of the small areas. A set of defocus amounts corresponding to each of the small areas obtained over the entire effective pixel area is referred to as a defocus map. The parallax correction unit 63 temporarily stores the obtained defocus map in the RAM 50 or the like. The parallax correction unit 63 may detect feature points and corresponding points between the viewpoint images, and create a defocus map based on a difference in position information between the feature points and the corresponding points.

The ROM 55 restores the image height (distance from the image center, typically the distance from the optical axis center L of the imaging lens 12) and the defocus amount (or subject distance) of each small area in each viewpoint image. Saving the filter. The parallax correction unit 63 deconvolves the small area with the restoration filter selected for each small area of each viewpoint image, and restores the corresponding small area of the viewpoint image. Thereby, parallax according to the defocus amount (blur amount) can be given to the image.

When the stereoscopic image display mode is selected from the operation unit 38, the 3D image signal processing unit 34 performs stereoscopic image data for the display device 30 to stereoscopically display the right-eye image and the left-eye image stored in the RAM 50. To synthesize. For example, when the display device 30 adopts the parallax barrier method, the 3D image signal processing unit 34 generates stereoscopic image data in which the image for the right eye and the image for the left eye are divided into strips and the strip images are alternately arranged. And output to the video encoder 28. The image selection unit 71 selects a viewpoint image to be combined with the right-eye image and the left-eye image from the four viewpoint images. The image rotation processing unit 72 rotates the image selected by the image selection unit 71 by an angle at which the image is stereoscopically displayed. The line-of-sight display control unit 73 displays information indicating the orientation of the display device on the display device 30 such that the image selected by the image selection unit 71 is stereoscopically displayed.

2 (a) is a diagram showing a configuration example of the light receiving surface of the CCD 16. FIG. As shown in part (a) of FIG. 2, on the light receiving surface of the CCD 16, a red, blue or green color filter, a pixel group of four viewpoints of pixels A, B, C and D corresponding to the color filter, and Microlenses ML for the four pixel groups are arranged. The light receiving units including the color filter, the four pixel group, and the microlens are arranged in a Bayer shape.

The Bayer array has three primary colors of red (R, r), green (G, g), and blue (B, b) on the four pixel groups formed in a square lattice pattern on the semiconductor substrate surface of the CCD 16. When the color filters are arranged, a row in which the red and green filters are alternately arranged and a row in which the green and blue filters are alternately arranged are alternately provided in the column direction.

In FIG. 2, the XY plane is the light receiving surface of the CCD 16, X is the row direction, and Y direction is the column direction. From the viewpoint of the photographer, the pixels A and C are located on the left side, and the pixels B and D are located on the right side. Z is the optical axis direction of the lens 12, and the direction toward the subject (in this figure, the direction from the front to the back of the page) is defined as the positive direction. Hereinafter, when X is in the horizontal direction and Y is in the vertical direction, it is assumed that the monocular three-dimensional imaging device 1 is “horizontal”. When X is in the vertical direction and Y is in the horizontal direction, the monocular stereoscopic imaging device 1 is assumed to be “vertically placed”.

As shown in FIG. 2A, when the monocular stereoscopic imaging device 1 is placed horizontally and the positive direction of Y (the direction from the column C toward the column A) is vertically upward, the monocular stereoscopic imaging device 1 is assumed to be “ordinary horizontal placement” or “0 ° horizontal placement”.

In addition, although the monocular stereoscopic imaging device 1 is placed horizontally, if the positive direction of Y is directed vertically downward, it is assumed that the monocular stereoscopic imaging device 1 is “laterally placed at + 180 °”.

In addition, as shown in FIG. 2B, the monocular stereoscopic imaging device 1 is placed vertically, but the positive direction of X (the direction from the pixel A to the pixel B) is vertically upward, that is, a monocular. When the stereoscopic imaging device 1 is rotated 90 ° counterclockwise as viewed from the photographer, it is assumed that the monocular stereoscopic imaging device 1 is “+ 90 ° portrait”.

In addition, as shown in FIG. 2C, the monocular stereoscopic imaging device 1 is placed vertically, but when the positive direction of X is directed vertically downward, that is, the monocular stereoscopic imaging device 1 is viewed from the photographer. It is assumed that the monocular three-dimensional imaging device 1 is “vertically placed at −90 °” when rotated 90 ° clockwise.

Attitude information including the rotation direction and rotation angle of the CCD 16 around the Z axis is detected by an attitude sensor 70 such as a gyroscope. The CPU 40 controls the posture sensor 70 to detect posture information at the timing of starting imaging when the shutter operation unit is half-pressed or fully pressed. As a result of this control, the posture information obtained from the posture sensor 70 is recorded on the recording medium 57 in association with the vertical, horizontal, and parallax images. For example, this posture information is recorded as tag information (such as an Exif rotation tag) of an image file that stores vertical, horizontal, and parallax images. By referring to this posture information, the monocular stereoscopic imaging device 1 at the time of capturing the four-viewpoint image can perform “normal horizontal placement”, “+ 180 ° horizontal placement”, “−90 ° vertical placement”, “+ 90 °”. It is possible to determine which state is “vertically placed”.

The subject light incident on the pixel group is divided in the horizontal direction (left-right direction) and the vertical direction (up-down direction) by pupil dividing means such as a light shielding member and a mirror (not shown). Accordingly, a subject image having parallax in the vertical (up and down) direction and the horizontal (left and right) direction is formed on each pixel constituting the four pixel group.

FIG. 3 is a flowchart of the synthesis process executed by the monocular stereoscopic imaging device 1. A program for causing the monocular stereoscopic imaging device 1 to execute the following processing is recorded in a computer-readable recording medium such as the ROM 55. This process starts in response to the selection of the stereoscopic display mode from the operation unit 38. This process can also be executed by an information processing apparatus other than the monocular stereoscopic imaging apparatus 1, such as a stereoscopic display apparatus 10 or a personal computer described later.

In S <b> 1, the image selection unit 71 reads a set of four parallax images and posture information at the time of acquisition of the parallax images from the RAM 50 and the recording medium 57. In the recording medium 57, a set of four parallax images is recorded as an MP file. The image selection unit 71 determines whether or not the parallax image is captured “0 ° horizontally” from the read posture information. If Yes, the process proceeds to S2. If No, the process proceeds to S3.

In S <b> 2, the image selection unit 71 selects the image signal from the pixel A and the image signal from the pixel C as a synthesis source image of the synthesized image L, and synthesizes the image signal from the pixel B and the image signal from the pixel D. Select the image from which the image R is to be synthesized. The 3D image signal processing unit 34 combines the image signal from the pixel A and the image signal from the pixel C, the image signal from the pixel B, and the image signal from the pixel D according to the selection of the image selection unit 71. A composite image R is created.

In S3, the 3D image signal processing unit 34 determines whether the parallax image was shot with “+ 90 ° portrait” or the parallax image was taken with “−90 ° portrait” from the read posture information. . If Yes (the parallax image was captured with “+ 90 ° portrait”), the process proceeds to S4. If No (the parallax image was captured with “−90 ° portrait”), the process proceeds to S5.

In S4, the image selection unit 71 selects the image signal from the pixel A and the image signal from the pixel B as a synthesis source image of the synthesized image L, and synthesizes the image signal from the pixel C and the image signal from the pixel D. Select the image from which the image R is to be synthesized. The 3D image signal processing unit 34 creates a composite image L of the pixels A and B and a composite image R of the pixels C and D according to the selection of the image selection unit 71.

In S <b> 5, the image selection unit 71 selects the image signal from the pixel C and the image signal from the pixel D as the source image of the composite image L, and combines the image signal from the pixel A and the image signal from the pixel B. Select the image from which the image R is to be synthesized. The 3D image signal processing unit 34 creates a composite image L of the pixels C and D and a composite image R of the pixels A and B according to the selection by the image selection unit 71.

In S6, the 3D image signal processing unit 34 controls the video encoder 28 to display the composite image L on the display device 30 as the left-eye display image and the composite image R as the right-eye display image.

FIG. 4 shows a parallax image synthesized corresponding to the arrangement direction of the monocular stereoscopic imaging device 1. When the monocular three-dimensional imaging device 1 is in “ordinary horizontal position” as viewed from the observer as in the part (a) of FIG. 4, the orientation of the imaging surface is as in the part (a) of FIG. 2. In this case, a left-right parallax exists between the composite image L of the pixels A and C generated in S2 and the composite image R of the pixels B and D. Therefore, the composite image L is an image for left eye display, and the composite image R is an image for right eye display.

When the monocular three-dimensional imaging device 1 becomes “+ 90 ° vertically placed” (portion (b) in FIG. 2) rotated + 90 ° counterclockwise from “normal landscape”, the pixel A generated in S4 A parallax in the left-right direction exists between the combined image L of B and B and the combined image R of the pixels C and D. Therefore, the composite image L is an image for left eye display, and the composite image R is an image for right eye display.

When the monocular 3D imaging device 1 becomes “-90 ° vertical” (part (c) of FIG. 2) rotated by −90 ° counterclockwise from “normal horizontal placement”, it is generated in S5. There is a left-right parallax between the synthesized image L of the pixels C and D and the synthesized image R of the pixels A and B. For this reason, the synthesized image L is an image for left-eye display, and the synthesized image R is an image for right eye display.

Although not shown, when the XY plane of FIG. 2 is rotated by + 180 ° about the Z axis, and the monocular stereoscopic imaging device 1 is rotated by + 180 ° from “horizontal placement” to “+ 180 ° landscape orientation”, A parallax in the left-right direction exists between the composite image L of the pixels B and D and the composite image R of the pixels A and C. Therefore, the composite image L is an image for left eye display, and the composite image R is an image for right eye display. However, since it is not normally assumed that an image is captured in “+ 180 ° landscape”, the description is omitted.

Thus, the composite images L and R can be generated in accordance with the orientation of the monocular stereoscopic imaging device 1 during imaging, and the composite images L and R can be displayed as stereoscopic images.

In the above composition process, composite images L and R for left eye display and right eye display were created and displayed. The monocular three-dimensional imaging device 1 can also record the combination of the composite images L and R on the recording medium 57 instead of displaying the composite images L and R. In this way, in the monocular three-dimensional imaging apparatus 1 capable of acquiring the vertical and horizontal parallax images, if only the vertical parallax image or the horizontal parallax image is combined and recorded, all the vertical and horizontal parallax images are recorded. Data amount is halved compared to recording.

However, in this case, the recording medium 57 may include a frame composed of the composite images L and R (two viewpoint images) and a frame composed of the upper, lower, left, and right parallax images that are not combined (four viewpoint images). Further, since the orientation of the monocular stereoscopic imaging device 1 at the time of imaging does not necessarily match the orientation of the display device 30 at the time of stereoscopic display, even if a frame composed of the composite images L and R is displayed as it is, stereoscopic display may not be obtained. .

For example, as shown in FIG. 5A, frames Image1 to Image6 are recorded on the recording medium 57 in accordance with the shooting date and time. Here, as shown in FIG. 5B, frames Image1 to Image3 are a set of four images having parallax in the horizontal and vertical directions. Further, as shown in part (c) of FIG. 5, the frames Image 4 and 5 are a set of two images having parallax in the horizontal direction. Also, as shown in FIG. 5D, Image 6 is a set of images having parallax in the vertical direction. Image4 · 5 is generated in S4 of the above-described combining process, and Image6 is generated in S5 of the above-described combining process.

Image1 to Image5 are recognized as stereoscopic images if a set of images having a parallax in the horizontal direction is placed on the display device 30 placed horizontally at 0 ° as it is. This is the same as S2 in the above synthesis process.

However, for Image6, the orientation of the monocular stereoscopic imaging device 1 at the time of imaging is vertical at −90 ° and does not coincide with the horizontal orientation of 0 ° at the time of viewing the image. For this reason, when the image 6 is displayed as it is with the monocular stereoscopic imaging device 1 placed horizontally at 0 °, an image rotated by −90 ° is displayed. In addition, since the parallax direction of Image6 is the vertical direction, Image6 is not stereoscopically displayed.

When the user browses the images Image1 to Image6 continuously, the images 1 to 5 are displayed in 3D, but the image 6 is not displayed in 3D. This makes the user feel uncomfortable and causes eye fatigue.

Therefore, the next rotation processing is performed for the frames of the two viewpoint images that cannot be browsed as a three-dimensional image as it is.

FIG. 6 shows a flowchart of the rotation process. This process starts in response to the selection of the stereoscopic display mode from the operation unit 38. It is assumed that the monocular three-dimensional imaging device 1 is horizontally placed at 0 ° when this process is executed. The same processing steps as those already described are given the same reference numerals.

In S11, the 3D image signal processing unit 34 reads the selected parallax image frame from the recording medium 57 in accordance with the frame selection operation to the operation unit 38. The 3D image signal processing unit 34 determines whether the read frame is a four-viewpoint image having parallax in the vertical and horizontal directions, or a two-viewpoint image having parallax in the vertical and horizontal directions. In the case of Yes (four viewpoint images), the process proceeds to the composition process. If No (two viewpoint images), the process proceeds to S13. Note that the 3D image signal processing unit 34 performs trimming from the top, bottom, left, and right viewpoint image data so that a region arbitrarily designated by the operation unit 38 or the like has a parallax in any one direction, and the trimmed two viewpoint images May be read out as a frame of a parallax image, and the process may proceed to S13.

In S <b> 13, the 3D image signal processing unit 34 determines whether the read frame is a two-viewpoint image having a parallax in the same horizontal direction as the display parallax direction of the display device 30 or a vertical parallax different from the display parallax direction of the display device 30. To determine whether the image is a two-viewpoint image. In the case of Yes (parallax in the left-right direction), the process proceeds to S14. In the case of No (vertical parallax), the process proceeds to S15.

In S14, the 3D image signal processing unit 34 synthesizes stereoscopic image data for performing stereoscopic display from the read parallax images of the frames. The display device 30 performs stereoscopic display based on the stereoscopic image data.

In S15, the image rotation processing unit 72 calculates the angle difference from the posture information at the time of obtaining the parallax image to the current posture information, and rotates the parallax image of the read frame in a direction to eliminate the angle difference. The 3D image signal processing unit 34 synthesizes stereoscopic image data for performing stereoscopic display from the parallax image after rotation. The display device 30 performs stereoscopic display based on the stereoscopic image data. When the trimmed two-viewpoint image is read out, the image rotation processing unit 72 reads out the posture information at the time of acquiring the corresponding parallax image before trimming as the posture information at the time of acquiring the two-viewpoint image. In this way, the rotation process can be similarly performed on the trimmed viewpoint image.

Through the above processing, a two-viewpoint image that cannot be stereoscopically displayed as it is can be stereoscopically displayed by being rotated in an appropriate direction, and the user does not feel uncomfortable.

For example, when Image 6 that is a frame imaged by the monocular stereoscopic imaging device 1 in “−90 ° portrait” is displayed in “normal landscape”, the parallax direction of the parallax image does not match the display parallax direction. . Therefore, as shown in FIG. 7, by rotating the composite image L • R of Image6 by + 90 °, the parallax direction of the composite image L • R of Image6 matches the display parallax direction of the monocular stereoscopic imaging device 1 at the time of viewing. Image6 can also be viewed as a stereoscopic image.

<Second Embodiment>
In the rotation processing of the first embodiment, the two-viewpoint image is automatically rotated in the direction in which stereoscopic display is possible. However, instead of the rotation processing, processing that indicates information indicating the direction in which stereoscopic display is possible to the user is executed. May be.

FIG. 8 shows a flowchart of notification processing according to the second embodiment. This process starts in response to the selection of the stereoscopic display mode from the operation unit 38. Hereinafter, the same processing steps as those described above are denoted by the same reference numerals.

S11 to S13 are the same as the rotation process. However, when it determines with No in S13, it progresses to S21.

In S21, the 3D image signal processing unit 34 notifies that the parallax direction of the parallax image is different from the display parallax direction. For example, it is assumed that a parallax image Image6 made up of the combined images R and L as shown in part (a) of FIG. 9 is displayed. The composite images R and L are created in S4 of the composite process. In this case, as shown in part (b) of FIG. 9, the posture information at the time of photographing (the top and bottom direction at the time of photographing of the image is represented by a graphic, and the text explaining the rotation direction in which stereoscopic display is possible is also displayed. If the user recognizes this notification and rotates the image, the user can understand that the image is displayed three-dimensionally.After this notification processing, the rotation processing may be executed in accordance with an instruction from the user.

<Third Embodiment>
In the first and second embodiments, the rotation process or the notification process may be executed in response to an instruction to turn on or off the function of rotating the viewpoint image. FIG. 10 shows the flowchart.

In S31, it is determined whether or not the operation unit 38 has instructed to turn on or off the function of automatically rotating the viewpoint image. When on is instructed, the process proceeds to the rotation process of the first embodiment, and when off is instructed, the process proceeds to the notification process of the second embodiment.

Through the above processing, notification that the image is automatically rotated or the parallax direction of the parallax image is different from the display parallax direction is performed in accordance with an instruction from the user.

<Fourth embodiment>
When the display device 30 can stereoscopically display parallax not only in the horizontal direction but also in the vertical direction (for example, Patent Document 3), in the notification processing of the second and third embodiments, instead of indicating the rotation direction of the image, the stereoscopic view is displayed. You may notify the information which shows the rotation direction of the display apparatus 30 which becomes possible.

FIG. 11 is a flowchart of notification processing according to the fourth embodiment.

S11 to S13 are the same as above. However, if No in S13, the process proceeds to S61.

In S61, the line-of-sight display control unit 73 displays information indicating the rotation direction of the display device 30 that enables stereoscopic viewing.

Suppose, for example, that Image1 to Image6 as shown in part (a) of Fig. 5 are selected and displayed by frame advance. Image6 is created in S4 of the synthesis process and has vertical parallax.

In this case, since Image 6 has parallax in the vertical direction, if the display device 30 is placed 90 ° vertically rotated by 90 ° from the normal landscape, the parallax direction of Image 6 matches the binocular parallax direction on the left and right. Is recognized as a stereoscopic image. Therefore, as shown in FIG. 12, information I such as graphics and characters indicating the orientation of the display device 30 capable of stereoscopic viewing is displayed. This process is performed instead of S22 of the notification process.

When the user rotates the display device 30 by + 90 ° according to this information I, the parallax direction of Image6 and the display parallax direction of the display device 30 coincide with each other, so that the user can visually recognize the stereoscopic image. The image may be automatically rotated as in the above rotation process, but if the user himself / herself rotates the display device 30 in an appropriate direction and stereoscopic display is established, the user is shown the appropriate rotation direction, The actual rotation may be left to the user.

Note that the 3D image signal processing unit 34 may indicate information such as a figure and characters in a direction in which stereoscopic display is possible when a set of images having parallax in the vertical and horizontal directions is selected (FIG. 13). For a set of planar images that cannot be stereoscopically displayed, information such as graphics and characters indicating that planar display is possible may be displayed (FIG. 14).

<Fifth Embodiment>
In the first to fourth embodiments, information indicating that a stereoscopic image in the vertical direction can be displayed may be indicated when a set of images having parallax in both the vertical and horizontal directions is selected.

FIG. 15 is a flowchart of the notification process according to the fifth embodiment.

S11 to S15 are the same as above, but if S11 is Yes, the process proceeds to the synthesis process, and then proceeds to S41.

In S41, the line-of-sight display control unit 73 displays an icon M indicating that a vertical stereoscopic image can be displayed on the selected frame. FIG. 16 shows an example.

When the parallax images in the vertical and horizontal directions are acquired, both the image set consisting only of the parallax images in the horizontal direction and the image set consisting of the parallax images in the vertical direction can be stereoscopically displayed, but only one image set is stereoscopically displayed. The viewer may not be aware of the presence of the other image set. However, the user can know the presence of a parallax image that can display a stereoscopic image by rotating by visually recognizing the icon M.

In the above processing, instead of the icon M as shown in FIG. 16, a representative image (planar image, thumbnail image, etc.) of an image having vertical parallax that can be stereoscopically displayed may be displayed.

For example, as shown in FIG. 17, a thumbnail J1 of an image having parallax in the vertical direction of Image1 is displayed together with Image1. This thumbnail is created by reducing Image1-U or Image1-D having the vertical parallax in part (b) of FIG.

<Sixth Embodiment>
In the fifth embodiment, the representative image of the image having the parallax in the vertical direction of the selected frame is displayed. Instead, all the parallax images in each direction that can be stereoscopically displayed may be displayed.

For example, it is assumed that a frame Image1 having a parallax image in the vertical and horizontal directions is selected from Image1 to Image6 as shown in part (a) of FIG. As shown in FIG. 5B, Image1 includes Image1-1 and Image1-2. In addition, Image1-1 and Image1-2 are a set of two parallax images each having a parallax in the horizontal direction and a parallax in the vertical direction. In this case, as shown in FIG. 18, stereoscopic images obtained by rotating a set of images Image1-1 having parallax in the left-right direction and a set of images Image1-2 having binocular parallax in the vertical direction by 90 ° are sequentially displayed.

This allows the user to sequentially view stereoscopic images in different parallax directions, and to compare and examine their favorite stereoscopic images.

Alternatively, all images in each direction that can be stereoscopically displayed may be displayed on one screen.

For example, as shown in FIG. 19, each of an image set Image1-1 having binocular parallax in the horizontal direction and a stereoscopic image Image1-2 obtained by rotating a set of images having binocular parallax in the vertical direction by 90 ° on one screen indicate.

In the first frame display, both stereoscopic images with different parallax directions may be displayed, and in the second frame display, only the one selected from both stereoscopic images by the operation unit 38 may be displayed. .

This allows the user to view stereoscopic images in different parallax directions at the same time, and to compare and examine their favorite stereoscopic images.

It should be noted that the same 3D display as Image 1 is performed even when a frame Image 2 or Image 3 having parallax in the vertical and horizontal directions is selected. When a frame Image4 / Image5 having a parallax image in the horizontal direction is selected, a set of images having binocular parallax in the horizontal direction is displayed as it is. For Image6, the rotation processing and notification processing of the above embodiment are performed.

<Seventh embodiment>
The various processes (compositing process, rotation process, notification process) described in the first to sixth embodiments can be executed by a display device having blocks related to display equivalent to the monocular stereoscopic imaging device 1.

FIG. 20 is a block diagram of the stereoscopic display device 10 according to the seventh embodiment. Blocks relating to display of a stereoscopic image equivalent to the monocular stereoscopic imaging device 1 are denoted by the same reference numerals as those of the monocular stereoscopic imaging device 1. The recording medium 57 records the parallax image captured by the monocular stereoscopic imaging device 1. The operation unit 38 mainly accepts operations related to image display and does not need to accept operations related to imaging.

The above processing can be repeated every time a set of parallax images (frames) capable of displaying a composite image is selected from the recording medium 57.

<Eighth Embodiment>
In the seventh embodiment, it is not necessary for the stereoscopic display device 10 itself to record a parallax image. For the parallax image input from the outside, the various processes (compositing process, rotation process, notification process) described in the first to sixth embodiments are performed. ) May be executed.

FIG. 21 is a block diagram of the stereoscopic display device 100 according to the eighth embodiment. Blocks relating to display of a stereoscopic image equivalent to the monocular stereoscopic imaging device 1 are denoted by the same reference numerals as those of the monocular stereoscopic imaging device 1.

The stereoscopic display device 100 is connected to an external device 80 storing images such as a personal computer, a game machine, a mobile phone, a smartphone, an HDD recorder, and a Blu-ray recorder from a wired communication path such as an HDMI (High-Definition Multimedia Interface) cable, Bluetooth (registered) A recording medium 57 that is a detachable storage medium such as a USB memory or a memory card or a built-in storage medium is mounted via an external input unit 75 including a wireless communication path such as a trademark) or a wireless LAN station. An input of a set of parallax images is received from the media recording control unit 52.

FIG. 22 is a flowchart of the composition process executed by the stereoscopic display device 100. A program for causing the stereoscopic display device 100 to execute the following processing is recorded on a computer-readable recording medium such as the ROM 55.

In S0, the stereoscopic display device 100 receives an input of a set of four parallax images from the external device 80 or the recording medium 57.

After S1, the stereoscopic display device 100 performs the same processing as in FIG. 3 on the set of parallax images input from the external device 80 or the recording medium 57. Thereby, also about the group of parallax images input from the external device 80, a stereoscopic image can be combined according to the direction at the time of imaging and displayed on the display device 30, as in the first embodiment.

Although the details are omitted, similarly, the stereoscopic display device 100 can perform the processes of FIGS. 6, 8, 10, 11, and 15 on the set of parallax images input from the external device 80 or the recording medium 57.

Furthermore, a stereoscopic image obtained by executing the various processes (compositing process, rotation process, notification process) described in the first to sixth embodiments is stored in a buffer of the display device 30 (not shown), and the display device 30 is displayed. May display stereoscopic images stored in this buffer. That is, the stereoscopic display device 100 only needs to include blocks necessary for executing each process other than the stereoscopic image display, and the display device 30 does not necessarily have to be provided integrally. An external device such as a compatible TV may be used. That is, the stereoscopic display device 100 may control image display on the external display device 30.

In this case, the stereoscopic display device 100 includes means for acquiring the display parallax direction of the external display device 30. For example, the display parallax direction (horizontal direction or the like) of the display device 30 is stored in the ROM 55 in advance, and the CPU 40 acquires this from the ROM 55 when the above processing is executed. Alternatively, the CPU 40 may acquire the display parallax direction of the display device 30 from the external device 80 via a network such as the external input unit 75. Alternatively, the CPU 40 may communicate with the external display device 30 and acquire the display parallax direction from the display device 30. Then, in S13 of the process of FIG. 6 described above, if the read frame is the same as the acquired display parallax direction, the process proceeds to S14, and if different, the process proceeds to S15. As described above, the display parallax direction of the external display device 30 can also be made to coincide with the parallax direction of the read frame by rotating the two-viewpoint image. Similarly, the processes of FIGS. 8, 11, and 15 can be applied to the external display device 30.

30: Display device, 34: 3D image signal processing unit, 71: Image selection unit, 72: Image rotation processing unit, 73: Gaze display control unit, 75: External input unit, 80: External device

Claims (17)

1. An input unit capable of inputting a set of images having parallax from the outside;
   A selection unit for selecting a desired set of images from the set of images input to the input unit;
   Whether a predetermined display parallax direction of the display unit capable of stereoscopically displaying a set of images having parallax matches an image parallax direction that is a parallax direction of a selected image set that is a set of images selected by the selection unit A determination unit for determining
   In response to determining that the display parallax direction and the image parallax direction do not coincide with each other, the determination unit rotates each image of the selected image set by an angle at which the display parallax direction and the image parallax direction coincide with each other. A display control unit for controlling the display unit to perform stereoscopic display;
   A stereoscopic image display control device comprising:
2. When the display parallax direction is a horizontal direction and the image parallax direction is a vertical direction, the display control unit performs control to rotate each image of the selected image set by 90 ° and to perform stereoscopic display on the display unit. The stereoscopic image display control apparatus according to claim 1.
3. A setting unit for setting whether to rotate the selected image set;
   When the setting unit is configured to rotate the image, the display control unit rotates each image of the selected image set by an angle at which the display parallax direction and the image parallax direction coincide with each other on the display unit. The stereoscopic image display control apparatus according to claim 1, wherein the stereoscopic image display control device controls the stereoscopic display.
4. The display control unit, when the setting unit is set not to rotate the image, information indicating a rotation angle of the image or a rotation direction of the display unit such that the display parallax direction matches the image parallax direction. The stereoscopic image display control device according to claim 3, wherein the stereoscopic image display control device controls the display to be displayed on the display unit.
5. An input unit capable of inputting a set of images having parallax from the outside;
   A selection unit for selecting a desired set of images from the set of images input to the input unit;
   Whether a predetermined display parallax direction of the display unit capable of stereoscopically displaying a set of images having parallax matches an image parallax direction that is a parallax direction of a selected image set that is a set of images selected by the selection unit A determination unit for determining
   When the determination unit determines that the display parallax direction and the image parallax direction do not coincide with each other, the rotation angle of the image or the display unit such that the display parallax direction coincides with the image parallax direction is determined. A display control unit that controls to display information indicating a rotation direction on the display unit;
   A stereoscopic image display control device comprising:
6. 6. The stereoscopic image display control device according to claim 5, wherein the information is a figure and / or a character indicating a rotation angle of the image or a rotation direction of the display unit.
7. The display parallax direction is a horizontal direction;
   The input unit can input a set of images having parallax in a single horizontal or vertical direction and a set of images having parallax in a plurality of horizontal and vertical directions. The three-dimensional image display control apparatus according to any one of the above.
8. When the selected image set has parallax in the horizontal and vertical directions, the display control unit stereoscopically displays the set of images having parallax in the horizontal direction on the display unit, and also displays the parallax in the vertical direction. The stereoscopic image display control apparatus according to claim 7, wherein control is performed so that information indicating that the group can be stereoscopically displayed is displayed on the display unit.
9. When the selected image set has parallax in the horizontal and vertical directions, the display control unit stereoscopically displays the set of images having parallax in the horizontal direction on the display unit, and also displays the parallax in the vertical direction. The stereoscopic image display control apparatus according to claim 7, wherein a control is performed to display a representative image indicating a state in which the set is rotated so as to coincide with the display parallax direction on the display unit.
10. The display control unit, when the selected image set has parallax in the horizontal and vertical directions, a set of rotated images obtained by rotating the set of images having parallax in the vertical direction by an angle that matches the display parallax direction; The stereoscopic image display control device according to claim 8, wherein the stereoscopic image display control device controls to display a set of images having parallax in the horizontal direction on the display unit sequentially or simultaneously.
11. The luminous flux from the subject is divided into four luminous fluxes by pupil division in the vertical and horizontal directions, and the four luminous fluxes are imaged and photoelectrically converted into four photoelectric conversion element groups corresponding to the color filters in a predetermined arrangement. An imaging unit that outputs a set of four viewpoint image signals having color components corresponding to each color filter,
    Out of a set of four viewpoint image signals having color components corresponding to each color filter output by the imaging unit, two viewpoint image signals obtained from photoelectric conversion elements adjacent in the vertical direction or adjacent in the horizontal direction. A synthesis unit that creates a set of images having parallax in the horizontal direction or a set of images having parallax in the vertical direction by synthesizing two viewpoint image signals obtained from the photoelectric conversion elements;
    With
    The stereoscopic image display control according to any one of claims 7 to 10, wherein the input unit inputs a set of images having parallax in the horizontal direction or a set of images having parallax in the vertical direction created by the combining unit. apparatus.
12. The synthesizing unit includes two viewpoint image signals obtained from photoelectric conversion elements adjacent in the vertical direction among a set of four viewpoint image signals having color components corresponding to the color filters output from the imaging unit, and By combining two viewpoint image signals obtained from photoelectric conversion elements adjacent in the horizontal direction, a set of images having parallax in the horizontal direction and a set of images having parallax in the vertical direction are created,
    The stereoscopic image display control device according to claim 11, wherein the input unit inputs a set of images having parallax in the horizontal direction and a set of images having parallax in the vertical direction created by the combining unit.
13. When the selected image set has a parallax in the horizontal and vertical directions, the display control unit selects a set of images having a parallax that matches a display parallax direction at the time of output of the imaging unit, and The three-dimensional image display control device according to claim 12, wherein the three-dimensional image display control is performed so as to perform three-dimensional display of the set.
14. A posture information detection unit for detecting posture information at the time of outputting a set of four viewpoint image signals of the imaging unit;
    The combining unit determines a set of left viewpoint image signals and a set of right viewpoint image signals obtained from the photoelectric conversion elements adjacent in the vertical direction based on the posture information detected by the posture information detection unit, 12. A set of images having a parallax in the horizontal direction is created by combining the determined set of left viewpoint image signals and combining the determined set of right viewpoint image signals. The stereoscopic image display control device according to any one of 13.
15. A display parallax direction acquisition unit that acquires a predetermined display parallax direction of the display unit;
    The determination unit determines whether or not a display parallax direction acquired by the display parallax direction acquisition unit and an image parallax direction that is a parallax direction of a selected image set that is a set of images selected by the selection unit match. The three-dimensional image display control device according to any one of claims 1 to 14.
16. Stereoscopic image display control device
    Inputting a set of images having parallax from the outside;
    Selecting a desired set of images from the set of input images;
    It is determined whether a predetermined display parallax direction of a display unit capable of stereoscopically displaying a set of images having parallax and an image parallax direction that is a parallax direction of a selected image set that is a set of the selected images are determined. Steps,
    In response to determining that the display parallax direction and the image parallax direction do not coincide with each other, the respective images of the selected image set are rotated and displayed in three dimensions by an angle at which the display parallax direction and the image parallax direction coincide with each other. Controlling to
    3D image display control method for executing
17. Stereoscopic image display control device
    Inputting a set of images having parallax from the outside;
    Selecting a desired set of images from the set of input images;
    It is determined whether a predetermined display parallax direction of a display unit capable of stereoscopically displaying a set of images having parallax and an image parallax direction that is a parallax direction of a selected image set that is a set of the selected images are determined. Steps,
    The display parallax direction coincides with the image parallax direction when it is determined that the predetermined display parallax direction of the display unit capable of stereoscopically displaying a set of images having parallax and the image parallax direction do not coincide with each other. Controlling to display information indicating the rotation angle of the image or the rotation direction of the display unit;
    3D image display control method for executing

Images