JP2013207678A - Image pickup device and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device which can smoothly switch from stereoscopic vision display to two-dimensional image display and a control method thereof.SOLUTION: The image pickup device comprises an image sensor 105 composed to have a plurality of photodiodes for a microlens constituting each pixel unit. A first PD selection-synthesis unit 106 has a right eye selection-synthesis mode and a left eye selection-synthesis mode in which outputs of the plurality of photodiodes are selected and synthesized, and a nonselection-synthesis mode which is set in 2D display control. A CPU 109 determines which operation to perform among 3D display control, 2D display switchover preparatory processing and 2D display control depending on camera manipulation or camera state. In preparatory processing for transition from 3D display to 2D display, the CPU 109 generates 2-dimensional display image data from right-eye or left-eye image data, and when switching over to 2D display, controls the display of the generated image data and also controls a subsequent image data generation process.

Description

  The present invention relates to an imaging apparatus having an imaging element having a configuration in which a plurality of light receiving elements are arranged for each microlens and a control method thereof.

  Conventionally, a stereo camera capable of capturing an image for 3D (three-dimensional) display or stereoscopic display has been proposed. In the apparatus disclosed in Patent Literature 1, two optical means and two image sensors are used to acquire stereo image data including a left-eye image and a right-eye image. Patent Document 2 discloses a method for acquiring data of a left-eye image and a right-eye image using one optical means. In addition, in a solid-state imaging device, a configuration having a microlens and a plurality of divided photodiodes constituting one pixel portion is known. Stereo image data can be acquired by utilizing the fact that each photodiode receives light from a different pupil plane.

  On the other hand, while performing the 3D display of the image data captured from the image sensor (live view), the user performs a predetermined camera operation or switches from the 3D display to the 2D display with a transition to a predetermined camera state as a trigger. The camera is known. In particular, before and after the touch panel operation, before and after the start of a predetermined setting state, the convenience of the user operation is improved by enabling switching between 3D display and 2D display. At that time, it is important to smoothly switch between 3D display and 2D display without causing the user to feel stress.

JP-A-01-202985 JP-A-58-24105

By the way, as the number of photodiodes arranged in one pixel portion increases, the amount of data read from the image sensor increases, and accordingly the processing time becomes longer. Accordingly, when switching between 3D display image capturing and 2D display image capturing in order, it is difficult to smoothly switch between displays. Therefore, it is required that the user can comfortably switch the display and check the image.
An object of the present invention is to provide an imaging apparatus capable of smoothly switching from stereoscopic display to two-dimensional image display and a control method thereof.

  In order to solve the above-described problems, an apparatus according to the present invention includes a plurality of photoelectric conversion units that receive and photoelectrically convert light beams that have passed through different regions of the exit pupil of the imaging optical system for each microlens. An image pickup apparatus that performs display control of image data acquired by the image pickup device, and generates right-eye image data and left-eye image data based on signals obtained from the plurality of photoelectric conversion units, and Data processing means for performing processing for generating first image data for two-dimensional display from signals output from the plurality of photoelectric conversion units; and stereoscopic display control using the right-eye image data and the left-eye image data; Control means for performing two-dimensional display control using the first image data is provided. The control means determines whether to perform the stereoscopic display control, to perform a preparation process for shifting from the stereoscopic display control to the two-dimensional display control, or to perform the two-dimensional display control, and to perform the stereoscopic display control. When generating the second image data for two-dimensional display from the image data for the right eye or the image data for the left eye in the preparation process for shifting from the visual display control to the two-dimensional display control, and switching to the two-dimensional display control The display control of the second image data is performed and the generation processing of the first image data by the data processing means is controlled.

  According to the present invention, it is possible to smoothly switch from stereoscopic display to two-dimensional image display.

It is a figure which shows the structural example of an imaging device, in order to demonstrate embodiment of this invention combined with FIG. 2 thru | or FIG. It is a figure which shows the structural example of an image pick-up element roughly. It is a figure which shows schematically the pixel array of an image pick-up element. It is a figure which shows typically the imaging relationship of an object. It is a flowchart which shows the example of switching control processing of a stereoscopic display and a two-dimensional display. It is a flowchart which shows the example of a determination process of a display switching state. It is a flowchart which shows the example of a display control process with FIG. It is a flowchart which shows the process example following FIG.

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings. Note that three-dimensional display or stereoscopic display is abbreviated as 3D display, and two-dimensional display is abbreviated as 2D display.
FIG. 1 is a schematic configuration diagram of an imaging apparatus according to an embodiment of the present invention. The imaging device of this embodiment is a digital camera, for example. The apparatus main body 100 includes a lens unit or a detachable lens apparatus.

  A CPU (Central Processing Unit) 109 controls the entire imaging apparatus. The power supply 110 supplies power to each circuit in the apparatus main body 100. A memory card 122 that is a removable recording medium can be inserted into the card slot 121. With the memory card 122 inserted in the card slot 121, the memory card 122 is electrically connected to the card input / output unit 120. In this embodiment, the memory card 122 is used as a recording medium. However, other recording media such as a hard disk, an optical disk, a magneto-optical disk, a magnetic disk, and other solid-state memories may be used.

  Of the lens group constituting the lens unit, FIG. 1 shows only the lenses 101 and 102. The lens 102 is a focus lens, and performs focus adjustment by moving back and forth in the direction along the optical axis. The diaphragm 103 adjusts the amount of light to the image sensor 105 with respect to light incident through the lens group. The lens 101, the focus lens 102, and the diaphragm 103 constitute an imaging optical system. The image sensor 105, the video signal processing unit 107, and the frame memory 108 constitute a photoelectric conversion system. This photoelectric conversion system converts the subject image formed by the imaging optical system into a digital image signal or image data.

  The imaging element 105 functions as a photoelectric conversion unit that photoelectrically converts a subject image formed by the imaging optical system and outputs an image signal. The image sensor 105 is an image sensor using a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The image sensor 105 has an electronic shutter function and can adjust the exposure time. Alternatively, the exposure time may be adjusted with a mechanical shutter instead of the electronic shutter function. The imaging element 105 has a configuration in which a plurality of photodiodes (hereinafter abbreviated as PD) are arranged for each microlens. The first PD selection / combination unit 106 has a function of selecting a necessary image signal from the outputs of a plurality of PDs and a function of combining and outputting image signals from the selected PDs. The first PD selection / combination unit 106 may be provided outside the image sensor 105.

  FIG. 2 is a diagram schematically illustrating a configuration example of an imaging element applied to the imaging apparatus of the present embodiment. FIG. 2A shows the overall configuration of the image sensor. The image sensor 105 includes a pixel array 301, a vertical selection circuit 302 that selects a row in the pixel array 301, and a horizontal selection circuit 304 that selects a column in the pixel array 301. The readout circuit 303 reads out the signal of the pixel portion selected by the vertical selection circuit 302 among the pixel portions in the pixel array 301. The reading circuit 303 includes a memory for storing signals, a gain amplifier, an A (Analog) / D (Digital) converter, and the like for each column.

  A serial interface (SI) unit 305 determines an operation mode of each circuit according to an instruction from the CPU 109. The vertical selection circuit 302 sequentially selects a plurality of rows of the pixel array 301 and extracts pixel signals to the readout circuit 303. The horizontal selection circuit 304 sequentially selects a plurality of pixel signals read by the reading circuit 303 for each column. In addition to the components shown in FIG. 2A, the imaging element 105 includes, for example, a timing generator that provides a timing signal to the vertical selection circuit 302, the horizontal selection circuit 304, the readout circuit 303, a control circuit, and the like. Although they exist, detailed description thereof will be omitted.

  FIG. 2B illustrates a configuration example of a pixel portion of the image sensor 105. A pixel portion 400 illustrated in FIG. 2B includes a microlens 401 as an optical element and a plurality of PDs 402a to 402i as light receiving elements. These PDs function as a photoelectric conversion unit that photoelectrically converts a received light beam to generate an image signal. Note that in the example illustrated in FIG. 2B, the number of PDs included in one pixel portion is nine, but the number of PDs may be an arbitrary number of two or more. In addition to the illustrated components, the pixel unit 400 includes, for example, a pixel amplification amplifier for reading a PD signal to the reading circuit 303, a selection switch for selecting a row, a reset switch for resetting the PD signal, and the like.

  FIG. 3 is a diagram illustrating a pixel array. In order to provide a two-dimensional image, the pixel array 301 is configured by arranging N pixel units 400 in the horizontal direction and M pixel units 400 in the vertical direction in a two-dimensional array. Each pixel unit 400 includes a color filter. For example, odd rows are repetitions of red (R) and green (G) color filters, and even rows are repetitions of green (G) and blue (B) color filters. That is, the plurality of pixel portions included in the pixel array 301 are arranged according to a predetermined pixel arrangement (in this example, a Bayer arrangement).

  Next, light reception of the image sensor having the pixel configuration shown in FIG. 3 will be described. FIG. 4 is a conceptual diagram showing a state in which a light beam emitted from the exit pupil of the photographing lens enters the image sensor 105. Each pixel array in the cross section 501 of the pixel array includes a microlens 502, a color filter 503, PDs 504 and 505. The PD 504 corresponds to the PD 402a in FIG. The PD 505 corresponds to the PD 402c in FIG.

  In FIG. 4, the center of the light beam emitted from the exit pupil 506 is indicated by an optical axis 509 with respect to the pixel portion having the exit pupil 506 of the photographing lens and the microlens 502. Light emitted from the exit pupil 506 is incident on the image sensor 105 with the optical axis 509 as the center. Partial areas 507 and 508 in the exit pupil 506 are different divided areas. Light rays 510 and 511 are outermost light rays among the light passing through the partial region 507. Light rays 512 and 513 are the outermost peripheral light rays among the light beams passing through the partial region 508. Of the light flux from the exit pupil 506, the light flux shown on the upper side in FIG. 4 is incident on the PD 505 with the optical axis 509 as a boundary, and the lower light flux is incident on the PD 504. That is, the PD 504 and the PD 505 have a characteristic of receiving light in different regions with respect to the exit pupil 506 of the photographing lens. Taking advantage of this characteristic, the imaging apparatus can acquire at least two images having parallax. For example, the imaging device may acquire two images in a region in the pixel unit using data obtained from a plurality of left PDs as a first line and data obtained from the plurality of right PDs as a second line. it can. The imaging apparatus can detect the phase difference using these two images and realize phase difference AF (autofocus). Furthermore, the imaging apparatus can execute a stereo image data generation process using the two images with parallax as a left-eye image and a right-eye image, and display a stereoscopic image on the stereo display device.

  Returning to FIG. 1, the description will be continued. The first PD selection / synthesis unit 106 includes the vertical selection circuit 302, the readout circuit 303, the horizontal selection circuit 304, and the SI unit 305 described with reference to FIG. The first PD selection / combination unit 106 operates according to an operation mode set by the CPU 109. This operation mode is set according to the type of shooting whose start is detected in the imaging apparatus, and includes a still image mode, a normal live view mode, and a stereo live view mode. In the still image mode, the first PD selecting / composing unit 106 outputs all PD outputs to the video signal processing unit 107 as image data. Further, in the normal live view mode, the first PD selection / combination unit 106 performs an addition average calculation on the outputs of all the PDs in the pixel, and sets the calculation result as an output value of the pixel. In the case of the stereo live view mode, the first PD selection / combination unit 106 performs the addition average value corresponding to each output of the PDs 402a, 402d, and 402g illustrated in FIG. 2B and the addition average corresponding to the outputs of the PDs 402c, 402f, and 402i. Calculate the value. These added average values are output values of the pixel portion. Thus, the amount of data can be reduced by adding and synthesizing the outputs of a plurality of PDs and using these calculation results as the output of one pixel.

  The first PD selection / combination unit 106 that performs the first selection / combination processing is a component that performs main data processing together with a second PD selection / combination unit 151 described later, and a process for generating right-eye image data and left-eye image data, respectively. A process of generating image data for two-dimensional display is performed. Hereinafter, in each selection / synthesis unit, the first mode related to the generation of the right-eye image data is referred to as a “right-eye selection / synthesis mode”, and the second mode related to the generation of the left-eye image data is referred to as a “left-eye selection / synthesis mode”. To do. Further, the third mode that is not directly involved in the data selection / synthesis process for the right-eye image or the left-eye image is referred to as a “non-selection synthesis mode”. The first PD selection / synthesis unit 106 selects and synthesizes signals related to the image data for the right eye from the outputs of the plurality of photoelectric conversion units in each pixel in the right-eye selection / synthesis mode. The first PD selection / combination unit 106 also selects and synthesizes signals related to the image data for the left eye from the outputs of the plurality of photoelectric conversion units in each pixel unit in the left-eye selection / combination mode. In the non-selective synthesis mode, the first PD selective synthesis unit 106 acquires all the outputs of the plurality of photoelectric conversion units of each pixel unit and sends them to the video signal processing unit 107.

  In the live view mode, a reduction in the amount of data output from the image sensor 105 is further required to increase the frame rate. For this reason, the first PD selection / combination unit 106 performs an averaging operation on the pixels of the same color filter at the closest position to obtain an output value. In the present embodiment, during live view, a process of averaging three pixel outputs of the same color in the horizontal direction and reading one pixel every three pixels in the vertical direction is executed. Thereby, the amount of data can be reduced to 1/9. In that case, the pixel to be used will be specifically described with reference to FIG. Here, in FIG. 3, the pixel portion in the nth row and the mth column is referred to as an “nm pixel portion”. The ranges of the natural number variables n and m are N ≧ n ≧ 1, M ≧ m ≧ 1, respectively.

  What is used for the arithmetic mean calculation related to the output of three pixels in the horizontal direction is the first red pixel portion in the first row, that is, the 1-1 pixel portion, the 1-3 pixel portion, and the 1-5 pixel portion. Each output. An arithmetic operation is performed on the output of each pixel unit selected according to the above-described mode, and a value for the corresponding pixel is calculated. The arithmetic processing is similarly executed for the next red pixel portion, which is the 1-7 pixel portion, the 1-9 pixel portion, and the 1-11 pixel portion. The arithmetic processing is similarly performed on the first green pixel portion in the first row, that is, the 1-2 pixel portion, the 1-4 pixel portion, and the 1-6 pixel portion. When the process of thinning out two pixels in the vertical direction is performed, the next row is the fourth row, and the first green pixel portion, that is, the 4-1 pixel portion, the 4-3 pixel portion, and the 4-5 pixel portion In the same manner, arithmetic processing is executed. The first blue pixel portion in the fourth row is a 4-2 pixel portion, a 4-4 pixel portion, and a 4-6 pixel portion, and arithmetic processing is similarly performed on these pixels.

  The video signal processing unit 107 in FIG. 1 performs known image signal processing such as shading correction on a digital image signal by a camera. The video signal processing unit 107 corrects image density non-linearity due to the characteristics of the image sensor 105 and image color bias due to the light source. The frame memory 108 functions as a buffer memory that temporarily stores image data generated by the image sensor 105 and the video signal processing unit 107. The image data stored in the frame memory 108 has been subjected to correction processing and the like, and corresponds to the digital data of the charge energy accumulated in each pixel portion of the image sensor 105. Hereinafter, the image data stored in the frame memory 108 is referred to as RAW data. In particular, the RAW data acquired by the first PD selection / combination unit 106 in the still image mode is referred to as all PD-RAW data. The RAW data acquired by the first PD selection / combination unit 106 in the case of the stereo live view mode is referred to as RAW data for both eyes. For both-eye RAW data, an averaging operation is performed on the output of the PD in each pixel unit in order to increase the frame rate. On the other hand, all PD-RAW data is intended to obtain as much detailed information as image data in order to handle the data in various image processing in a later process. Therefore, all PD-RAW data is not subjected to selection processing or composition processing, and data based on the output information of each PD in each pixel unit is individually stored in the frame memory 108. A parameter relating to the image quality of the RAW data image is called an imaging parameter, and includes a setting value Av of the aperture 103, a shutter speed Tv, ISO sensitivity, and the like.

  Connected to the bus 150 are a CPU 109, a power source 110, a nonvolatile memory 111, a development processing unit 112, a RAM (Random Access Memory) 113, a display control device 114, and a main switch 116. The CPU 109 controls the image signal readout of the image sensor 105 and controls the operation timing of each unit from the image sensor 105 to the frame memory 108. Also, release switches 117 and 118 that are operated by a user with a release button are connected to the bus 150. Hereinafter, the first release switch 117 is abbreviated as a first switch, and the second release switch 118 is abbreviated as a second switch. The shooting preparation operation is started by operating the first switch 117, and the shooting operation is performed by operating the second switch 118. Examples of operation members that constitute an operation unit connected to the bus 150 include a left selection button 140, a right selection button 141, a setting button 142, a development parameter change button 143, a live view start / end button 144, and a quick setting button 145. . In addition, a card input / output unit 120, a second PD selection / combination unit 151, a touch panel detection unit 119, and a drive unit 163 are connected to the bus 150. The touch panel detection unit 119 performs touch detection and proximity detection of the touch panel. The touch panel is a contact-type operation unit provided in the display unit, and is used by a user for operating instructions to the imaging apparatus. The driving unit 163 drives the focus lens 102, the diaphragm 103, and the like of the imaging optical system.

  The nonvolatile memory 111 stores an initial camera setting value to be set in the camera when the power supply 110 is turned on by operating the main switch 116. The non-volatile memory 111 is configured by an EEPROM (Electrically Erasable Programmable Read-Only Memory) or the like so that recorded data is not lost even when the power supply 110 is shut off.

  The second PD selection / synthesis unit 151 performs data processing related to the selection / synthesis of RAW data on the frame memory 108 or the RAM 113 based on the setting of the pixel selection / synthesis parameter, and stores the processed image data in the RAM 113. The pixel selection / synthesis parameter is a parameter for determining which PD output is combined with the RAW data to form one pixel data, and defines the contents of the selection / synthesis process of the second PD selection / synthesis unit 151. Information. The second PD selection / synthesis unit 151 has a right-eye selection / synthesis mode and a left-eye selection / synthesis mode. When the CPU 109 sets the right-eye selection / combination mode, the second PD selection / composition unit 151 selects the PD output necessary for obtaining the right-eye RAW data from all the PD-RAW data or the both-eye RAW data, and performs the composition processing. . In addition, when the CPU 109 sets the left-eye selection / combination mode, the second PD selection / composition unit 151 selects the output of the PD necessary for obtaining the left-eye RAW data from all the PD-RAW data or the both-eye RAW data, and performs the composition processing. Do. In this embodiment, the RAW data is input to the second PD selection / combination unit 151 and the development processing unit 112 via the frame memory 108 or the RAM 113, but even if the RAW data is directly input from the video signal processing unit 107. I do not care.

  The development processing unit 112 performs image processing on the raw data read out by the CPU 109 and synthesized for each pixel on the frame memory 108 or the RAM 113 based on the development parameter setting. The development parameter is a parameter relating to the image quality of the digital image data, and includes parameters relating to white balance, color interpolation, color correction, γ conversion, edge enhancement, resolution, image compression, and the like of the digital image data. Hereinafter, processing for adjusting or changing the image quality of digital image data using one or more development parameters is referred to as development processing. Each image data subjected to the development processing is stored in the RAM 113 as image data in the YUV422 format or YUV411 format, for example. The image data that has been subjected to development processing including compression processing is stored in the RAM 113 as image data in JPEG format, for example. JPEG is an abbreviation for “Joint Photographic Experts Group”. The RAM 113 temporarily stores data used when the CPU 109 performs various processes in addition to the image data that is the development processing result.

  The display control device 114 drives a TFT (thin film transistor) 115 of a liquid crystal display element and controls a display unit mounted on the imaging device main body. Further, the display control device 114 outputs the image data arranged in the display image format in the RAM 113 to the external display device via the video output terminal 132, the D terminal 133, the HDMI terminal 134, and the like. The RAM 113 in which the display image data is arranged is called a VRAM. The display device used in this embodiment can perform stereo display. In order to perform stereo display, the VRAM includes a right-eye image VRAM and a left-eye image VRAM. Stereo display is performed by arranging right-eye image data by the right-eye image VRAM and left-eye image data by the left-eye image VRAM on the display screen.

  The USB control device 127 controls communication between the imaging device and the external device via the USB terminal 128. The LAN control device 129 controls communication between the imaging device and the external device via the wired LAN terminal 130 or the wireless LAN interface unit 131.

Next, the operation of the imaging apparatus will be described.
When the user turns on the main switch 116, the CPU 109 executes a predetermined program. When the main switch 116 is turned off, the CPU 109 executes a predetermined program and sets the camera to the standby mode. By operating the release button, the first switch 117 is turned on in the first stroke (half-pressed state), and the second switch 118 is turned on in the second stroke (fully pressed state) of the release button. When the first switch 117 is turned on, the CPU 109 executes shooting preparation processing (including focus detection processing). When the second switch 118 is turned on, the CPU 109 detects a shooting start instruction and executes a shooting operation.

  Further, the CPU 109 performs control according to the operation state of the left selection button 140, the right selection button 141, and the setting button 142 and the operation state of the digital camera. For example, if the operation state of the digital camera is the playback state, control is performed such that the previous image data is displayed with the left selection button 140 and the next image data is displayed with the right selection button 141. Further, the user can confirm and set the development parameter setting by a graphical user interface by a setting change operation using the development parameter change button 143 or the quick setting button 145. Further, the CPU 109 accepts the operation of the live view start / end button 144, performs a RAW data capturing process from the image sensor 105 periodically (for example, 30 times per second), and places the developed data in the VRAM. Let Thereby, the image taken from the image sensor 105 is displayed in real time. When the user operates the live view start / end button 144 during the live view operation, the live view ends.

  In the reproduction state, menu state, and live view state, the CPU 109 analyzes proximity detection information and contact detection information by the touch panel detection unit 119 in addition to operation signals from the operation members. Thereby, the user can perform an operation instruction to the imaging apparatus using the touch panel.

Next, display switching control in this embodiment will be described. FIG. 5 is a flowchart illustrating a switching process between the stereoscopic display control and the two-dimensional display control in the live view.
In the display switching state determination process in S100, the display switching state is determined. The determination process is realized by the CPU 109 executing a program, and details of the process will be described with reference to FIG. Next, it is determined whether or not the display switching state is a state before the 2D display switching preparatory operation (hereinafter simply referred to as a pre-preparation state) (S101). This state before the preparatory operation is a state before the preparatory operation for switching to the display state of the two-dimensional image is performed. When it is determined that the display switching state is the pre-preparation operation state, the process proceeds to S104. When it is determined that the display switching state is not the pre-preparation operation state, the process proceeds to S102. In S104, 3D display control is performed. The 3D display control is performed by the CPU 109 and variable display control means using a program, and details thereof will be described with reference to FIG.

  In S102, it is determined whether or not the display switching state is a state after a 2D display switching preparation operation (hereinafter simply referred to as a state after a preparation operation). The state after the preparatory operation is a state in which it is determined that the preparatory operation for switching to the display state of the two-dimensional image has been performed. If it is determined that the display switching state is the state after the preparatory operation, the process proceeds to S105. If it is determined that the display switching state is not the state after the preparatory operation, the process proceeds to S103. In S105, display control before 2D display switching / 3D display is performed. In the preparation process for switching from 3D display to 2D display, second image data to be displayed in 2D is generated before the process of generating first image data for 2D display. Details thereof will be described with reference to FIG.

  In S103, it is determined whether or not the display switching state is a state after 2D display switching operation (hereinafter referred to as a state after switching operation) (S103). The state after the switching operation is a state in which it is determined that the switching operation to the display state of the two-dimensional image has been performed. When it is determined that the display switching state is the state after the switching operation, the process proceeds to S106, and when it is determined that the display switching state is not the state after the switching operation, the process proceeds to S107.

  In S106, display control for 2D display is performed. Details thereof will be described with reference to FIG. If it is determined after S104, S105, S106 or not in the post-switching state in S103, the state of the power switch (see main switch 116 in FIG. 1) is determined in S107. If the power switch is in the On state, the process returns to S100, and if the power switch is in the Off state, the process ends (S108).

Next, the display switching state determination process will be described with reference to the flowchart of FIG. This process is executed by the CPU 109 and display switching state determination means using a program.
First, in S200, the CPU 109 receives a detection signal from the touch panel detection unit 119, and determines whether or not the touch panel proximity detection state is set. The touch panel proximity detection state means not a contact state where a detection target such as a finger or a stylus touches the touch panel, but an approach state with a predetermined distance. When the distance between the detection target and the touch panel reaches the threshold value and the touch panel proximity detection state is determined, the process proceeds to S203. When it is determined that the touch panel proximity detection state is not detected, the process proceeds to S201. In S203, the display switching state is set to the post-preparation state, and the process proceeds to return processing (S206).

  In S201, the CPU 109 receives a detection signal from the touch panel detection unit 119, and determines whether or not it is in a touch panel contact detection state (S201). The touch panel contact detection state is a state in which it is detected that a finger, a stylus, or the like has touched the touch panel. If the determination result is a touch panel contact detection state, the process proceeds to S204. If it is determined that the touch panel contact detection state is not detected, the process proceeds to S202. In S204, the display switching state is set to the state after the switching operation, and the process proceeds to S206.

  In S202, the CPU 109 determines whether or not the current mode is various setting modes. The various setting modes include a mode in which a user performs a desired setting operation or change operation using an operation member (such as the quick setting button 145 in FIG. 1) provided in the apparatus. If the CPU 109 receives the operation signal by operating the operation member and determines that the current mode is the various setting modes, the CPU 109 advances the processing to S204 and sets the display switching state to the state after the switching operation. On the other hand, if the CPU 109 determines that the current mode is not the various setting mode, the display switching state is set to the pre-preparation operation state (S205), and the process proceeds to return processing (S206).

  Next, the display control from S104 to S106 in FIG. 5 will be described with reference to the flowcharts in FIGS. This process is executed by the CPU 109 and variable display control means using a program. 7 illustrates the display control in S104 and S105 in FIG. 5, and FIG. 8 illustrates the display control in S106 in FIG.

In S300 of FIG. 7, it is determined whether or not the current display form is 3D display. If the determination result is 3D display, the process proceeds to S301. If the determination result is not 3D display, the process proceeds to S307.
3D display control is performed from S301 to S306.

  The first PD selection / synthesis unit 106 is set to the right-eye selection / synthesis mode (S301). In the right-eye selection / combination mode, in order to generate right-eye image data, the first PD selection / composition unit 106 outputs a plurality of PDs (see PDs 402a, 402d, and 402g in FIG. 3A) selected from each pixel unit. Then, the averaging operation is performed to obtain the output value of each pixel unit. Next, the shooting operation starts with the shooting parameters for live view, and the frame memory 108 stores the right-eye RAW data acquired from the video signal processing unit 107. The development processing unit 112 performs development processing using the display development parameters, and generates right-eye image data (S302). The developed right-eye image data is placed in the right-eye VRAM (S303).

  Next, the first PD selection / synthesis unit 106 is set to the left-eye selection / synthesis mode (S304). In the left-eye selection / combination mode, the first PD selection / combination unit 106 generates the left-eye image data with respect to the outputs of a plurality of PDs selected from the pixel units (see PDs 402c, 402f, and 402i in FIG. 3A). Then, the averaging operation is performed to obtain the output value of each pixel unit. Next, the shooting operation starts with the shooting parameters for live view, and the frame memory 108 stores the left-eye RAW data acquired from the video signal processing unit 107. The development processing unit 112 performs development processing using the display development parameters, and generates image data for the left eye (S305). The developed left-eye image data is placed in the left-eye VRAM (S306), and the process proceeds to return processing (S322). In the 3D display control, right-eye image data and left-eye image data are generated by development processing, and stereoscopic display is performed.

  In S307, it is determined whether or not the current display mode is 3D display before 2D display switching. If the determination result is 3D display before 2D display switching, the process proceeds to S308. If not, the process proceeds to S315 in FIG.

  The display control shown from S308 to S314 is performed during a preparation period when switching from 3D display to 2D display. The processing from S308 to S310 is the same as the processing from S301 to S303. The first PD selection / combination unit 106 is set to the right-eye selection / combination mode (S308), the shooting operation is started with the live view imaging parameters, and the development processing unit 112 generates the right-eye image data using the display development parameters. (S309). The developed right-eye image data is placed in the right-eye VRAM (S310).

  The processing from S311 to S313 is the same as the processing from S304 to S306. The first PD selection / combination unit 106 is set to the left-eye selection / combination mode (S311), the photographing operation starts with the live view imaging parameters, and the development processing unit 112 generates the left-eye image data using the display development parameters. (S312). The developed left-eye image data is arranged in the left-eye VRAM (S313).

  In S314, 2D display image data (second image data) is generated using the right-eye image data or the left-eye image data. In other words, the right-eye image data or the left-eye image data arranged in each VRAM is used as it is in accordance with the display application to obtain 2D display image data, or data obtained by performing predetermined image processing on the 2D display data. Processing for display image data is performed. The generated image data is temporarily stored in the RAM 113 as extended 2D VRAM data, and the process proceeds to S322. In 3D display before 2D display switching, right-eye image data and left-eye image data are respectively generated by development processing to perform stereoscopic display, and 2D display images using right-eye image data or left-eye image data are displayed. Data generation processing is performed.

In S315 of FIG. 8, it is determined whether or not the current display form is 2D display. If the determination result is 2D display, the process proceeds to S316. If the determination result is not 2D display, the process proceeds to return processing (S322).
In S316 to S321, 2D display control is performed.

  It is determined whether or not the first selective combining unit 106 is set to the non-selective combining mode (S316). In a state where the imaging apparatus performs 2D display in the normal live view mode or the like, the first selection combining unit 106 is in the non-selection combining mode. In this case, the first PD selection / synthesis unit 106 remains set to the non-selection synthesis mode (S317). In the case of a setting mode other than the non-selective synthesis mode, the first PD selective synthesis unit 106 is set to the non-selective synthesis mode in S317.

  Next, the shooting operation starts with the shooting parameters for live view, all PD-RAW data is acquired on the frame memory 108 (S318), and the development processing unit 112 performs development processing using the development parameters for RAW images. 2D display image data is generated. The developed 2D display image data is placed in the 2D VRAM (S319), and the process proceeds to return processing (S322).

  On the other hand, when it is determined in the determination process of S316 that the mode is not the non-selective synthesis mode, it is determined whether or not the extended 2D VRAM data (see S314 in FIG. 7) has been generated (S320). When the extended 2D VRAM data has been generated, a process of acquiring the data from the RAM 113 and arranging it in the 2D VRAM is executed (S321), and 2D display is immediately performed. Thereafter, the processing from S317 to S319 is performed. That is, in S321, an image can be displayed on the display screen using the extended 2D VRAM data, and subsequent image data (first image data) can be generated for 2D display during the background process.

  As described above, according to the present embodiment, the 3D display state before switching to the 2D display is provided in the transition period from the 3D display state to the 2D display state, and the preparation process for 2D display is performed in this state, thereby 3D Switching from display to 2D display is performed promptly. Therefore, the user can visually switch the display from the 3D display to the 2D display smoothly without feeling the stress due to the switching waiting time.

  In the embodiment, the image data processing realized by the first PD selection / synthesis unit 106 that performs the first selection / synthesis process and the second PD selection / synthesis unit 151 that performs the second selection / synthesis process is exemplified. With this configuration, after the image data output from the first PD selection / combination unit 106 is temporarily stored in the storage unit, the second PD selection / combination unit 151 can generate stereo image data. However, the present invention is not limited to this, and an embodiment in which the first and second selection / combination processes are performed by one data processing unit, or an embodiment having a plurality of data processing units that perform processing for each setting mode may be used.

[Other Embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

105 Image sensor 106 First PD selection / synthesis unit 109 CPU
114 Display Control Device 119 Touch Panel Detection Unit 151 Second PD Selection / Composition Unit 401 Micro Lens 402a to i Photodiode

Claims (7)

The imaging device has a plurality of photoelectric conversion units that receive and photoelectrically convert light beams that have passed through different areas of the exit pupil of the imaging optical system for each microlens, and control display of image data acquired by the imaging device. An imaging device to perform,
Processing for generating right-eye image data and left-eye image data based on signals obtained from the plurality of photoelectric conversion units, and first image data for two-dimensional display from signals output from the plurality of photoelectric conversion units. Data processing means for performing the generation process;
Control means for performing stereoscopic display control using the right-eye image data and left-eye image data and two-dimensional display control using the first image data;
The control means determines whether to perform the stereoscopic display control, to perform a preparation process for shifting from the stereoscopic display control to the two-dimensional display control, or to perform the two-dimensional display control, and to perform the stereoscopic display control. When generating the second image data for two-dimensional display from the image data for the right eye or the image data for the left eye in the preparation process for shifting from the visual display control to the two-dimensional display control, and switching to the two-dimensional display control An image pickup apparatus that controls display of the second image data and controls generation processing of the first image data by the data processing means. The data processing unit is configured to select and synthesize a signal related to the right-eye image data from outputs of the plurality of photoelectric conversion units; and to output the left-eye image data from outputs of the plurality of photoelectric conversion units. A first selection / synthesis unit having a second mode for selecting and combining signals and a third mode for acquiring all the outputs of the plurality of photoelectric conversion units;
The control means includes the right-eye image data acquired by setting the first selection / combination means to the first mode in the preparation process for shifting from the stereoscopic display control to the two-dimensional display control, or The imaging apparatus according to claim 1, wherein a process for generating the second image data from the left-eye image data acquired by setting the first selection / combination unit to the second mode is controlled. Storage means for storing the image data output by the first selection / combination means set in the third mode;
The data processing unit selects and combines data used for generating the right-eye image data from the image data stored in the storage unit; and the left-eye image data from the image data stored in the storage unit The imaging apparatus according to claim 2, further comprising a second selection / combination unit having a mode for selecting and combining data used for generating the data. Contact detection means used for operating instructions to the imaging device,
The control unit performs a preparation process for determining an approaching state of a detection target based on a signal from the contact detection unit and shifting from the stereoscopic display control to the two-dimensional display control. 4. The imaging device according to any one of 3.   The imaging apparatus according to claim 4, wherein the control unit determines the contact state of a detection target based on a signal from the contact detection unit and performs the two-dimensional display control. An operation member used when performing a setting operation of the imaging apparatus;
The imaging apparatus according to claim 1, wherein the control unit performs the two-dimensional display control when an operation signal from the operation member is acquired. The imaging device has a plurality of photoelectric conversion units that receive and photoelectrically convert light beams that have passed through different areas of the exit pupil of the imaging optical system for each microlens, and control display of image data acquired by the imaging device. A control method executed by an imaging device to perform,
A determination step of determining whether to perform stereoscopic display control, to perform two-dimensional display control, or to perform preparation processing for shifting from the stereoscopic display control to the two-dimensional display control;
If it is determined in the determination step that the stereoscopic display control is to be performed, display control is performed by generating right-eye image data and left-eye image data based on a signal selected and acquired from the plurality of photoelectric conversion units. Steps,
When it is determined in the determination step that the two-dimensional display control is performed, the first image data for two-dimensional display is generated from the signals output from the plurality of photoelectric conversion units, and the display control is performed;
If it is determined in the determination step that preparation processing for shifting from the stereoscopic display control to the two-dimensional display control is performed, a second image for two-dimensional display is generated from the right-eye image data or the left-eye image data. A method for controlling an imaging apparatus, comprising the steps of performing display control of the second image data and generation processing of the first image data when generating data and switching to the two-dimensional display control.

Images