JP2013078038A - Imaging device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a sufficient three-dimensional image by taking two pieces of two-dimensional images using one imaging part.SOLUTION: The imaging device generates image data 801 for a left eye when a shutter button 181 is pressed down. The imaging device generates stereoscopic image data 830 using the image data 801 for the left eye and through image data 820. The imaging device displays a stereoscopic image on the basis of the created stereoscopic image data 830 on a display. The imaging device generates image data 802 for a right eye when the shutter button 181 is pressed down again while the stereoscopic image is displayed on the display. The imaging device generates the stereoscopic image data 803 using the image data 801 for the left eye and the image data 802 for the right eye when the image data 802 for the right eye is generated. The imaging device stores the generated stereoscopic image data 803 in a flash memory 17.

Description

  The present invention relates to an imaging apparatus, an image processing method, and a program. In particular, the present invention relates to an imaging device that generates stereoscopic image data, an image processing method in the imaging device, and a program for controlling the imaging device.

  2. Description of the Related Art Conventionally, there has been known an imaging apparatus that generates stereoscopic image data for displaying a stereoscopic image based on left-eye image data and right-eye image data obtained by imaging. Patent Document 1 discloses a monocular digital camera as the imaging apparatus. When the digital camera is set to the stereoscopic shooting mode, the digital camera continuously performs regular shooting at regular time intervals while the release button is pressed halfway. In the stereoscopic shooting mode, the user takes a picture of the subject while panning the camera with the release button half pressed. A digital camera takes two sets of images that have been panned most smoothly from two sets of images taken during continuous panning from images that have been shot continuously and taken at the position closest to the binocular parallax. Are extracted as a set of two images most suitable for a stereoscopic image. Furthermore, the digital camera detects the panning direction and sorts and records each of the two sets of images most suitable for the stereoscopic image into a left-eye image and a right-eye image.

  Patent Document 2 discloses a monocular digital video camera. The digital video camera captures the first still image by first pressing the shutter button, and then waits for the second shutter button to be pressed to instruct the second still image to be shot. An image obtained by blending the image based on the preview image and the first still image is generated and displayed as a display image. The image based on the still image is, for example, an image in the face area extracted from the first still image. The user operates the camera so that the same subject in the image based on the still image and the preview image overlaps on the display screen, and presses the shutter button for the second time, so that the first and first shooting ranges that coincide with each other. 2 still images are obtained.

  Patent Document 3 discloses a monocular imaging device that synthesizes a background image and a foreground image. The imaging device specifies a synthesis area in which the subject image in the background image is to be synthesized. In addition, the imaging device performs predetermined image processing on at least one of the subject image and the background image based on the synthesis region in which the identified subject image is to be synthesized. Furthermore, the imaging apparatus generates a subject composite image by combining the subject image on which the predetermined image processing has been performed and the background image.

  Patent Document 4 discloses a twin-lens digital stereoscopic camera. The digital stereoscopic camera includes a plurality of photographing means for photographing a subject, a display means capable of displaying a three-dimensional image composed of a plurality of images obtained by photographing the same subject from a plurality of viewpoints by a plurality of photographing means, and a display OSD control means for controlling the display of OSD information displayed superimposed on the image displayed on the means. The OSD control means erases the display of the OSD information displayed on the display means in relation to the shutter operation of the photographing means.

  Conventionally, a display using a parallax barrier method is known. When the imaging apparatus includes a display using a parallax barrier method, the user can visually recognize a stereoscopic image with the naked eye.

JP 2009-103980 A JP 2010-263269 A JP 2011-77850 A JP 2011-24123 A

  However, in the digital camera of Patent Document 1, it is necessary to obtain three or more two-dimensional images by imaging. Further, the digital video camera disclosed in Patent Document 2 and the imaging device disclosed in Patent Document 3 cannot generate and display a three-dimensional image. Furthermore, in the digital stereoscopic camera of Patent Document 4, imaging by two imaging units is necessary to obtain a stereoscopic image.

  The present invention has been made in view of the above problems, and its purpose is to obtain a suitable three-dimensional image by imaging two two-dimensional images using one imaging unit (camera 11). The present invention relates to a possible imaging device, an image processing method in the imaging device, and a program for controlling the imaging device.

  According to an aspect of the present invention, the imaging device generates stereoscopic image data for displaying a stereoscopic image by a parallax barrier method. The imaging device includes a processor, a memory, an imaging device for imaging a subject, and a display. The processor generates image data of one of the left-eye image data and the right-eye image data by using the image sensor based on accepting an external input based on pressing of the shutter button or selection of an object displayed on the display. To do. Based on the generation of one image data, the processor generates stereoscopic image data using the one image data and through image data obtained by imaging with the imaging element. The processor causes a display to display a stereoscopic image based on the generated stereoscopic image data. When the processor receives the external input again in a state where the stereoscopic image is displayed on the display, the processor generates the other image data of the left-eye image data and the right-eye image data using the imaging element. Based on the generation of the other image data, the processor generates stereoscopic image data using the one image data and the other image data. The processor stores stereoscopic image data generated using one image data and the other image data in a memory.

  Preferably, the imaging apparatus further includes an electronic shutter for generating through image data and a mechanical shutter for generating left-eye image data and right-eye image data. The processor generates image data of one of the left-eye image data and the right-eye image data using the mechanical shutter based on the reception of the external input. Based on the generation of one image data, the processor generates stereoscopic image data using the one image data and through image data obtained by imaging using the electronic shutter. When the processor receives the external input again in a state where the stereoscopic image is displayed on the display, the processor generates the other image data of the left-eye image data and the right-eye image data using the mechanical shutter.

  Preferably, the imaging device further includes a sensor that detects a change in posture of the imaging device. The memory stores a threshold value related to the change in posture in advance. The processor does not generate the other image data even if the external input is received again when the change in the posture is equal to or greater than the threshold when the one image data is generated.

  Preferably, when the deflection angle of the optical axis of the imaging apparatus accompanying a change in posture becomes equal to or greater than the threshold value, the processor does not generate the other image data even if a second external input is received.

  Preferably, the sensor detects the inclination of the imaging apparatus with the axis perpendicular to the vertical axis and the optical axis when one of the image data is generated as the rotation axis as the change in posture.

  Preferably, the sensor detects the inclination of the imaging apparatus with the vertical axis as the rotation axis as the change in posture.

  Preferably, the sensor detects an inclination of the imaging device with an axis parallel to the optical axis of the imaging device as a rotation axis as a change in posture.

  Preferably, when the change in posture is equal to or greater than the threshold, the processor displays a guidance display for setting the change in posture below the threshold on the display.

  According to another aspect of the present invention, the image processing method is executed in an imaging device that generates stereoscopic image data for displaying a stereoscopic image by a parallax barrier method. The image processing method is based on the fact that the imaging device accepts an external input based on pressing of the shutter button or selection of an object displayed on the display, and uses the image sensor to select one of the left-eye image data and the right-eye image data. Stereo image data using the step of generating one of the image data, and the imaging device using the one image data and the through image data obtained by imaging with the imaging device based on the generation of the one image data. A step in which the imaging device displays a stereoscopic image based on the generated stereoscopic image data on the display, and the imaging device accepts an external input again while the stereoscopic image is displayed on the display. A device for generating the other image data of the left-eye image data and the right-eye image data using the element. And the step of generating stereoscopic image data using the one image data and the other image data based on the generation of the other image data, and the imaging device Storing the stereoscopic image data generated using the data and the other image data in a memory.

  According to still another aspect of the present invention, the program is used to control an imaging apparatus that generates stereoscopic image data for displaying a stereoscopic image by a parallax barrier method. The program generates image data of one of the left-eye image data and the right-eye image data using the image pickup device based on accepting an external input based on pressing of the shutter button or selection of an object displayed on the display. A step of generating stereoscopic image data using the one image data and through image data obtained by imaging by the imaging device based on the generation of the one image data, and the generated stereoscopic image The step of displaying a stereoscopic image based on the image data on the display, and receiving the external input again in a state where the stereoscopic image is displayed on the display, the other of the left-eye image data and the right-eye image data using the image sensor Based on the step of generating image data and the generation of the other image data Generating stereo image data using one image data and the other image data; and storing stereo image data generated using the one image data and the other image data in a memory. And causing the imaging apparatus to execute.

  According to the present invention, it is possible to obtain a suitable three-dimensional image by imaging two two-dimensional images using one imaging unit.

It is the figure which showed the external appearance of the portable telephone. It is a figure for demonstrating the example of the data processing performed with a portable telephone, when a to-be-photographed object is imaged in 3D imaging mode. It is the figure which showed the hardware constitutions of the portable telephone. It is the figure which showed the principal part of the hardware constitutions of a portable telephone. It is a flowchart for demonstrating a part of data processing in a portable telephone. It is a flowchart for demonstrating the remaining processes of the data processing in a mobile telephone. It is a figure for demonstrating the change of the attitude | position of a mobile telephone. It is a figure for demonstrating the data table showing the threshold value of the inclination of a mobile telephone. It is a figure for demonstrating the guidance display displayed on a display. It is a flowchart for demonstrating a part of data processing in the portable telephone in a 1st modification. It is the figure which showed the external appearance of the other portable telephone. It is the figure which showed the principal part of the hardware constitutions of the portable telephone of FIG.

  Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  Examples of the imaging apparatus include digital single-lens reflex cameras, compact digital cameras, digital video cameras, portable telephones including smartphones, and electronic devices such as PDAs (Personal Digital Assistants). In the following, a portable phone with a camera will be described as an example of the imaging device.

<A. Appearance of mobile phone>
FIG. 1 is a diagram showing the appearance of the mobile phone 1. FIG. 1A is a front view of the mobile phone 1. FIG. 1B is a back view of the mobile phone 1. FIG. 1C illustrates a state when the subject is imaged.

  With reference to FIG. 1A, the mobile phone 1 includes a display 20 and operation buttons 18 including a shutter button 181. Note that the shutter button 181 may function as a button for instructing execution of processing other than imaging when the mode is not for imaging. Referring to FIG. 1B, the mobile phone 1 includes a camera 11 as an imaging unit and a strobe 19 on the surface opposite to the surface on which the display 20 is provided. Referring to FIG. 1C, the user holds the mobile phone 1 with the left hand 901 and the right hand 902 in order to take a picture of the subject.

  The mobile phone 1 has an imaging mode for generating a 3D image (stereoscopic image) (hereinafter, also referred to as “3D imaging mode”) and an imaging mode for generating a 2D image (hereinafter, “2D imaging mode”). And). Note that the switching process between the 3D imaging mode and the 2D imaging mode is performed according to a user instruction. When operating in the 3D imaging mode, the mobile phone 1 generates a 3D image that is viewed stereoscopically from the observer based on the left-eye image and the right-eye image captured by the camera 11. The mobile phone 1 displays a 3D image on the display 20 by a parallax barrier method.

  By the way, the mobile phone 1 is a monocular imaging device provided with only one camera. Therefore, in order to obtain a 3D image, the user needs the following operations. First, the user obtains an image for the left eye by imaging using the camera 11. Thereafter, the user moves the mobile phone 1 in the direction of the arrow 990. Finally, the user obtains an image for the right eye by imaging using the camera 11. Details of imaging will be described later.

  The mobile phone 1 also has a live view function for displaying a through image (preview image) on the display 20.

<B. Overview of mobile phone data processing>
FIG. 2 is a diagram for explaining an example of data processing performed by the mobile phone 1 when a subject is imaged in the 3D imaging mode.

  Referring to FIG. 2A, when the shutter button 181 is pressed by the user, the mobile phone 1 uses left-eye image data 801 obtained by imaging using the camera 11 as an image processing engine 13 described later. It is stored in the RAM 310 (see FIG. 4). Referring to FIG. 2B, when the left-eye image data 801 is stored in the RAM 310, the mobile phone 1 continuously generates through image data 820 by imaging using the camera 11. The through image data 820 is overwritten and updated in the RAM 310.

  Referring to FIG. 2C, when the through image data 820 is generated, the mobile phone 1 uses the left eye image data 801 and the through image data 820 obtained by the first imaging to generate a 3D image. Data 830 is generated. The 3D image data 830 is overwritten and updated in the RAM 310. The mobile phone 1 displays a 3D image based on the generated 3D image data 830 on the display 20. That is, the mobile phone 1 uses the left-eye image data 801 obtained by the previous imaging and the through image data 820 obtained after the imaging to display a 3D image on the display 20 in a live view.

  The user confirms the 3D image displayed in live view while moving the mobile phone 1 in the direction of the arrow 990 in FIG. For example, if the user moves the mobile phone 1 excessively in the direction of the arrow 990, an appropriate 3D image cannot be obtained. Further, when the user moves the mobile phone 1 in a direction other than the arrow, an appropriate 3D image cannot be obtained. For this reason, the user determines at which position the right-eye image is to be captured while visually recognizing the 3D image displayed in live view. When the user determines that an appropriate 3D image has been obtained, the user presses the shutter button 181 again.

  With reference to FIG. 2D, when the user presses the shutter button 181 again, the mobile phone 1 uses the camera 11 to perform imaging for obtaining right-eye image data. The mobile phone 1 stores the right-eye image data 802 obtained from the imaging in the RAM 310.

  Referring to FIG. 2 (e), when the right eye image data 802 is stored in the RAM 310, the mobile phone 1 stores the right eye image data 802 and the left eye image data 810 previously stored in the RAM 310. Is used to generate 3D image data 803. The mobile phone 1 stores the generated 3D image data in the RAM 310.

  Referring to FIG. 2 (f), the mobile phone 1 uses left-eye image data 801, right-eye image data 802, and 3D image data 803 as RAM (Random) provided outside the image processing engine 13. Access Memory) 16 (see FIGS. 3 and 4).

  Referring to FIG. 2 (g), the cellular phone 1 further receives image data saving instructions from the user or automatically without waiting for the instructions, the image data 801 and 802 stored in the RAM 16. , 803 are written into a flash memory 17 (see FIG. 3) to be described later.

  Instead of writing the image data 801, 802, 803 stored in the RAM 16 in the flash memory 17, it may be written in an IC (Integrated Circuit) recording medium 221 (see FIGS. 3 and 4) described later. In the following, for the sake of convenience of explanation, a configuration in which image data 801, 802, and 803 stored in the RAM 16 are written in the flash memory 17 will be described as an example.

  As described above, in the mobile phone 1, the user can visually recognize the 3D image displayed in the live view on the display 20 before capturing the right-eye image. Therefore, the user can take an image for the right eye at an appropriate position. Therefore, the user can obtain a suitable 3D image by imaging two two-dimensional images using one imaging unit. Note that the mobile phone 1 can generate 3D image data not only for horizontal shooting but also for vertical shooting, as in the case of horizontal shooting.

  Hereinafter, a specific configuration of the mobile phone 1 for realizing such data processing will be described.

<C. Hardware configuration>
FIG. 3 is a diagram showing a hardware configuration of the mobile phone 1. Referring to FIG. 3, a mobile phone 1 includes a camera 11, an image processing engine 13, a CPU (Central Processing Unit) 14 that executes a program, and a ROM (Read Only Memory) 15 that stores data in a nonvolatile manner. A RAM 16 for storing data in a volatile manner, a NAND flash memory 17, an operation button 18 for receiving an instruction input by a user of the mobile phone 1, a strobe 19, a display 20, and a communication IF (Interface). 21, a reader / writer 22, a power supply unit 23, and a gyro sensor 29. The configuration using the gyro sensor 29 will be described later (“<F. Modification>”).

  The image processing engine 13 generates image data based on charges generated by irradiating each image sensor included in the camera 11 with light. Details of the image processing engine 13 will be described later. The camera 11 is typically modularized (unitized).

The CPU 14 performs data processing other than the data processing performed by the image processing engine 13.
The flash memory 17 is a nonvolatile semiconductor memory. The flash memory 17 volatilely stores various data such as various programs for controlling the mobile phone 1, data generated by the mobile phone 1, and data acquired from an external device of the mobile phone 1.

  The communication IF 21 performs signal processing for the mobile phone 1 to perform wireless communication with an external communication device. The power supply unit 23 supplies power to the camera 11, the image processing engine 13, the CPU 14, the strobe 19, the display 20, the communication IF 21, the reader / writer 22, the gyro sensor 29, and the like via the data bus.

  Each component 11-23, 29 is mutually connected by the data bus. An IC recording medium 221 is attached to the reader / writer 22.

  The processing in the mobile phone 1 is realized by software executed by each hardware, the image processing engine 13 and the CPU 14. Such software may be stored in advance in the image processing engine 13 and the flash memory 17. The software may be stored in the IC recording medium 221 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the reader / writer 22 or other reading device, or downloaded via the communication IF and then temporarily stored in the flash memory 17. The software is read from the flash memory 17 by the CPU 14 and further stored in the flash memory 17 or the image processing engine 13 in the form of an executable program. The image processing engine 13 or the CPU 14 executes the program.

  Each component constituting the mobile phone 1 shown in the figure is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the image processing engine 13, the flash memory 17, the IC recording medium 221 and other storage media, or the software downloadable via the network. Since the operation of each hardware of mobile phone 1 is well known, detailed description will not be repeated.

  The recording medium is not limited to an IC card, but is a DVD-ROM, CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / It carries a fixed program such as a semiconductor memory such as a DVD (Digital Versatile Disc), an optical card, a mask ROM, an EPROM (Electronically Programmable Read-Only Memory), an EEPROM (Electronically Erasable Programmable Read-Only Memory), or a flash ROM. It may be a medium. The recording medium is a non-temporary medium that can be read by the computer.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

  FIG. 4 is a diagram showing a main part of the hardware configuration of the mobile phone 1. Referring to FIG. 4, a mobile phone 1 includes a camera 11, an image processing engine 13, a RAM 16, a flash memory 17, a display 20, a reader / writer 22, and an A / D (Analog / Digital) converter. 24. An IC recording medium 221 is attached to the reader / writer 22.

  The camera 11 includes a lens 111, a mechanical shutter 112, and an image sensor 113. Note that the imaging element is also called a light receiving element or an image sensor.

  The lens 111 collects external light. Specifically, the lens 111 condenses light reflected by the subject.

  The mechanical shutter 112 is a device for switching whether to irradiate the image sensor 113 with the light that has passed through the lens 111. The mechanical shutter 112 operates when the shutter button 181 is pressed. More specifically, the mechanical shutter 112 operates based on a command from the CPU 14. Here, “pressing the shutter button 181” excludes half-pressing of the shutter button 181 for focusing by the autofocus function.

  The image sensor 113 converts light into an electrical signal. Further, the image sensor 113 sends an electric signal to the A / D converter 24. The image sensor 113 is, for example, a CCD (Charge Coupled Device) or a CMOS (Charge Coupled Device).

  The A / D converter 24 converts the analog signal output from the image sensor 113 into a digital signal. The A / D converter 24 sends the digital signal obtained by the conversion to the image processing engine 13.

  Next, the image processing engine 13 will be described. The image processing engine 13 includes a processor 300, a RAM 310, and a ROM 320. The processor 300 includes a CPU 301 and a DSP (Digital Signal Processor) 302. The CPU 301, the DSP 302, the RAM 310, and the ROM 320 are connected to each other via a data bus.

  The processor 300 executes processing based on a program stored in the ROM 320. The CPU 301 executes processing related to overall control and communication in the image processing engine 13. The CPU 301 is, for example, a RISC (Reduced Instruction Set Computer) architecture CPU. The DSP 302 is a processor dedicated to image processing. Note that the number of DSPs 302 constituting the processor 300 is not limited to one.

  The RAM 310 functions as a buffer memory that temporarily stores the digital signal input from the A / D converter 24. The RAM 310 also functions as a work memory when data processing is performed by the CPU 301 and the DSP 302. ROM 320 includes a program executed by processor 300.

  The CPU 301 executes data writing to the RAMs 310 and 16, data reading from the RAMs 310 and 16, and erasing of data recorded in the RAMs 310 and 16. In addition, the CPU 301 controls writing of data to the IC recording medium 221, reading of data from the IC recording medium 221, and erasing of data recorded on the IC recording medium 221.

  Other processes executed by the image processing engine 13 will be described later (FIGS. 5 and 6).

<D. Control structure>
FIG. 5 is a flowchart for explaining a part of data processing in the mobile phone 1. FIG. 6 is a flowchart for explaining the remaining data processing in the mobile phone 1. Specifically, FIG. 5 and FIG. 6 show the flow of data processing in a state where the operation mode of the mobile phone 1 is switched to the 3D imaging mode.

  Referring to FIG. 5, in step S2, CPU 14 determines whether shutter button 181 has been pressed by the user. When the CPU 301 of the image processing engine 13 receives an instruction from the CPU 14 that the shutter button 181 has not been pressed (NO in step S2), in step S4, the subject is imaged using the electronic shutter. The electronic shutter is a shutter realized by the image processing engine 13 and does not require a physical light shielding plate.

  When imaging using an electronic shutter, for example, the CPU 301 opens the mechanical shutter 112 and closes the mechanical shutter 122. The CPU 301 receives a digital signal based on the analog signal output from the image sensor 113. Both mechanical shutters 112 may be opened.

  In step S6, the CPU 301 generates digital data (hereinafter referred to as “RAW data”). In step S8, the CPU 301 develops the generated RAW data. By the development processing, the RAW data becomes visible image data that can be recognized as an image by human eyes. In step S <b> 10, the CPU 301 displays a through image on the display 20. Thereby, the user can visually recognize the image of the subject imaged using the electronic shutter on the display 20 in real time.

  When the CPU 301 receives an instruction from the CPU 14 that the shutter button 181 has been pressed (YES in step S2), the CPU 301 performs imaging of the subject using the mechanical shutter 112 in step S12. Specifically, the CPU 301 performs imaging for generating a left eye image. In step S <b> 14, the CPU 301 generates RAW data based on the digital signals output from the A / D converter 24.

  In step S16, the CPU 301 performs development processing. Specifically, the CPU 301 performs sensor correction processing, demosaic processing, pixel interpolation processing, color correction processing (white balance adjustment, color matrix conversion, gamma conversion), and RGB image processing (sharpness correction, tone correction, exposure correction, etc.). Do.

  In step S18, the CPU 14 determines whether or not the shutter button 181 has been pressed by the user. If the CPU 14 determines that the shutter button 181 has not been pressed (NO in step S18), the CPU 14 performs imaging of the subject using the electronic shutter in step S20. In step S22, the CPU 301 generates RAW data. In step S24, the CPU 301 develops the generated RAW data.

  In step S <b> 26, the CPU 301 obtains image data obtained by imaging using the mechanical shutter 112 (that is, image data for the left eye) and image data obtained by imaging using the electronic shutter (that is, through image data). To generate 3D image data. In step S <b> 28, the CPU 301 displays a 3D image based on the generated 3D image data on the display 20. That is, the CPU 301 displays a 3D image on the display 20 in a live view.

  On the other hand, when CPU 14 determines that shutter button 181 has been pressed (YES in step S18), CPU 14 executes step S30 shown in FIG. Referring to FIG. 6, in step S <b> 30, the subject is imaged using mechanical shutter 112. Specifically, the CPU 301 performs imaging for generating a right eye image. In step S <b> 32, the CPU 301 generates RAW data based on the digital signals output from the A / D converter 24. In step S34, the CPU 301 performs development processing.

  In step S <b> 36, the CPU 301 generates 3D image data from two pieces of image data (that is, left-eye image data and right-eye image data) obtained by imaging using the mechanical shutter 112.

  In step S38, the CPU 301 writes the left-eye image data and right-eye image data obtained by the development processing in steps S16 and 34 and the 3D image data obtained in the processing in step S36 into the RAM 16. Specifically, the CPU 301 transfers the left-eye image data, the right-eye image data, and the 3D image data to the RAM 16 in association with each other.

  In step S40, the CPU 301 performs irreversible compression on the left-eye image data, the right-eye image data, and the 3D image data. For example, the CPU 301 converts left-eye image data, right-eye image data, and 3D image data into image data in a predetermined format (JPEG format, DCF format, Exif format, or TIFF format). In step S42, the CPU 301 writes the image data compressed in step S40 into the flash memory 17.

As described above, the CPU 301 ends the series of processes.
<E. Summary of mobile phone 1>
As described above, the mobile phone 1 is an imaging device that generates stereoscopic image data for displaying a stereoscopic image by the parallax barrier method. The mobile phone 1 includes a CPU 301, a flash memory 17, a camera 11 for imaging a subject, and a display 20. The camera 11 includes an image sensor 113, an electronic shutter for generating through-image data, and a mechanical shutter 112 for generating left-eye image data and right-eye image data.

  The CPU 301 generates image data for the left eye using the image sensor 113 and the mechanical shutter 112 based on the reception of the external input. Based on the generation of the left-eye image data, the CPU 301 generates stereoscopic image data using the left-eye image data and through image data obtained by imaging with the electronic shutter. The CPU 301 causes the display 20 to display a stereoscopic image based on the generated stereoscopic image data.

  When the CPU 301 accepts an external input again in a state where the stereoscopic image is displayed on the display 20, the CPU 301 uses the image sensor 113 and the mechanical shutter 112 to generate right-eye image data. The CPU 301 generates stereoscopic image data using the left-eye image data and the right-eye image data based on the generation of the right-eye image data. The CPU 301 stores the generated stereoscopic image data in the flash memory 17.

  Therefore, in the mobile phone 1, the user can visually recognize the 3D image displayed in the live view on the display 20 before capturing the right-eye image. Therefore, the user can take an image for the right eye at an appropriate position. Thereby, the user can obtain a suitable 3D image by imaging two two-dimensional images using one imaging unit (camera 11).

<F. Modification>
(F1. First modification)
In the above, the posture of the mobile phone 1 may not be appropriate when the shutter button 181 is pressed again to generate the image data for the right eye. For example, when the mobile phone 1 is tilted back and forth after the user presses the shutter button 181 to generate image data for the left eye, the mobile phone 1 is tilted left or right, the mobile phone 1 This is a case where the telephone 1 is tilted up and down.

  FIG. 7 is a diagram for explaining a change in the posture of the mobile phone 1. In the following, description will be made using an xyz-axis coordinate system. For the sake of simplicity, the positive direction of the z-axis is assumed to be vertically upward.

  The mobile phone 1 detects a change in the posture of the mobile phone 1 by a gyro sensor 29 (see FIG. 3).

  FIG. 7A is a diagram for explaining a case where the optical axis of the camera 11 is shaken with a change in the posture of the mobile phone 1. Specifically, FIG. 7A shows that the mobile phone 1 rotates an axis (an axis parallel to the y axis) perpendicular to the vertical axis (z axis) and the optical axis 1101 when the left-eye image data is generated. It is a figure for demonstrating the case where it has rotated as an axis | shaft.

  Referring to FIG. 7A, gyro sensor 29 detects the inclination of mobile phone 1 in the directions of arrows 510 and 520 as the forward and backward inclinations. The detection result by the gyro sensor 29 is sent to the CPU 301. When the tilt angle (the swing angle of the optical axis 1101) is equal to or greater than a predetermined threshold, the CPU 301 does not generate right-eye image data using the mechanical shutter 112 even when the shutter button 181 is pressed. With such a configuration, the mobile phone 1 can generate suitable 3D image data. Hereinafter, the inclination angle when the mobile phone 1 is inclined in the direction of the arrow 510 or the arrow 520 is referred to as a “pitch angle” for convenience of explanation.

  FIG. 7B is a diagram for explaining a case where the optical axis 1101 of the camera 11 swings with a change in the attitude of the mobile phone 1, as in FIG. 7A. More specifically, FIG. 7B is a diagram for explaining a case where the mobile phone 1 has rotated about the vertical axis (z axis) as a rotation axis.

  Referring to FIG. 7B, gyro sensor 29 detects the tilt of mobile phone 1 in the directions of arrows 530 and 540 as the tilt to the left and right. The detection result by the gyro sensor 29 is sent to the CPU 301. When the tilt angle (the swing angle of the optical axis 1101) is equal to or greater than a predetermined threshold, the CPU 301 does not generate right-eye image data using the mechanical shutter 112 even when the shutter button 181 is pressed. With such a configuration, the mobile phone 1 can generate suitable 3D image data. Hereinafter, the tilt angle when the mobile phone 1 tilts in the direction of the arrow 530 or the arrow 540 is referred to as “yaw angle” for convenience of explanation.

  FIG. 7C illustrates a case where the mobile phone 1 rotates as an axis parallel to the optical axis 1101 of the camera 11 (an axis parallel to the x axis) as a change in posture of the mobile phone 1. FIG.

  Referring to FIG. 7C, gyro sensor 29 detects the tilt of mobile phone 1 in the directions of arrows 550 and 560 as the vertical tilt. The detection result by the gyro sensor 29 is sent to the CPU 301. When the tilt angle is equal to or greater than a predetermined threshold, the CPU 301 does not generate right-eye image data using the mechanical shutter 112 even when the shutter button 181 is pressed. With such a configuration, the mobile phone 1 can generate suitable 3D image data. Hereinafter, the tilt angle when the mobile phone 1 tilts in the direction of the arrow 550 or the arrow 560 is referred to as a “roll angle” for convenience of explanation.

  FIG. 8 is a diagram for explaining a data table 600 that represents the tilt threshold of the mobile phone 1. The data table 600 is stored in advance in the flash memory 17, for example. Referring to FIG. 8, in data table 600, “Th1” is stored as a threshold value for the pitch angle. In the data table 600, “Th2” is stored as a threshold value for the yaw angle. Furthermore, in the data table, “Th3” is stored as a threshold value for the roll angle. The CPU 301 refers to each threshold value in the data table 600 and determines whether to generate image data for the right eye using the mechanical shutter 112.

  Note that the mobile phone 1 may be configured so that the values of the threshold values Th1 to Th3 can be changed by the user.

  As described above, the mobile phone 1 includes the gyro sensor 29 that detects a change in the posture of the mobile phone 1. The flash memory 17 stores in advance threshold values Th1 to Th3 related to posture changes. When the change in the attitude of the mobile phone 1 is equal to or greater than the threshold when the left-eye image data is generated, the CPU 301 sets the right-eye image data using the mechanical shutter 112 even when the shutter button 181 is pressed. Does not generate.

  FIG. 9 is a diagram for explaining the guidance display displayed on the display 20. As described above, when the change in the attitude of the mobile phone 1 is equal to or greater than the threshold value, the right eye image data is not generated using the mechanical shutter 112 even when the shutter button 181 is pressed. That is, the mobile phone 1 does not permit the generation of right-eye image data. Therefore, the mobile phone 1 causes the display 20 to display a guidance display for setting the change in the posture of the mobile phone 1 to be less than the threshold value.

  Referring to FIG. 9, for example, when the mobile phone 1 is tilted by the threshold Th1 or more in the direction of the arrow 520 (that is, backward) as compared with the first shooting, the CPU 14 of the mobile phone 1 Then, a guidance display 700 for guiding the user to tilt the mobile phone 1 forward is displayed on the display 20. On the other hand, when the mobile phone 1 is tilted by the threshold Th1 or more in the direction of the arrow 510, the CPU 14 displays guidance for guiding the user to tilt the mobile phone 1 backward on the display 20.

  When the mobile phone 1 is tilted by the threshold Th2 or more in the direction of the arrow 530, the CPU 14 displays guidance for guiding the user to tilt the mobile phone 1 to the right on the display 20. On the other hand, when the mobile phone 1 is tilted by the threshold Th2 or more in the direction of the arrow 540, the CPU 14 displays guidance for guiding the user to tilt the mobile phone 1 leftward on the display 20.

  When the mobile phone 1 is tilted by the threshold Th3 or more in the direction of the arrow 550, the CPU 14 displays a guide for guiding the user to tilt the mobile phone 1 counterclockwise on the display 20. . On the other hand, when the mobile phone 1 is tilted in the direction of the arrow 560 by the threshold Th3 or more, the CPU 14 displays guidance for guiding the user to tilt the mobile phone 1 clockwise on the display 20.

  With this configuration, the user can move the mobile phone 1 to an imaging position where generation of right-eye image data using the mechanical shutter 112 is permitted.

  In the above description, the positive direction of the z-axis is described as being vertically upward for simplification of description, but the present invention is not limited to this. In order to obtain a 3D image, the user may take an image of a subject with the mobile phone 1 tilted. Even in this case, as described above, by using the deflection angle of the optical axis 1101 and the inclination angle of the mobile phone 1 with the axis parallel to the optical axis 1101 as the rotation axis, the positive z-axis is obtained. As in the case where the orientation is vertically upward, the mobile phone 1 can generate suitable 3D image data.

  FIG. 10 is a flowchart for explaining part of the data processing in the mobile phone 1 according to the first modification. Specifically, FIG. 10 shows the flow of data processing when the operation mode of the mobile phone 1 is switched to the 3D imaging mode. Specifically, FIG. 10 illustrates, as an example, a configuration that does not permit generation of right-eye image data using the mechanical shutter 112 when the pitch angle and the yaw angle are equal to or greater than the threshold values Th1 and Th2, respectively. FIG. Hereinafter, differences from the flowchart of FIG. 5 will be mainly described.

  Prior to step S2, in step S102, the CPU 14 starts detecting the pitch angle P and the yaw angle Y. When the shutter button 181 is pressed in step S2, prior to step S12, in step S104, the CPU 14 stores the current pitch angle P and yaw angle Y in the RAM 16 as the pitch angle Po and yaw angle Yo, respectively. To do. “Current pitch angle P and yaw angle Y” refer to the pitch angle P and yaw angle Y when the CPU 14 detects that the shutter button 181 has been pressed.

  After step S28, in step S106, the CPU 14 determines whether or not both of the following expressions (1) and (2) are established.

| Po-P | <Th1 (1)
| Yo-Y | <Th2 (2)
When equations (1) and (2) are both established (YES in step S106), CPU 14 advances the process to step S18. On the other hand, when at least one of the formula (1) and the formula (2) is not established, the CPU 14 causes the display 20 to display the guidance described above in step S108.

  If it is determined in step S18 that the shutter button 181 has been pressed, the CPU 14 executes the process of step S110 prior to step S30 (see FIG. 6). Specifically, in step S110, the CPU 14 executes the same determination process as in step S106.

  When Expression (1) and Expression (2) are both established (YES in Step S110), CPU 14 advances the process to Step S30. On the other hand, if at least one of Expression (1) and Expression (2) is not satisfied, the CPU 14 causes the display 20 to display that the imaging by the mechanical shutter 112 is not permitted in Step S112. After step S112, the CPU 14 advances the process to step S20.

  In the above description, the configuration in which the generation of the image data for the right eye using the mechanical shutter 112 is not permitted when the pitch angle and the yaw angle are equal to or greater than the threshold values Th1 and Th2 has been described as an example. It is not limited to. For example, when the pitch angle, yaw angle, and roll angle are equal to or greater than the threshold values Th1, Th2, and Th3, respectively, the mobile phone 1 is configured not to permit generation of right-eye image data using the mechanical shutter 112. May be.

(F2. Second modification)
FIG. 11 is a diagram showing an appearance of the mobile phone 1A. FIG. 11A is a front view of the mobile phone 1A. FIG. 11B is a back view of the mobile phone 1A. FIG. 11C is a diagram illustrating a state when the subject is imaged.

  With reference to FIG. 11A, the mobile phone 1 </ b> A includes a display 20 and operation buttons 18 including a shutter button 181. Referring to FIG. 11B, mobile phone 1A includes cameras 11 and 12 and strobe 19 on the surface opposite to the surface on which display 20 is provided. Referring to FIG. 11C, the user holds portable phone 1A with right hand 902 in order to take a picture of the subject.

  The mobile phone 1A includes a 3D imaging mode and a 2D imaging mode. When operating in the 3D imaging mode, the mobile phone 1A generates a 3D image that can be viewed stereoscopically from the observer based on the left-eye image captured by the camera 11 and the right-eye image captured by the camera 12. To do. The mobile phone 1A displays a 3D image on the display 20 by a parallax barrier method.

  The mobile phone 1A can generate 3D image data by capturing a subject using the two cameras 11 and 12 simultaneously in the case of landscape shooting. However, in the mobile phone 1A, 3D image data cannot be generated even when the two cameras 11 and 12 are used at the same time during vertical shooting.

  Therefore, in the mobile phone 1 </ b> A, when the mobile phone 1 </ b> A is in the vertical shooting posture, the imaging of either the camera 11 or the camera 12 is stopped. As a result, the mobile phone 1 </ b> A has the same configuration as a monocular camera like the mobile phone 1. Therefore, 3D image data can be generated even in vertical shooting by performing the same imaging process as that of the mobile phone 1 in the mobile phone 1A.

  FIG. 12 is a diagram showing a main part of the hardware configuration of the mobile phone 1A. Referring to FIG. 12, a mobile phone 1A includes a camera 11, a camera 12, an image processing engine 13, a RAM 16, a flash memory 17, a display 20, a reader / writer 22, and an A / D converter 24. And an A / D converter 25. An IC recording medium 221 is attached to the reader / writer 22.

The camera 12 includes a lens 121, a mechanical shutter 122, and an image sensor 123.

  The lens 121 collects external light. The mechanical shutter 122 is a device for switching whether to irradiate the image sensor 123 with light that has passed through the lens 121. The mechanical shutter 122 operates when the shutter button 181 is pressed. More specifically, the mechanical shutter 122 operates based on a command from the CPU 14. The image sensor 123 converts light into an electrical signal. Further, the image sensor 123 sends an electric signal to the A / D converter 25. The image sensor 123 is, for example, a CCD (Charge Coupled Device) or a CMOS (Charge Coupled Device).

  The A / D converter 25 converts the analog signal output from the image sensor 123 into a digital signal. The A / D converter 25 sends the digital signal obtained by the conversion to the image processing engine 13.

  Thus, the mobile phone 1A is different from the hardware configuration of the mobile phone 1 in that the camera 12 and the A / D converter 25 are further provided. Since other hardware configurations are the same as those of the mobile phone 1, the description thereof will not be repeated here.

(F3. Third modification)
In the above description, the configuration in which the mechanical shutter 112 operates based on the depression of the shutter button 181 has been described as an example. However, the present invention is not limited to this. The mechanical shutter 112 may be operated based on receiving an external input for selecting an object displayed on the display 20 instead of an external input by pressing the shutter button. As an example of the object, an image for allowing the user to recognize that the button is a software button for performing camera photographing can be given.

(F4. Fourth modification)
In the above description, the configuration of generating the right-eye image data after generating the left-eye image data has been described as an example, but the generation order of the left-eye image data and the right-eye image data is limited to this. is not. The portable telephones 1 and 1A may be configured to generate the image data for the left eye after the image data for the right eye is generated. In this case, after the first press of the shutter button 181, the user moves the mobile phone 1, 1 </ b> A in the direction opposite to the direction indicated by the arrow 990 in FIG. It is necessary to press the shutter button 181 for the second time.

(F5. Fifth modification)
In the above description, the mobile phone 1, 1 </ b> A has been described with reference to the configuration including the electronic shutter and the mechanical shutter, but the present invention is not limited to this. A configuration including only an electronic shutter without including a mechanical shutter may be employed. That is, the through image data, the left-eye image data, and the right-eye image data may be obtained by the electronic shutter.

(F6. Sixth Modification)
In the above description, the case where the gyro sensor 29 is used as the means for detecting the change in the attitude of the mobile phone 1 has been described as an example, but the present invention is not limited to this. For example, an acceleration sensor and a geomagnetic sensor may be used instead of the gyro sensor.

  The embodiment disclosed this time is an exemplification, and the present invention is not limited to the above contents. The scope of the present invention is defined by the terms of the claims, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1, 1A mobile phone, 11, 12 camera, 13 image processing engine, 16, 310 RAM, 17 flash memory, 18 operation buttons, 20 display, 22 reader / writer, 29 gyro sensor, 111, 121 lens, 112, 122 mechanical Shutter, 113, 123 Image sensor, 181 Shutter button, 221 IC recording medium, 300 processor, 320 ROM, 600 data table, 700 guidance display, 801, 810 Left-eye image data, 802 Right-eye image data, 803, 830 Stereoscopic image Data, 820 thru image data, 1101 optical axis.

Claims (10)

An imaging apparatus that generates stereoscopic image data for displaying a stereoscopic image by a parallax barrier method,
A processor;
Memory,
An image sensor for imaging a subject;
With a display,
The processor is
Based on accepting an external input based on pressing of a shutter button or selection of an object displayed on the display, image data for one of left-eye image data and right-eye image data is generated using the imaging device. ,
Based on the generation of the one image data, three-dimensional image data is generated using the one image data and the through image data obtained by imaging with the imaging element,
Displaying a stereoscopic image based on the generated stereoscopic image data on the display;
When the external input is received again in a state where the stereoscopic image is displayed on the display, the image data for the left eye and the image data for the right eye are generated using the imaging element,
Based on the generation of the other image data, the one image data and the other image data are used to generate stereoscopic image data,
An imaging apparatus that stores the stereoscopic image data generated using the one image data and the other image data in the memory. An electronic shutter for generating the through image data;
A mechanical shutter for generating the left-eye image data and the right-eye image data;
The processor is
Based on receiving the external input, using the mechanical shutter, to generate one image data of the left-eye image data and the right-eye image data,
Based on the generation of the one image data, the stereoscopic image data is generated using the one image data and the through image data obtained by imaging using the electronic shutter,
When receiving the external input again in a state where the stereoscopic image is displayed on the display, the other image data of the left-eye image data and the right-eye image data is generated using the mechanical shutter. Item 2. The imaging device according to Item 1. A sensor for detecting a change in posture of the imaging apparatus;
The memory stores in advance a threshold related to the change in posture,
The processor does not generate the other image data even if the second external input is accepted when the change in the posture is equal to or greater than the threshold value when the one image data is generated. 2. The imaging device according to 2.   The processor does not generate the other image data even if the second external input is accepted when the deflection angle of the optical axis of the imaging apparatus accompanying the change in the posture is equal to or greater than the threshold value. The imaging device described in 1.   5. The sensor according to claim 4, wherein the sensor detects an inclination of the imaging apparatus with a vertical axis and an axis perpendicular to an optical axis when the one image data is generated as the change in posture. 6. Imaging device.   The imaging device according to claim 4, wherein the sensor detects an inclination of the imaging device with a vertical axis as a rotation axis as the change in the posture.   The imaging device according to claim 3, wherein the sensor detects an inclination of the imaging device with an axis parallel to the optical axis of the imaging device as a rotation axis as the change in the posture.   8. The processor according to claim 3, wherein when the change in the posture is equal to or greater than the threshold, the processor displays a guidance display for setting the change in the posture to be less than the threshold on the display. 9. The imaging device described in 1. An image processing method in an imaging apparatus for generating stereoscopic image data for displaying a stereoscopic image by a parallax barrier method,
The image pickup device accepts an external input based on pressing of a shutter button or selection of an object displayed on a display, and one image data of left eye image data and right eye image data using an image pickup device. A step of generating
The imaging device, based on the generation of the one image data, generating stereoscopic image data using the one image data and through image data obtained by imaging with the imaging element;
The imaging device displaying a stereoscopic image based on the generated stereoscopic image data on a display;
When the imaging device receives the external input again in a state where the stereoscopic image is displayed on the display, the other image data of the left-eye image data and the right-eye image data is obtained using the imaging device. Generating step;
The imaging device generates stereoscopic image data using the one image data and the other image data based on the generation of the other image data;
An image processing method, comprising: a step of storing the stereoscopic image data generated using the one image data and the other image data in a memory. A program for controlling an imaging device that generates stereoscopic image data for displaying a stereoscopic image by a parallax barrier method,
Generating image data of one of left-eye image data and right-eye image data using an imaging device based on receiving an external input based on pressing of a shutter button or selection of an object displayed on a display; ,
Generating one-dimensional image data using the one image data and the through-image data obtained by imaging by the imaging element based on the generation of the one image data;
Displaying a stereoscopic image on the display based on the generated stereoscopic image data;
When receiving the external input again in a state where the stereoscopic image is displayed on the display, generating the other image data of the left-eye image data and the right-eye image data using the imaging device;
Generating stereoscopic image data using the one image data and the other image data based on the generation of the other image data;
A program that causes the imaging apparatus to execute the step of storing the stereoscopic image data generated using the one image data and the other image data in a memory.

Images