JP5001960B2 - 3D image display apparatus and method - Google Patents

Description

  The present invention relates to a three-dimensional image display apparatus and method, and more particularly, to a three-dimensional image display apparatus and method for sequentially reading out and reproducing images from a recording medium in which two-dimensional images and three-dimensional images are recorded together.

  A digital camera having a photographing function of a two-dimensional image (2D image) and a three-dimensional image (3D image) records a 2D image and a 3D image in a mixed state on one recording medium. Since a 2D image and a 3D image are mixedly recorded on a recording medium on which an image is recorded by this type of digital camera, the image on the recording medium is recorded by, for example, a file name order (photographing). When sequentially reproduced in the order of date and time), 2D images and 3D images are mixedly reproduced.

  When a plurality of images in which a 2D image and a 3D image are mixed are sequentially played back sequentially, the display changes from a 2D image to a 3D image, or from a 3D image to a 2D image. The change of the image without feeling continues, and may give fatigue to the observer of the image.

  Conventionally, in order to solve this feeling of fatigue, it is determined whether an input image is a 2D image or a 3D image. When the input image is determined to be a 2D image, a pseudo parallax image is formed from the 2D image by calculation (a 2D image is converted into a 2D image). An image display device that has been converted to a 3D image and displayed in 3D has been proposed (Patent Document 1).

  Further, Patent Document 1 does not describe a specific calculation method for forming a parallax image in a pseudo manner by calculation from a 2D image. However, as a method for forming a 3D image from a 2D image by calculation, Patent Document 2 can be used. The method described in is known.

  The method for converting a 2D image into a 3D image described in Patent Document 2 generates depth information for each setting region of the 2D image, generates disparity information from the depth information, and generates a 3D image from the disparity information. The depth information generates an image feature amount from a 2D image, and performs grouping for each object in the screen from the image feature amount, while generating a background weight for each setting region, Calculation is performed by combining the image feature amount and the background weight for each group from the feature amount, the information about the group, and the background weight.

JP 2008-42645 A JP-A-11-98531

  However, as in the inventions described in Patent Documents 1 and 2, the method of converting a 2D image into a 3D image by calculation requires a large amount of calculation for converting the 3D image into a 3D image, and requires a lot of time to calculate the 3D image. And the problem of requiring resources.

  The present invention has been made in view of such circumstances. Even when a 2D image and a 3D image are mixedly input sequentially, when a 2D image is input, a parallax image is obtained from the 2D image. Because it is generated and played back, only images with a three-dimensional effect can be played back continuously, which can reduce the fatigue of the image observer, especially from 2D images easily and instantaneously. It is an object of the present invention to provide a three-dimensional image display apparatus and method capable of generating and reproducing an image.

In order to achieve the object, a three-dimensional image display device according to claim 1 is a recording medium in which a two-dimensional image and a three-dimensional image composed of a plurality of images obtained by photographing the same subject from a plurality of viewpoints are mixedly recorded. An image input means for sequentially inputting a two-dimensional image or a three-dimensional image, a determination means for determining whether the image input from the image input means is a two-dimensional image or a three-dimensional image, and a two-dimensional image by the determination means. A parallax image generating means for generating a parallax image for three-dimensional display based on the input two-dimensional image when it is determined that the two-dimensional image is input, wherein the two-dimensional image is uniformly left and right by a predetermined amount of parallax; A parallax image generating unit that generates a plurality of parallax images that are shifted to each other, and a tertiary that displays a three-dimensional image input from the image input unit or a plurality of parallax images generated by the parallax image generating unit Comprising an image display unit, and a parallax amount acquisition means for acquiring a parallax quantity of information of a three-dimensional image of the vicinity of the two-dimensional image recorded on the recording medium, the parallax image generating means, the two-dimensional image As a predetermined amount of parallax when generating a parallax image for three-dimensional display based on the above, a parallax amount of a three-dimensional image in the vicinity of the two-dimensional image acquired by the parallax amount acquisition unit is used. Yes.

According to the first aspect of the present invention, when an image in which a two-dimensional image and a three-dimensional image are mixed is sequentially input and reproduced, if it is determined that the input image is a two-dimensional image, Since a plurality of parallax images formed by shifting the two-dimensional image uniformly in the left-right direction by a predetermined amount of parallax based on the two-dimensional image and displaying the plurality of parallax images, a three-dimensional image Can be reproduced continuously, and the fatigue of the image observer can be reduced. In addition, since a plurality of parallax images are generated by shifting a two-dimensional image uniformly in the left-right direction by a predetermined amount of parallax, many calculations for converting a two-dimensional image into a three-dimensional image by calculation are not required. Thus, a plurality of parallax images can be generated easily and instantaneously. Furthermore, the change in parallax when switching between the display of the three-dimensional image and the display of the parallax image generated from the two-dimensional image is further reduced, and fatigue can be further reduced.

  The three-dimensional image display device according to claim 1, wherein the parallax image generation unit includes a right-eye image obtained by shifting the two-dimensional image to the left by a half of the predetermined parallax amount. And generating a left-eye image obtained by shifting the two-dimensional image to the right by a half of the predetermined amount of parallax. By adding parallax to the 2D image in this way, the 2D image can be projected from the display screen uniformly.

As shown in claim 3, in the three-dimensional image display device according to claim 1 or 2, the parallax image generation means includes a two-dimensional predetermined background image and an image obtained by shifting the two-dimensional image in the left-right direction. It is characterized by generating a synthesized image. As a result, the 2D image can be projected out of the background image uniformly.

  The three-dimensional image display device according to claim 3, wherein the two-dimensional image is a color image, and the predetermined background image is a monochrome image of the two-dimensional image. . Thereby, a color 2D image can be projected evenly from a monochrome image of the 2D image.

  According to a fifth aspect of the present invention, in the three-dimensional image display device according to the third or fourth aspect, the predetermined background image is a negative image obtained by inverting the negative and positive of the two-dimensional image. When the two-dimensional image is a color image, the negative image may be a color image obtained by negative / positive inversion, or a color image monochrome image obtained by negative / positive inversion.

  The three-dimensional image display device according to claim 1 or 2, wherein the parallax image generation unit shifts in the left-right direction with a plurality of frame images for three-dimensional display prepared in advance. A composite image is generated by combining a plurality of two-dimensional images. Thereby, while being able to display a frame image as a favorable 3D image, it is possible to show a 2D image that has a sense of perspective by being inserted into the frame image.

In the three-dimensional image display apparatus according to claim 1 as illustrated in claim 7, wherein the recording medium includes a plurality of individual images for three-dimensional display, a accessory information attached to each individual image, the representative A three-dimensional image is recorded as a three-dimensional image file in which auxiliary information including the amount of parallax is stored, and the parallax amount acquisition unit acquires the representative amount of parallax included in the auxiliary information from the three-dimensional image file. It is a feature.

In the three-dimensional image display apparatus according to claim 1 as shown in claim 8, wherein the recording medium includes a plurality of individual images for three-dimensional display, a accessory information attached to each of the individual images, each of A three-dimensional image is recorded as a three-dimensional image file in which accessory information including coordinate values of feature points whose features match between the individual images is stored, and the parallax amount acquisition unit is configured to acquire the accessory information from the three-dimensional image file. The feature is that the coordinate value of the feature point of each individual image included in the information is read, and the amount of parallax is acquired based on the difference of the coordinate value of the feature point of each individual image.

According to the invention according to claim 7 or 8 , it is possible to easily acquire the parallax amount of the three-dimensional image in the vicinity of the two-dimensional image, and generate a plurality of parallax images from the two-dimensional image with the acquired parallax amount. The amount of parallax at the time.

The three-dimensional image display method according to claim 9 is a two-dimensional image or a three-dimensional image recorded from a recording medium in which a two-dimensional image and a three-dimensional image composed of a plurality of images taken from the same subject are recorded together. An image input step for sequentially inputting images, a determination step for determining whether the image input by the image input step is a two-dimensional image or a three-dimensional image, and a determination that a two-dimensional image has been input by the determination step A parallax image generating step for generating a parallax image for three-dimensional display based on the input two-dimensional image, wherein the two-dimensional image is uniformly shifted in the left-right direction by a predetermined amount of parallax. A parallax image generating step for generating a parallax image; and when the input image is determined to be a two-dimensional image, the three-dimensional image is displayed on a three-dimensional image display means, and the input When the image is determined to 2-dimensional images, a three-dimensional image display step of displaying a plurality of parallax images produced by the parallax image generating step on the basis of the two-dimensional image to the three-dimensional image display means, the recording medium A parallax amount acquisition step of acquiring information on a parallax amount of a three-dimensional image in the vicinity of the two-dimensional image recorded in the two-dimensional image, wherein the parallax image generation step is for three-dimensional display based on the two-dimensional image. As the predetermined amount of parallax when generating the parallax image, the amount of parallax of the three-dimensional image in the vicinity of the two-dimensional image acquired by the parallax amount acquisition step is used .

  According to the present invention, even when a 2D image and a 3D image are mixedly input sequentially, when a 2D image is input, a parallax image is generated from the 2D image and reproduced. It is possible to continuously reproduce only a three-dimensional image, thereby reducing the fatigue of the image observer. In particular, when generating a parallax image from a 2D image, a two-dimensional image is converted into a predetermined parallax. Since the plurality of parallax images are generated by shifting the amount uniformly in the left-right direction, the parallax images can be generated and reproduced easily and instantaneously.

FIG. 1 is an external view showing a three-dimensional image display apparatus according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an internal configuration of the three-dimensional image display apparatus shown in FIG. FIG. 3 is a diagram showing a directory structure of a memory card in which image files are recorded. FIG. 4 is a diagram showing a procedure for generating a parallax image according to the present invention from a 2D image. FIG. 5 is a rear view of a digital camera provided with the three-dimensional image display device according to the second embodiment of the present invention, and shows a state in which the rear LCD displays a parallax image. FIG. 6 is a block diagram showing an internal configuration of the digital camera shown in FIG. FIG. 7 is a rear view of the digital camera, and shows a state in which the rear LCD displays an index image. FIG. 8 is a rear view of the digital camera, and shows a state in which the rear LCD displays a parallax image including a frame image. FIG. 9 is a diagram illustrating a data structure of a 3D image file for 3D images. FIG. 10 is a diagram illustrating an example of parallax amount metadata in a 3D image file for 3D images. FIG. 11 is a diagram illustrating an example of assigning a plurality of imaging units and viewpoint numbers. FIG. 12 is a chart showing an example of the representative parallax amount of each viewpoint. FIG. 13 is a diagram illustrating an example of feature point metadata in a 3D image file for 3D images. FIG. 14 is a diagram illustrating an example of feature points in an image. FIG. 15 is a diagram illustrating an example of feature point information of each viewpoint. FIG. 16 is a flowchart showing the first embodiment of the three-dimensional image display method according to the present invention. FIG. 17 is a flowchart showing a second embodiment of the three-dimensional image display method according to the present invention. FIG. 18 is a flowchart showing a third embodiment of the three-dimensional image display method according to the present invention.

  Embodiments of a 3D image display apparatus and method according to the present invention will be described below with reference to the accompanying drawings.

[First embodiment of three-dimensional image display apparatus]
FIG. 1 is an external view showing a three-dimensional image display apparatus according to a first embodiment of the present invention.

  As shown in FIG. 1, this three-dimensional image display device (3D image display device) 10 is a digital photo frame equipped with a color three-dimensional liquid crystal display (hereinafter referred to as “3D LCD”) 12. An operation unit 14 for selecting a power switch, normal display / slide show, and the like is provided, and a memory card slot 16 is provided on a side surface.

  FIG. 2 is a block diagram showing an internal configuration of the 3D image display apparatus 10.

  As shown in FIG. 2, the 3D image display apparatus 10 includes a central processing unit (CPU) 20, a work memory 22, a card interface (card I / F) 24, a display controller 26, in addition to the 3D LCD 12 and the operation unit 14. A buffer memory 28, an EEPROM 30, and a power supply unit 32 are provided.

  The 3D LCD 12 displays a plurality of images (right-eye image, left-eye image) with a directional image having a predetermined directivity by a lenticular lens, a parallax barrier, etc., polarizing glasses, liquid crystal shutter glasses, etc. Applicable are those that allow the user to see the right-eye image and the left-eye image individually by wearing dedicated glasses.

  The CPU 20 functions as a control unit that performs overall control of the overall operation of the 3D image display apparatus 10 according to a predetermined control program based on an input from the operation unit 14. The contents of control by the CPU 20 will be described later.

  The work memory 22 includes a calculation work area for the CPU 20 and a temporary storage area for image data.

  The card I / F 24 is electrically connected to the memory card 34 when a memory card 34 that is a recording medium of the digital camera is inserted into the memory card slot 16, and data (image data) is exchanged with the memory card 34. It is an apparatus for performing transmission and reception.

  The display controller 26 repeatedly reads out 3D display image data (a plurality of image data) from the buffer memory 28, which is a temporary storage area dedicated to display image data, and converts it into a 3D display signal on the 3D LCD 12. Output to the 3D LCD 12. As a result, a 3D image is displayed on the 3D LCD 12.

  The power supply unit 32 controls power from a battery (not shown) or a commercial power supply and supplies operating power to each unit of the 3D image display device 10.

<Memory card directory structure>
FIG. 3 shows an example of the directory structure of the memory card 34 mounted on the 3D image display device 10.

  As shown in FIG. 3, a 2D image or a 3D image file is arranged in a folder (100_PICT) in a hierarchy immediately below the image root directory DCIM (Digital Camera IMages).

  Each image file is given a file name based on a predetermined naming rule, and in this embodiment, the file name is given by four alphabetic characters (free characters) + four digit file number. The file number is automatically assigned a sequential number of the maximum number + 1 in the order of image file creation.

  The extension of the image file is different between the 2D image and the 3D image. In this embodiment, the extension of the image file of the 2D image is “JPG” representing the JPEG image, and the extension of the image file of the 3D image. The extension is “F3D” representing one 3D display multi-picture file (3D image file) created by connecting a plurality of JPEG images. Details of the data structure of the 3D image file for 3D display will be described later.

<Operation of 3D Image Display Device 10>
When the power switch of the operation unit 14 is turned on and slide show playback is set as the playback mode, the CPU 20 starts from the memory card 34 installed in the memory card slot 16 in the order of file numbers via the card I / F 24. Read the image file at predetermined intervals.

  The CPU 20 determines whether the image stored in the read image file is a 2D image or a 3D image based on the extension of the image file. When the image stored in the image file is determined to be a 2D image (extension “JPG” is detected), the CPU 20 generates a parallax image for 3D display based on the 2D image.

  FIG. 4 shows an example of generating a parallax image from a 2D image.

  As shown in FIG. 4, first, a background image is generated from a 2D image. In this embodiment, a monochrome image is generated from a color 2D image and used as a background image.

  Then, two parallax images (a right-eye image and a left-eye image) are generated based on the color 2D image and a predetermined parallax amount set in advance. The parallax image for the right eye is superimposed on the background image by shifting the 2D image to the left by a half of a predetermined parallax amount, while the parallax image for the left eye is superimposed on the background image. The 2D image is generated by shifting and synthesizing the 2D image to the right by a half of a predetermined amount of parallax. Note that the predetermined amount of parallax used here is an eigenvalue with which the amount of projection of the parallax image is a standard amount on the 3D LCD 12. Further, the predetermined amount of parallax may be set according to the preference of the observer.

  The CPU 20 temporarily stores the parallax image for the right eye and the parallax image for the left eye generated from the 2D image as described above in the buffer memory 28, and the display controller 26 reads the two parallax images from the buffer memory 28, The signal is converted into a 3D display signal and output to the 3D LCD 12. Thereby, a 3D image (left and right parallax images) is displayed on the 3D LCD 12.

  Note that a parallax image generated from a 2D image with uniform parallax as described above appears to jump out of the background image by an amount corresponding to the parallax amount, but the 2D image itself is an image that looks planar. .

  On the other hand, when the CPU 20 determines that the image stored in the image file is a 3D image (detects the extension “F3D”), the image for the right eye and the image for the left eye taken from different viewpoints from the image file of the 3D image. Is taken out and temporarily stored in the buffer memory 28. The display controller 26 reads the right-eye image and the left-eye image from the buffer memory 28, converts them into 3D display signals, and outputs them to the 3D LCD 12. As a result, a 3D image is displayed on the 3D LCD 12.

  As described above, when the images are sequentially read out from the memory card 34 in which the 2D image and the 3D image are recorded and reproduced, and the read image is a 2D image, a predetermined amount of parallax from the 2D image is obtained. Since a plurality of parallax images that are shifted in the horizontal direction uniformly are generated and these parallax images are displayed, only a stereoscopic image can be continuously reproduced, and an image observer Can reduce fatigue.

[Second Embodiment of Three-Dimensional Image Display Device]
FIG. 5 is a rear view of a digital camera provided with the three-dimensional image display device according to the second embodiment of the present invention, and shows a state in which a 3D LCD 150 on the rear surface displays a parallax image generated from a 2D image.

  As shown in FIG. 5, a 3D LCD 150, an optical viewfinder eyepiece 104, and an operation unit 116 are provided on the back of the digital camera 100.

  FIG. 6 is a block diagram showing an internal configuration of the digital camera 100 shown in FIG.

  As shown in FIG. 6, the digital camera 100 is a compound eye camera including two photographing units 112R and 112L. Note that there may be two or more photographing units 112.

  The digital camera 100 can record a 3D image including a plurality of images captured by the plurality of imaging units 112R and 112L as one 3D display 3D image file.

  A main CPU 114 (hereinafter referred to as “CPU 114”) controls the overall operation of the digital camera 100 according to a predetermined control program based on an input from the operation unit 116.

  A ROM 124, an EEPROM 126, and a work memory 128 are connected to the CPU 114 via a system bus 122. The ROM 124 stores a control program executed by the CPU 114 and various data necessary for control. The EEPROM 126 stores various setting information relating to the operation of the digital camera 100 such as user setting information. The work memory 128 includes a calculation work area for the CPU 114 and a temporary storage area for image data.

  The operation unit 116 is a means for a user to perform various operation inputs, and includes a power / mode switch, a mode dial, a release switch, a cross key, a zoom button, a MENU / OK button, a DISP button, and a BACK button. . The operation display unit 118 is a means for displaying an operation input result from the operation unit 116, and includes, for example, a liquid crystal panel or a light emitting diode (LED).

  The power / mode switch is means for switching on / off the power of the digital camera 100 and switching the operation mode (playback mode and photographing mode) of the digital camera 100. When the power / mode switch is turned on, power supply from the power supply unit 120 to each unit of the digital camera 100 is started, and various operations of the digital camera 100 are started. When the power / mode switch is turned off, the supply of power from the power supply unit 120 to each unit of the digital camera 100 is stopped.

  The mode dial is an operating means for switching the shooting mode of the digital camera 100. The 2D still image shooting mode for shooting a 2D still image according to the setting position of the mode dial, and the 2D movie shooting for shooting a 2D movie. The shooting mode is switched between a mode, a 3D still image shooting mode for shooting a 3D still image, and a 3D moving image shooting mode for shooting a 3D moving image. When the shooting mode is set to the 2D still image shooting mode or the 2D moving image shooting mode, a flag indicating that the 2D mode for shooting a 2D image is set in the shooting mode management flag 130. When the shooting mode is set to the 3D still image shooting mode or the 3D moving image shooting mode, the shooting mode management flag 130 is set to indicate a 3D mode for shooting a 3D image. The CPU 114 refers to the shooting mode management flag 130 to determine the shooting mode setting.

  The release switch is composed of a two-stage stroke type switch composed of so-called “half-press” and “full-press”. When the release switch is pressed halfway in still image shooting mode, shooting preparation processing (for example, AE (Automatic Exposure) processing, AF (Auto Focus) processing, AWB (Automatic White Balance)) (Balance) processing) is performed, and when the release switch is fully pressed, still image shooting / recording processing is performed. In the moving image shooting mode, moving image shooting starts when the release switch is fully pressed, and moving image shooting ends when the release switch is fully pressed again. Note that a release switch for still image shooting and a release switch for moving image shooting may be provided separately.

  The 3D LCD 150 is a 3D image display similar to the 3D LCD 12 of the 3D image display device 10 illustrated in FIG. 1, and functions as an image display unit for displaying a captured 2D image or 3D image, and at the time of various settings. It functions as a GUI. The 3D LCD 150 functions as an electronic viewfinder for confirming the angle of view in the shooting mode.

  The vertical / horizontal shooting detection circuit 132 includes, for example, a sensor for detecting the orientation of the digital camera 100, and inputs the detection result of the orientation of the digital camera 100 to the CPU 114. The CPU 114 switches between vertical shooting and horizontal shooting when the digital camera 100 is oriented.

  Next, the shooting function of the digital camera 100 will be described. In FIG. 6, the respective parts in the photographing units 112R and 112L are distinguished from each other by adding reference signs R and L, but the functions of the respective parts are substantially the same. The description is omitted.

  The photographing lens 160 includes a zoom lens, a focus lens, and a diaphragm. The zoom lens and the focus lens move back and forth along the optical axis (LR and LL in the drawing) of each photographing unit. The CPU 114 performs zooming by controlling the position of the zoom lens by controlling the driving of a zoom actuator (not shown) via the photometry / ranging CPU 180, and controls the driving of the focus actuator via the photometry / ranging CPU 180. Thus, focusing is performed by controlling the position of the focus lens. Further, the CPU 114 controls the aperture amount (aperture value) of the diaphragm by controlling the driving of the diaphragm actuator via the photometry / ranging CPU 180, and controls the amount of light incident on the image sensor 162.

  When shooting a plurality of images in the 3D mode, the CPU 114 drives the shooting lenses 160R and 160L of the shooting units 112R and 112L in synchronization. That is, the photographing lenses 160R and 160L are always set to the same focal length (zoom magnification). In addition, the aperture is adjusted so that the same incident light amount (aperture value) is always obtained. Further, in the 3D mode, focus adjustment is performed so that the same subject is always in focus.

  The flash light emitting unit 176 is constituted by, for example, a discharge tube (xenon tube), and emits light as necessary when photographing a dark subject or in backlight. The charge / light emission control unit 178 includes a main capacitor for supplying a current for causing the flash light emission unit 176 to emit light. The CPU 114 transmits a flash emission command to the photometry / ranging CPU 180 to perform charge control of the main capacitor, discharge (light emission) timing of the flash light emission unit 176, control of the discharge time, and the like. Note that a light emitting diode may be used as the flash light emitting unit 176.

  The imaging unit 112 captures a distance light emitting element 186 (for example, a light emitting diode) for irradiating the subject with light and an image of the subject irradiated with light by the distance light emitting element 186 (ranging image). A distance image sensor 184.

  The photometry / ranging CPU 180 causes the distance light emitting element 186 to emit light at a predetermined timing based on a command from the CPU 114 and controls the distance imaging element 184 to photograph a distance measuring image.

  The distance measurement image captured by the distance image sensor 184 is converted into digital data by the A / D converter 196 and input to the distance information processing circuit 198.

  The distance information processing circuit 198 uses the distance measurement image acquired from the distance image sensor 184 to connect between the subject photographed by the photographing units 112R and 112L and the digital camera 100 based on the principle of so-called triangulation. The distance (subject distance) is calculated. The subject distance calculated by the distance information processing circuit 198 is recorded in the distance information storage circuit 103.

  As a method for calculating the subject distance, the light flight time from when the distance light emitting element 186 emits light until the light emitted by the distance light emitting element 186 is reflected by the subject and reaches the distance imaging element 184 is used. A TOF (Time of Flight) method for calculating the subject distance from the (delay time) and the speed of light may be used.

  The photographing unit 112 includes an interval / convergence angle driving circuit 188 and an interval / convergence angle detection circuit 190.

  The interval / convergence angle driving circuits 188R and 188L drive the imaging units 112R and 112L, respectively. The CPU 114 operates the interval / convergence angle driving circuits 188R and 188L via the interval / convergence angle control circuit 192 to adjust the interval and the convergence angle between the photographing lenses 160R and 160L.

  The interval / convergence angle detection circuits 190R and 190L include means for transmitting and receiving radio waves, for example. The CPU 114 operates the interval / convergence angle detection circuits 190R and 190L via the interval / convergence angle control circuit 92 to transmit and receive radio waves, thereby measuring the interval and the convergence angle between the photographing lenses 160R and 160L. . The measurement results of the interval between the photographing lenses 160R and 160L and the convergence angle are stored in the lens interval / convergence angle storage circuit 102.

  The image sensor 162 is constituted by a color CCD solid-state image sensor, for example. A large number of photodiodes are two-dimensionally arranged on the light receiving surface of the image sensor 162, and color filters of three primary colors (R, G, B) are arranged in a predetermined arrangement in each photodiode. The optical image of the subject formed on the light receiving surface of the image sensor 62 by the photographing lens 160 is converted into signal charges corresponding to the amount of incident light by the photodiode. The signal charge accumulated in each photodiode is sequentially read out from the image sensor 162 as a voltage signal (R, G, B signal) corresponding to the signal charge based on a drive pulse given from the TG 164 according to a command from the CPU 114. The image sensor 162 has an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.

  As the image sensor 162, an image sensor other than a CCD such as a CMOS sensor can be used.

  The analog signal processing unit 166 outputs a correlated double sampling circuit (CDS), R, G, and B signals for removing reset noise (low frequency) included in the R, G, and B signals output from the image sensor 162. An AGS circuit for amplifying and controlling to a certain level is included. Analog R, G, and B signals output from the image sensor 162 are subjected to correlated double sampling processing and amplified by the analog signal processing unit 166. The analog R, G, and B signals output from the analog signal processing unit 166 are converted into digital R, G, and B signals by the A / D converter 168 and input to the image input controller (buffer memory) 170. The

  The digital signal processing unit 172 includes a synchronization circuit (a processing circuit that interpolates a spatial shift of the color signal associated with the color filter arrangement of the single-plate CCD and converts the color signal into a simultaneous expression), a white balance adjustment circuit, a gradation A conversion processing circuit (gamma correction circuit), a contour correction circuit, a luminance / color difference signal generation circuit, and the like are included. Digital R, G, and B signals input to the image input controller 170 are subjected to predetermined processing such as synchronization processing, white balance adjustment, gradation conversion, and contour correction by the digital signal processing unit 172, It is converted into a Y / C signal composed of a luminance signal (Y signal) and a color difference signal (Cr, Cb signal).

  When the live view image (through image) is displayed on the 3D LCD 150, the Y / C signal generated in the digital signal processing unit 172 is sequentially supplied to the buffer memory 144. The display controller 142 reads the Y / C signal supplied to the buffer memory 144 and outputs it to the YC-RGB conversion unit 146. The YC-RGB conversion unit 146 converts the Y / C signal input from the display controller 142 into R, G, and B signals and outputs them to the 3D LCD 150 via the driver 148. Thereby, the through image is displayed on the 3D LCD 150.

  Here, when the camera mode is the shooting mode and in the 2D mode, a recording image is shot by a predetermined one shooting unit (for example, 112R). In the 2D mode, an image photographed by the photographing unit 112R is compressed by the compression / decompression processing unit 174R. The compressed image data is recorded on the memory card 34 as an image file in a predetermined format via the memory controller 134 and the card I / F 138. For example, JPEG (Joint Photographic Experts Group) for still images, MPEG2 or MPEG4 for moving images, and H.264. It is recorded as a compressed image file conforming to the H.264 standard.

  The camera mode is a shooting mode, and in the 3D mode, images are shot in synchronization by the shooting units 112R and 112L. In the 3D mode, AF processing and AE processing are performed based on the image signal acquired by one of the imaging units 112R and 112L. In the 3D mode, the two viewpoint images captured by the imaging units 112R and 112L are respectively compressed by the compression / decompression processing units 174R and 174L, stored in one 3D image file, and recorded on the memory card 34. The 3D image file stores subject distance information, information on the distance between the taking lenses 160R and 160L, information on the convergence angle, and the like, as well as compressed image data of two viewpoints.

  On the other hand, when the operation mode of the camera is the playback mode, the last image file (last recorded image file) recorded on the memory card 34 is read out, and the compression / decompression processing unit 174 performs uncompressed Y / After being expanded to the C signal, it is input to the buffer memory 144. The display controller 142 reads the Y / C signal supplied to the buffer memory 144 and outputs it to the YC-RGB conversion unit 146. The YC-RGB conversion unit 146 converts the Y / C signal input from the display controller 142 into R, G, and B signals and outputs them to the 3D LCD 150 via the driver 148. As a result, the image of the image file recorded on the memory card 34 is displayed on the 3D LCD 150.

  Here, when the image file read from the memory card 34 is an image file of a 2D image, similarly to the 3D image display device 10 described above, based on the 2D image and a predetermined amount of parallax set in advance. Thus, two parallax images (right-eye image and left-eye image) are generated (see FIG. 4), and the two parallax images are displayed on the 3D LCD 150 as shown in FIG.

  On the other hand, when the image file read from the memory card 34 is a 3D image file, a right eye image and a left eye image taken from different viewpoints are extracted from the 3D image file, and the 3D LCD 150 is extracted. Displayed as a 3D image.

  Further, by operating the DISP button of the operation unit 116 in the playback mode, an index image composed of a plurality of reduced images (thumbnail images) can be displayed on the 3D LCD 150 as shown in FIG. Also in this index image, the thumbnail image of the 2D image is a parallax image with parallax added by the method shown in FIG. 4 and is a thumbnail image that protrudes in the same manner as the thumbnail image of the 3D image.

<Another Example of Parallax Image Generated from 2D Image>
FIG. 8 is a rear view of the digital camera 100 and shows a state in which a parallax image including a frame image is displayed on the 3D LCD 150.

  The method shown in FIG. 4 uses a monochrome image of a 2D image as a background image, combines a 2D image that is shifted by a predetermined amount of parallax (half the amount of parallax) in the horizontal direction with respect to the background image, In this embodiment, a frame image for 3D display is used instead of the background image, and a predetermined amount of parallax (1/2) is applied to the frame image in the left-right direction. 2D images shifted by each other) are combined to generate two parallax images.

  The two frame images for 3D display are created so that, for example, the innermost periphery of the frame has a predetermined amount of parallax and the amount of parallax gradually decreases toward the outer periphery of the frame, and is stored in the EEPROM 126 in advance. What is being used is used. Then, a 2D image shifted in the left-right direction by a predetermined amount of parallax is synthesized inside the frame of the two frame images.

  The parallax image combined with the frame image in this way is an image in which a 2D image is projected in the shape of a truncated pyramid.

  As two frame images for 3D display, the amount of parallax on the outermost periphery of the frame becomes a predetermined amount of parallax, the amount of parallax gradually decreases toward the inner periphery of the frame, and the amount of parallax on the innermost periphery of the frame You may use what was made so that becomes zero. When this frame image is used, the 2D image is synthesized with zero parallax (without parallax).

  Further, as another example of the parallax image generated from the 2D image, a single color background image is used instead of a monochrome 2D image as a background image, a color 2D image or a monochrome 2D image. It is possible to use a negative image obtained by reversing negative and positive, or a image obtained by shifting only a 2D image in the horizontal direction without using a background image.

  When the background image is not used, the 2D image appears to pop out from the back of the digital camera 100 (the surface of the 3D LCD 150). The monochrome image includes a monochrome image with a reduced number of gradations (a binary image with monochrome two gradations).

[Data structure of 3D image file]
FIG. 9 is a diagram showing an example of the data structure of a 3D image file for 3D display recorded on the memory card 34 by the digital camera 100.

  As shown in FIG. 9, the 3D image file F uses the Exif standard file format and adopts a file format in which a plurality of images are integrated and recorded. Individual image (1) (first image) A1, individual image (2) A2, individual images (2) A2,..., Individual images (n) An are connected.

  Each individual image area is divided into an SOI (Start of Image) indicating the start position of each individual image and an EOI (End of Image) indicating the end position. Exif attached information of the individual image follows the SOI. An APP1 marker segment to be recorded and an APP2 marker segment in which multi-viewpoint accessory information is recorded are provided, and then a viewpoint image (individual image) is recorded.

  The APP1 marker segment is provided with Exif identification information, a TIFF header, and an IFD (Image file directory) area (IFD0 area (0th IFD) and IFD1 area (1st IFD)). A thumbnail image generated from an individual image is stored in the IFD1 area (1st IFD). The APP2 marker segment includes individual information IFD.

  The individual information IFD includes the number of individual images, the individual image number, the convergence angle, the baseline length, and the like. In this embodiment, as shown in FIG. 10, the tag value of the reference viewpoint number and the representative parallax amount A tag value is further recorded.

  The representative parallax amount is a value representative of the parallax between the reference viewpoint (“1” in the example of FIG. 10) and the viewpoint (i) of the individual image, and the parallax at the time of AF focusing, the closest position parallax, the face Any parameter representing the parallax at the center of the recognition position can be used.

  FIG. 11 is a diagram showing an example of assigning a plurality of imaging units and viewpoint numbers. In FIG. 11, viewpoint numbers are assigned 1, 2, 3, and 4 from the left side.

  FIG. 12 is a chart showing an example of the representative parallax amount of each viewpoint in the case of the four viewpoints shown in FIG.

  As shown in FIG. 13, the individual information IFD of the 3D image file further records the coordinate values of the feature points 1, 2, 3,..., K in the individual image.

  As shown in FIG. 14, the feature point is a point having a feature that can be uniquely specified in an individual image, and the same feature point can be specified with another individual image.

  As shown in FIG. 15, the coordinate values of the same feature points l and m of four individual images taken from four viewpoints are different from each other. In the example shown in FIG. 15, the feature point l has a larger viewpoint number. As the point becomes (as the viewpoint position moves to the right as shown in FIG. 11), the x coordinate value increases, and the feature point m decreases as the viewpoint number increases.

  This is because the subject including the feature point l is a distant view at a position farther from the position where the optical axes of the imaging units intersect, and the subject including the feature point m is than the position where the optical axes of the imaging units intersect. Is also based on the close-up view at a close position.

  Various methods have been proposed to detect the above feature points. For example, the block matching method, KLT method (Tomasi & Kanade, 1991, Detection and Tracking of Point Features), SIFT (Scale Invariant Feature Transform), etc. In addition, recent face detection techniques can be applied to feature point detection.

<Another Example of Setting of Parallax Amount when Generating Parallax Image from 2D Image>
In the above-described embodiment, a device-fixed value (standard parallax amount) is used as a predetermined parallax amount when generating a parallax image from a 2D image. However, the present invention is not limited to this, and a 3D image in the vicinity of a 2D image file is used. A representative parallax amount of a 3D image of a file (a 3D image file immediately before or immediately after a 2D image file whose file number is close to that of the 2D image file) or a parallax amount calculated from the coordinates of corresponding feature points of a plurality of images. It may be used.

  If the image file read from the memory card 34 is a 2D image file, the representative parallax amount is read from the APP2 marker segment in which the multi-viewpoint attached information of the 3D image file near the 2D image file is recorded (FIG. 10). (Refer to FIG. 15), or the coordinate value of a predetermined feature point (for example, the feature point m included in the foreground in FIG. 14) is read from the APP2 marker segment, and the difference value of the x coordinate is calculated as the amount of parallax. The right-eye parallax image and the right-eye parallax image are generated.

Thereby, the change of the parallax at the time of switching between the display of the 3D image and the display of the parallax image generated from the 2D image is further reduced, and the fatigue of the image observer can be further reduced.
[First Embodiment of 3D Image Display Method]
FIG. 16 is a flowchart showing the first embodiment of the 3D image display method according to the present invention.

  First, an initial image is selected in the 3D image display apparatus 10 (or the digital camera 100 set to the reproduction mode) shown in FIG. 1 (step S10). The initial image can be selected by displaying an index image on the 3D LCD (see FIG. 7) and selecting a desired image on the index image. As the initial image, an image with the smallest file number recorded on the memory card 34, an image with the largest file number, or the last image reproduced last time may be set.

  Subsequently, the selected image file is input from the memory card 34 (step S12), and it is determined whether the input image file is a 2D image file or a 3D image file (step S14). The 2D / 3D determination can be made based on the extension of the image file, but may be determined based on the presence / absence of the multi-viewpoint attached information shown in FIG.

  If the input image file is a 2D image file, two parallax images for the right eye and the left eye are generated based on the 2D image recorded in the image file and a predetermined parallax amount set in advance. (Step S16), these parallax images are displayed on the 3D LCD (Step S18).

  On the other hand, if the input image file is a 3D image file, the individual image for the right eye and the individual image for the left eye are taken out from the 3D image file, and these individual images are displayed on the 3D LCD (step S18).

  Subsequently, the presence / absence of key input for the frame advance (front) key and the frame return (rear) key is determined (step S20). It is determined whether or not the key is operated (step S22).

  If it is determined that the frame advance (previous) key has been operated, the image recorded next to the currently displayed image (the image file with the next file number) is selected (step S24), and the process proceeds to step S12. Here, the next image file is input.

  On the other hand, if it is determined that the frame rewind (rear) key has been operated, the image recorded before the currently displayed image (the image file with the previous file number) is selected (step S26), and the switch The process proceeds to S12, where the previous image file is input.

  In this way, image files are recorded in ascending or descending order of file numbers from the memory card 34 in which 2D image files and 3D image files are recorded together by key operation of the frame advance (previous) key and frame reverse (rear) key. Are sequentially read out, and the parallax image generated from the 2D image or the individual image of the 3D image can be sequentially reproduced on the 3D LCD.

  In the first embodiment, the case where the frame advance (rear) key and the frame return (rear) key are sequentially operated for frame advance playback has been described. However, a plurality of images recorded on the memory card 34 are stored in a predetermined manner. It can also be applied to slide show playback, which automatically plays in order at intervals.

[Second Embodiment of 3D Image Display Method]
FIG. 17 is a flowchart showing a second embodiment of the 3D image display method according to the present invention. Steps common to the flowchart shown in FIG. 16 are denoted by the same step numbers, and detailed description thereof is omitted.

  If it is determined in step S14 in FIG. 17 that the input image file is a 3D image file, the number of individual images (number of file viewpoints (F)) recorded in the 3D image file and the 3D LCD are simultaneously displayed. The number of images to be displayed (display unit viewpoint number (D)) is compared.

  That is, the number of viewpoints (F) of a 3D image recorded in a 3D image file and the number of viewpoints (D) required for 3D display on a 3D LCD do not always match. Whereas a viewpoint image is required, a 3D image file contains four viewpoints of individual images, or conversely, a 3D LCD requires a four viewpoint image. On the other hand, there may be a case where two viewpoints of individual images are recorded in the 3D image file.

  Therefore, the number of file viewpoints (F) is compared with the number of display part viewpoints (D). If the number of display part viewpoints (D) is smaller than the number of file viewpoints (F) (when D <F), the file Individual images corresponding to the number of viewpoints (D) of the display unit are selected from the individual images having the number of viewpoints (F) (step S32).

  On the other hand, when the number of display part viewpoints (D) is larger than the number of file viewpoints (F) (when D> F), the individual image of the number of file viewpoints (F) is compared with the number of display part viewpoints (D). An insufficient image is generated (step S34). For example, when generating one intermediate image between two individual images from two individual images, the amount of parallax of each feature point (corresponding point) of the two individual images is interpolated for each feature point, and new The coordinate values of a plurality of feature points on the image to be generated are calculated, and the individual images are geometrically deformed from the coordinate values of the feature points of one individual image and the coordinate values of the feature points calculated by the interpolation. The geometric deformation parameter is obtained, and an individual image is geometrically deformed based on the geometric deformation parameter to generate an interpolated image. As the geometric deformation parameter, a projective transformation parameter when performing the projective transformation, a Helmart transformation parameter when performing the Helmart transformation, and the like can be considered.

  When the number of file viewpoints (F) and the number of display viewpoints (D) match (when D = F), all individual images recorded in the 3D image file are used as images for 3D display. The

  In step S16 for generating a parallax image from a 2D image, parallax images for the number of viewpoints (D) of the display unit of the 3D LCD are generated.

  Thereby, even if the number of file viewpoints (F) of the 3D image file recorded in the memory card 34 and the number of viewpoints (D) of the display part of the 3D LCD do not match, it is possible to display the 3D image properly. .

[Third Embodiment of 3D Image Display Method]
FIG. 18 is a flowchart showing a third embodiment of the 3D image display method according to the present invention. Steps common to the flowchart shown in FIG. 16 are denoted by the same step numbers, and detailed description thereof is omitted.

  If it is determined in step S14 of FIG. 18 that the input image file is a 2D image file, a 3D image file having a file number close to the file number of the 2D image file is selected (step S40).

  Subsequently, the metadata (multi-viewpoint attached information) of the selected 3D image file is read, and the representative parallax amount recorded as the metadata is determined as the parallax amount for the 2D image (steps S42 and S44). ). Then, a parallax image is generated by shifting the 2D image in the left-right direction based on the determined parallax amount (step S46).

  The parallax amount acquired from the 3D image file close to the 2D image file includes a representative value of the parallax amount between the feature points (the maximum parallax amount between the feature points, the positive parallax amount (the parallax in the protruding direction). An average value of the amount) may be determined as the parallax amount.

[Others]
The frame image for 3D display shown in FIG. 8 has a predetermined amount of parallax in advance, but a plurality of frame images having a plurality of amounts of parallax are stored in an EEPROM (nonvolatile memory). A frame image corresponding to the parallax amount acquired from the image file may be read and the frame image and the 2D image may be combined.

  Further, the 3D display is not limited to the 3D LCD, but may be another 3D display such as a 3D plasma display or a 3D organic EL display.

  Furthermore, the present invention is not limited to 2D still images but can be applied to 2D moving images. That is, according to the present invention, a parallax image is generated simply by shifting a 2D image uniformly. Therefore, a moving image with parallax can be generated from a 2D moving image in real time only by shifting each frame of the moving image.

  Moreover, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 10 ... Three-dimensional image display apparatus (3D image display apparatus) 12, 150 ... Three-dimensional liquid crystal display (3D LCD), 20, 114 ... Central processing unit (CPU), 22, 128 ... Work memory, 26, 142 ... Display Controller, 28, 136 ... Buffer memory, 30, 126 ... EEPROM

Claims (9)

Image input means for sequentially inputting a two-dimensional image or a three-dimensional image from a recording medium in which a two-dimensional image and a three-dimensional image composed of a plurality of images obtained by photographing the same subject from a plurality of viewpoints are mixedly recorded;
Determination means for determining whether the image input from the image input means is a two-dimensional image or a three-dimensional image;
When the determination unit determines that a two-dimensional image is input, the parallax image generation unit generates a parallax image for three-dimensional display based on the input two-dimensional image. A parallax image generating means for generating a plurality of parallax images that are uniformly shifted in the left-right direction by the amount of parallax;
Three-dimensional image display means for displaying a three-dimensional image input from the image input means or a plurality of parallax images generated by the parallax image generation means;
And a parallax amount acquisition means for acquiring a parallax quantity of information of a three-dimensional image of the vicinity of the two-dimensional image recorded on the recording medium,
The parallax image generation means is a three-dimensional image near the two-dimensional image acquired by the parallax amount acquisition means as a predetermined parallax amount when generating a parallax image for three-dimensional display based on the two-dimensional image. 3-dimensional image display device characterized by the use of a parallax amount of an image.   The parallax image generation means shifts the two-dimensional image to the left by a half of the predetermined amount of parallax and the right-eye image by right-shifting the two-dimensional image by a half of the predetermined amount of parallax. The three-dimensional image display device according to claim 1, wherein a left-eye image is generated. 3. The three-dimensional image according to claim 1, wherein the parallax image generation unit generates a composite image obtained by combining a predetermined two-dimensional background image and an image obtained by shifting the two-dimensional image in the left-right direction. Image display device.   The three-dimensional image display device according to claim 3, wherein the two-dimensional image is a color image, and the predetermined background image is a monochrome image of the two-dimensional image.   5. The three-dimensional image display device according to claim 3, wherein the predetermined background image is a negative image obtained by negative-positive reversal of the two-dimensional image. 6.   The parallax image generation means generates a composite image obtained by combining a plurality of frame images for 3D display prepared in advance and a plurality of two-dimensional images shifted in the left-right direction. The three-dimensional image display apparatus according to 1 or 2. The recording medium includes a three-dimensional image as a three-dimensional image file in which a plurality of individual images for three-dimensional display and auxiliary information that is attached to each individual image and includes representative parallax amounts are stored. Recorded,
The three-dimensional image display device according to claim 1 , wherein the parallax amount acquisition unit acquires a representative parallax amount included in the attached information from the three-dimensional image file. The recording medium includes a plurality of individual images for three-dimensional display, and accessory information attached to each individual image, the accessory information including coordinate values of feature points whose features match between the individual images. A 3D image is recorded as a stored 3D image file,
The parallax amount acquisition means reads the coordinate value of the feature point of each individual image included in the attached information from the three-dimensional image file, and acquires the parallax amount based on the difference of the coordinate value of the feature point of each individual image. The three-dimensional image display apparatus according to claim 1 . An image input step of sequentially inputting a two-dimensional image or a three-dimensional image from a recording medium in which a two-dimensional image and a three-dimensional image composed of a plurality of images obtained by photographing the same subject from a plurality of viewpoints are recorded;
A determination step of determining whether the image input in the image input step is a two-dimensional image or a three-dimensional image;
When it is determined in the determination step that a two-dimensional image has been input, a parallax image generation step of generating a parallax image for three-dimensional display based on the input two-dimensional image, wherein the two-dimensional image is predetermined A parallax image generation step for generating a plurality of parallax images that are uniformly shifted in the left-right direction by the amount of parallax;
When the input image is determined to be a two-dimensional image, the three-dimensional image is displayed on a three-dimensional image display means. When the input image is determined to be a two-dimensional image, the two-dimensional image is used as a basis. A three-dimensional image display step for displaying a plurality of parallax images generated by the parallax image generation step on a three-dimensional image display means;
Anda parallax amount acquisition step of acquiring a parallax amount of information of a three-dimensional image of the vicinity of the two-dimensional image recorded on the recording medium,
In the parallax image generation step, as a predetermined parallax amount when generating a parallax image for three-dimensional display based on the two-dimensional image, a three-dimensional image in the vicinity of the two-dimensional image acquired by the parallax amount acquisition step is used. 3-dimensional image display method characterized by using the amount of parallax of the image.

Images