JP2011082757A - Display mode determination device and method, and imaging device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine which one of three-dimensional display and two-dimensional display is suitable for displaying the image data. <P>SOLUTION: A digital camera that is an imaging device, photographs a subject from different points of viewing concurrently. With respect to photographing, strobe photographing and non-strobe photographing are performed in this order. Two sets of images with parallaxes are prepared through respective photographings. From one set of images, a parallax amount ΔP1 of a main subject and a parallax amount ΔP2 of a background are calculated (S18). When a parallax amount difference ΔX (=¾ΔP1-ΔP2¾) is less than a threshold TH1, a difference of images obtained by the strobe photographing and the non-strobe photographing is taken, and a shadow pixel number SS composing a shadow of the subject is calculated. When the shadow pixel number SS is equal to or more than a threshold TH2, it is determined that the image data is suitable for being displaced in a three-dimensional display mode. When the shadow pixel number SS is less than the threshold TH2, it is determined that the image data is suitable for being displayed in a two-dimensional display mode. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an apparatus and method for handling a plurality of images taken from different viewpoints in order to create a three-dimensional image, and more specifically, a display mode determination device for determining whether or not a plurality of images are suitable for a three-dimensional display mode. And an imaging apparatus and method having the same.

  A three-dimensional digital camera that captures a subject at the same time from two different viewpoints using two photographing units and creates two two-dimensional images that can be stereoscopically viewed is known (for example, see Non-Patent Document 1). Two two-dimensional images created by a three-dimensional digital camera have parallax.

  A display device such as an LCD panel provided in a digital photo frame or a three-dimensional digital camera displays two two-dimensional images so as to be separately observed by the left and right eyes of the observer (hereinafter, three-dimensional display). Called mode). This three-dimensional display mode enables stereoscopic viewing. Furthermore, this display device displays only one of two two-dimensional images taken by a three-dimensional digital camera as a conventional two-dimensional image (hereinafter referred to as a two-dimensional display mode). Is also possible.

3D digital camera (http://fujifilm.jp/personal/3d/camera/finepixreal3dw1/index.html)

  The display device described above is required to have a function of displaying in a display mode suitable for the image. In order to meet these demands, development of a method for determining whether or not a captured image obtained by a three-dimensional digital camera is suitable for a three-dimensional display mode (hereinafter referred to as a display mode determination method) has been underway.

  A perspective difference is used as an index of suitability of the three-dimensional display mode. Specifically, the suitability of the three-dimensional display mode increases as the perspective difference for each subject or the perspective difference between the main subject and the background increases. On the other hand, as the perspective difference for each subject or the perspective difference between the main subject and the background becomes smaller, the suitability of the three-dimensional display mode becomes lower. And the difference of each parallax amount obtained by the stereo matching method is set as each perspective difference, whereby the suitability of the image three-dimensional display mode can be easily determined.

  However, when the background is plain as in studio shooting, the amount of parallax of the background becomes a small value. As a result, the calculated difference in the amount of parallax cannot be said to reflect a substantial perspective difference between the main subject and the background.

  In the case of macro photography, focusing on the main subject makes it impossible to focus on the background. As a result, the difference in the amount of parallax obtained may be smaller than the substantial perspective difference between the main subject and the background.

  As described above, there is a limit to performing the display mode determination method using only the difference in the amount of parallax obtained by the stereo matching method as an index.

  The present invention solves such problems, and provides a display mode determination apparatus and method for determining a display mode suitable for each image, and an imaging apparatus and method having the display mode determination apparatus and method. For the purpose.

  The display mode determination apparatus according to the present invention uses a first parallax amount for a first subject and a second for a second subject based on a plurality of image data obtained by simultaneously photographing the subject from different viewpoints. When the difference between the first parallax amount and the second parallax amount is equal to or greater than a first threshold value, the plurality of image data are appropriate for three-dimensional display. And when the difference in the amount of parallax is less than the first threshold, the first display mode determination unit that puts the determination of the plurality of image data on hold, and the difference in the amount of parallax is the first difference. If it is less than the threshold value of 1, the shadow pixel number calculating means for calculating the number of pixels forming the shadow of the subject among the pixels forming the image data, and the calculated number of shadow pixels is the second If the threshold value is exceeded, the plurality of image data are displayed in three dimensions. Second display mode determining means for determining that the image is appropriate and, when the calculated number of shadow pixels is less than the second threshold, determining that the plurality of image data are not appropriate for three-dimensional display; It is characterized by having.

  The shadow pixel number calculating means calculates, for each pixel, the contrast difference of the image data when the subject is irradiated with light when the image data when the subject is not irradiated with light is used as a reference. It is preferable that the total number of pixels having the contrast difference of less than 0 be the number of shadow pixels. Further, an identifier for 3D display is assigned to the plurality of image data determined to be appropriate for 3D display, and 2 is added to the plurality of image data determined to be inappropriate for 3D display by the display mode determination device. It is preferable to have an identifier assigning means for assigning a dimension display identifier. Furthermore, it is preferable to have a display device that displays the plurality of image data including the three-dimensional display identifier as a three-dimensional image and displays the plurality of image data including the two-dimensional display identifier as a two-dimensional image. .

  When the three-dimensional display identifier is given to the plurality of image data, the plurality of image data is stored, and when the two-dimensional display identifier is given to the plurality of image data It is preferable to have a recording means for storing at least one of the plurality of image data. The recording unit preferably stores both the plurality of image data even when the two-dimensional display identifier is assigned to the plurality of image data.

  The imaging device of the present invention is provided with a plurality of optical systems that form an image of the subject so as to be exposed to a digital camera body, an imaging device that captures the combined image as image data, and the display mode determination described above A device is built in the digital camera body, and the light emitting device that emits the light to the subject is disposed at a position sandwiching the second optical system by the first optical system.

  According to the display mode determination method of the present invention, the first parallax amount for the first subject and the second for the second subject are based on a plurality of image data obtained by simultaneously photographing the subject from different viewpoints. If the difference between the parallax amount calculating step for calculating the parallax amount and the difference between the first parallax amount and the second parallax amount is equal to or greater than a first threshold value, the plurality of image data are appropriate for three-dimensional display. And when the difference between the parallax amounts is less than the first threshold, the first display mode determination step for holding the determination of the plurality of image data, and the difference between the parallax amounts If the threshold value is less than 1, the shadow pixel number calculating step for calculating the number of pixels forming the shadow of the subject among the pixels forming the image data; and the calculated number of shadow pixels is a second value. If the threshold value is exceeded, the plurality of image data A second display that determines that the three-dimensional display is appropriate, and determines that the plurality of image data are not appropriate for the three-dimensional display when the calculated number of shadow pixels is less than the second threshold value. And a mode determination step.

  The photographing method of the present invention includes the display mode determination method, an irradiation photographing step of simultaneously photographing the subject irradiated with light from different viewpoints to generate irradiation image data, and after the light irradiation is finished. A non-irradiation photographing step of simultaneously photographing the subject from different viewpoints to generate non-irradiation image data, and the irradiation photographing step and the non-irradiation photographing step are performed before the shadow pixel number calculating step, In the shadow pixel number calculating step, a contrast difference calculating step for calculating, for each pixel, a contrast difference of the irradiation image data when the non-irradiation image data is used as a reference, and the number of pixels in which the contrast difference is less than 0 And a pixel number step which is the number of shadow pixels.

  According to the present invention, the suitability of the three-dimensional display mode of the image data is evaluated based on not only the difference in the amount of parallax of each subject but also the size of the shadow of the subject, so that it can be evaluated with high accuracy. .

It is a perspective view of a digital camera when viewed from the front side. It is a perspective view of a digital camera when viewed from the back side. It is a block diagram which shows the structure of a digital camera. It is a block diagram which shows the structure of a display mode determination part. It is a flowchart which shows the 1st imaging | photography procedure with a digital camera. It is explanatory drawing which shows the outline | summary of the image obtained by strobe imaging | photography of a 1st imaging part. It is explanatory drawing which shows the outline | summary of the image obtained by flash photography of the 2nd imaging part. It is explanatory drawing which shows the outline | summary of the image obtained by the non-strobe imaging | photography of a 1st imaging part. It is explanatory drawing which shows the outline | summary of the image obtained by the non-stroboscopic imaging | photography of a 2nd imaging part. It is explanatory drawing which shows the outline | summary of the difference image of the image of strobe photography, and the image of non-strobe photography. It is explanatory drawing which shows the histogram of the contrast of the pixel which comprises a difference image. It is a flowchart which shows the display procedure of image data. It is a flowchart which shows the 2nd imaging | photography procedure with a digital camera.

  As shown in FIG. 1, the digital camera 11 of the present invention captures the same subject from different viewpoints at the same time by using a first photographing unit 12L and a second photographing unit 12R that are spaced apart in the horizontal direction, and has 2 parallaxes. One image is created. A first photographing unit 12L, a second photographing unit 12R, and a strobe 13 are provided on the front surface of the digital camera body 11a.

  The first photographing unit 12L includes an optical system 14R that forms an image of a subject, an image sensor (described later) that obtains image data from the formed image, and various control units (described later). Similarly, the second imaging unit 12R includes an optical system 14L, an image sensor, and various control units. The optical systems 14R and 14L are provided to be exposed on the front surface of the digital camera body 11a. The image sensor and various control units are built in the digital camera body 11a. The strobe 13 is provided at a position sandwiching the optical system 14R by the optical system 14L. The strobe 13 is preferably provided above at least one of the optical system 14L and the optical system 14R, and more preferably provided above both the optical system 14L and the optical system 14R.

  As shown in FIG. 2, an LCD panel 20 used as a viewfinder at the time of shooting and an image display monitor after shooting, and operation buttons 21 used for various operations of the digital camera 11 are provided on the back of the digital camera body 11a. , 22 are provided. A power / mode switch 27 and a release button 28 are provided on the top surface of the digital camera body 11a. A card slot 30 in which a memory card 29 for recording image data is loaded is provided on one side surface of the digital camera 11.

  The power / mode switch 27 switches on / off the power of the digital camera 11 and switches the operation mode (reproduction mode and photographing mode) of the digital camera 11. When the power / mode switch 27 is turned on, power supply to each part of the digital camera 11 is started, and a predetermined shooting mode can be selected. Various operations of the digital camera 11 are started. When the power / mode switch 27 is turned off, the supply of power to each part of the digital camera 11 is stopped.

  The photographing mode of the digital camera 11 can be switched by operating the power / mode switch 27. Examples of the shooting mode set by operating the power / mode switch 27 include a two-dimensional still image shooting mode for shooting a two-dimensional still image, a three-dimensional still image shooting mode for shooting a three-dimensional still image, and the like. .

  The release button 28 is formed of a two-stage stroke type switch composed of a so-called “half press” and “full press”. When the release button 28 is pressed halfway, shooting preparation processing (for example, AE (Automatic Exposure) processing, AF (Auto Focus) processing, AWB (Automatic White Balance) processing) is performed. Done. When the release button 28 is fully pressed, still image shooting / recording processing is performed.

  By operating the operation buttons 21 and 22, the flash emission mode can be switched between the on state and the off state. When the release button 28 is fully pressed while the flash emission mode is on, the flash 13 emits light. On the other hand, even if the release button 28 is fully pressed in a state where the flash emission mode is off, the flash 13 does not emit light.

  The LCD panel 20 is a parallax barrier type or lenticular lens type three-dimensional monitor, and uses two two-dimensional images created by the first photographing unit 12L and the second photographing unit 12R during photographing and reproduction, An image based on a display mode selected from the three-dimensional display mode and the two-dimensional display mode is displayed.

  As shown in FIG. 3, the digital camera 11 includes a first CPU 12L, a second camera 12R, an operation unit 31, a main CPU (hereinafter referred to as a CPU) 35, and the like. The operation unit 31 is a means for the user to perform various operation inputs, and includes the power / mode switch 27, the release button 28, the operation buttons 21, 22 and the like described above.

  A main CPU (hereinafter referred to as CPU) 35 performs overall control of the entire operation of the digital camera 11 in accordance with a predetermined control program based on an input from the operation unit 31. A ROM 37, EEPROM 38, and work memory 39 are connected to the CPU 35 via a system bus 36. The ROM 37 stores a control program executed by the CPU 35 and various data necessary for control. The EEPROM 38 stores various setting information relating to the operation of the digital camera 11 such as user setting information. The work memory 39 includes a calculation work area for the CPU 35 and a temporary storage area for image data.

  When the two-dimensional still image shooting mode is set as the shooting mode, a flag indicating the two-dimensional still image shooting mode is set in the shooting mode management flag 45. When the three-dimensional still image shooting mode is set as the shooting mode, a flag indicating the three-dimensional still image shooting mode is set in the shooting mode management flag 45. The CPU 35 refers to the shooting mode management flag 45 to determine the set shooting mode.

  Next, details of the first imaging unit 12L and the second imaging unit 12R will be described. In FIG. 3, each of the imaging units 12 </ b> L and 12 </ b> R is distinguished from each other by adding symbols L and R. However, since the functions of the components are substantially the same, the symbols L and R are used in the following description. The description is omitted.

  The imaging unit 12 includes an optical system 14, an image sensor 50, and other components. The optical system 14 includes a zoom lens, a focus lens, and a diaphragm. The zoom lens and the focus lens move back and forth along the optical axis (LL and LR in the drawing) of each photographing unit. The CPU 35 controls the position of the zoom lens and the position of the focus lens by controlling the driving of a zoom actuator (not shown) via the distance measurement control unit 52. Zooming can be performed by controlling the position of the zoom lens, and focusing can be performed by controlling the position of the focus lens. Further, the CPU 35 controls the aperture amount (aperture value) of the aperture by driving the aperture control 53 via the distance measurement control unit 52 and controls the amount of light incident on the image sensor 50.

  When shooting in the three-dimensional still image shooting mode, the CPU 35 drives the optical systems 14L and 14R of the shooting units 12L and 12R in synchronization. The optical systems 14L and 14R are always adjusted to have the same focal length (zoom magnification) and the same incident light quantity (aperture value). Further, in the three-dimensional still image shooting mode, focus adjustment is performed so that the main subject is always in focus.

  The strobe 13 is composed of, for example, a discharge tube (xenon tube), and emits light as necessary when photographing a dark subject or in backlight. The charging / light emission control unit 57 includes a main capacitor for supplying a current for causing the strobe 13 to emit light. The CPU 35 transmits a flash light emission command to the charge / light emission control unit 57, and performs charge control of the main capacitor, discharge (light emission) timing of the strobe 13, and control of the discharge time. A light emitting diode may be used as the strobe 13.

  The imaging unit 12 captures a distance light emitting element 61 (for example, a light emitting diode) for irradiating the subject with light, and an image of the subject irradiated with light by the distance light emitting element 61 (ranging image). A distance image sensor 62. The distance driving / control circuit 63 causes the distance light emitting element 61 to emit light at a predetermined timing based on a command from the CPU 35 and controls the distance imaging element 62 to photograph a distance measuring image.

  The distance measurement image captured by the distance image sensor 62 is converted into digital data by the A / D converter 66 and input to the distance information processing circuit 67. The distance information processing circuit 67 uses the distance measurement image acquired from the distance image sensor 62 and based on the principle of so-called triangulation, the main subject captured by the imaging units 12L and 12R and the digital camera 11. The distance between them (subject distance) is calculated. The subject distance calculated by the distance information processing circuit 67 is recorded in the distance information recording circuit 68.

  As a method for calculating the subject distance, the flight of light from the time when the distance light emitting element 61 emits light until the light irradiated by the distance light emitting element 61 is reflected by the main subject and reaches the distance image sensor 62. A TOF (Time of Flight) method for calculating the subject distance from the time (delay time) and the speed of light may be used.

  The imaging unit 15 includes an interval / convergence angle driving circuit 71 and an interval / convergence angle detection circuit 72. The interval / convergence angle driving circuits 71L and 71R drive the optical systems 14L and 14R, respectively. The CPU 35 operates the interval / convergence angle drive circuits 71L and 71R via the interval / convergence angle control circuit 73 to adjust the interval and the convergence angle between the optical systems 14L and 14R.

  The interval / convergence angle detection circuits 72L and 72R include means for transmitting and receiving radio waves, for example. The CPU 35 operates the interval / convergence angle detection circuits 72L and 72R via the interval / convergence angle control circuit 73 to transmit and receive radio waves, thereby measuring the interval and the convergence angle between the optical systems 14L and 14R. . The measurement results of the interval between the optical systems 14L and 14R and the convergence angle are stored in the lens interval / convergence angle storage circuit 74.

  The image sensor 50 is constituted by, for example, a CCD. A large number of photodiodes are two-dimensionally arranged on the light receiving surface of the image sensor 50, and color filters of three primary colors (R, G, B) are arranged in a predetermined arrangement in each photodiode. The optical image of the subject formed on the light receiving surface of the image sensor 50 by the optical system 14 is converted into signal charges corresponding to the amount of incident light by the photodiode. The signal charge accumulated in each photodiode is sequentially read out from the image sensor 50 as a voltage signal (R, G, B signal) corresponding to the signal charge based on a drive pulse given from the TG 77 according to a command of the CPU 35. The image sensor 50 has an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode. As the image sensor 50, an image sensor other than a CCD such as a CMOS sensor can be used.

  The analog signal processing unit 80 outputs correlated double sampling circuits (CDS), R, G, and B signals for removing reset noise (low frequency) included in the R, G, and B signals output from the image sensor 50. An AGS circuit for amplifying and controlling to a certain level is included. Analog R, G, B signals output from the image sensor 50 are subjected to correlated double sampling processing and amplified by the analog signal processing unit 80. The analog R, G, B signals output from the analog signal processing unit 80 are converted into digital R, G, B signals by the A / D converter 81 and input to the image input controller (buffer memory) 82. The

  The digital signal processing unit 85 includes a synchronization circuit (a processing circuit that converts a color signal into a simultaneous type by interpolating a spatial shift of the color signal associated with the color filter array of a single CCD), a white balance adjustment circuit, a gradation A conversion processing circuit (gamma correction circuit), a contour correction circuit, a luminance / color difference signal generation circuit, and the like are included. The digital R, G, and B signals input to the image input controller 82 are subjected to predetermined processing such as synchronization processing, white balance adjustment, gradation conversion, and contour correction by the digital signal processing unit 85. It is converted into a Y / C signal composed of a luminance signal (Y signal) and a color difference signal (Cr, Cb signal). The Y / C signal is stored in the work memory 39.

  When displaying the through image on the LCD panel 20, the Y / C signal generated in the digital signal processing unit 85 is sequentially supplied to the buffer memory 88. The display controller 89 reads the Y / C signal supplied to the buffer memory 88 and outputs it to the YC-RGB converter 90. The YC-RGB converter 90 converts the Y / C signal input from the display controller 89 into R, G, and B signals and outputs them to the LCD panel 20 via the driver 91. As a result, a through image is displayed on the LCD panel 20.

  In the playback mode, the final image data recorded on the memory card 29 (last recorded image data) is read by the memory controller 94 to the buffer memory 96 via the interface unit (I / F) 95, The data is decompressed to an uncompressed Y / C signal by the compression / expansion processing unit 97 and then input to the buffer memory 88. The display controller 89 reads the Y / C signal supplied to the buffer memory 88 and the display mode identifier included in the image data, and outputs it to the YC-RGB converter 90 according to the display mode corresponding to the display mode identifier. The YC-RGB converter 90 converts the Y / C signal input from the display controller 89 into R, G, and B signals and outputs them to the LCD panel 20 via the driver 91. As a result, the image data recorded on the memory card 29 is displayed on the LCD panel 20.

  The display mode determination unit 100 is connected to the CPU 35 and the like via the system bus 36. As shown in FIG. 4, the display mode determination unit 100 determines whether or not a plurality of image data obtained in the three-dimensional still image shooting mode is suitable for three-dimensional display. 102, a first determination circuit 103, a difference circuit 105, a shadow pixel number calculation circuit 106, and a second determination circuit 107. The parallax amount calculation circuit 102 calculates the parallax amount for each subject for a plurality of images obtained in the three-dimensional still image shooting mode. The first determination circuit 103 calculates a difference between the obtained parallax amounts. Further, the first determination circuit 103 determines whether or not the obtained difference in parallax amount is equal to or greater than a first threshold value TH1. The difference circuit 105 calculates a contrast difference ΔC between a plurality of images for each pixel. The shadow pixel number calculation circuit 106 calculates the number of pixels whose contrast difference ΔC is less than 0 (hereinafter referred to as the number of shadow pixels). The second determination circuit 107 determines whether or not the obtained number of shadow pixels is greater than or equal to the second threshold value TH2.

  The CPU 35 assigns a predetermined identifier to the image data according to the determination result by the display mode determination unit 100. When the first determination circuit 103 determines that the amount of parallax is greater than or equal to the first threshold TH1, or when the second determination circuit 107 determines that the amount of parallax is greater than or equal to the second threshold TH1, the CPU 35 A three-dimensional display identifier is assigned to the data. On the other hand, when the first determination circuit 103 determines that the amount of parallax is less than the first threshold TH1 and the second determination circuit 107 determines that the amount of parallax is less than the second threshold TH2, the CPU 35 A two-dimensional display identifier is assigned to the image data.

  Next, image capturing and recording processing will be described. In the two-dimensional still image shooting mode, a recording image is shot by a predetermined one shooting unit (for example, 12L). In the two-dimensional still image shooting mode, an image shot by the shooting unit 12L is compressed by the compression / expansion processing unit 97L. The compressed image data is recorded on the memory card 29 as image data in a predetermined format via the memory controller 94 and the I / F 95. For example, JPEG (Joint Photographic Experts Group) for still images, MPEG2 or MPEG4 for moving images, and H.264. It is recorded as compressed image data conforming to the H.264 standard.

  Next, image capturing and recording processing in the three-dimensional still image capturing mode will be described. As shown in FIG. 5, when the release button 28 is fully pressed (S10), shooting preparation processing (S12), strobe shooting (S14), and non-strobe shooting (S16) are sequentially performed. Although flash photography may be performed after non-flash photography, it is preferable to perform non-flash photography after flash photography from the viewpoint of preventing red-eye.

  In the shooting preparation process (S12), zooming, focusing, adjustment of the amount of light incident on the image sensor 50, AE processing, AWB processing, and the like are performed.

  In the flash photography (S14), photography by the photographing units 12L and 12R is performed simultaneously with the flash 13 emitting light. Two strobe ON images 120L (see FIG. 6) and 120R (see FIG. 7) photographed by the photographing units 12L and 12R are written in the work memory 39 through various image processing by the digital signal processing unit 85.

  In non-flash photography (S16), photography by the photography units 12L and 12R is performed simultaneously with the flash 13 not emitting light. Two strobe OFF images 121L (see FIG. 8) and 121R (see FIG. 9) photographed by the photographing units 12L and 12R are written in the work memory 39 through various image processing by the digital signal processing unit 85.

  After the non-strobe shooting (S16), the parallax amount calculation circuit 102 (see FIG. 4) performs a parallax amount calculation process for calculating the parallax amounts of the strobe OFF images 121L and 121R (S18).

  In the parallax amount calculation process (S18), first, corresponding points corresponding to each other between the images 121L and 121R are extracted using a technique such as stereo matching. Next, one of the images 121L and 121R is defined as a reference image, and the other is defined as a search image. Third, a plurality of feature points such as edges are extracted from the reference image, and the reference image is divided into a plurality of regions (blocks). Fourth, using the divided area as a template image, an area corresponding to the template image is searched from the search image. For the search of the corresponding region, SSD (Sum of Squared Difference) for obtaining the sum of squares of pixel value differences, SAD (Sum of Absolute Difference) for obtaining the sum of absolute values of pixel value differences, or the like can be used. . Fifth, corresponding points corresponding to the feature points of the reference image are extracted from the found search area. Sixth, the amount of parallax can be calculated from the coordinates of the feature points and the corresponding points.

  8 and 9 show an example in which feature points and corresponding points are extracted using the image 121L as a reference image and the image 121R as a search image. A search is performed on the feature point P_L (xL, y) of the reference image 121L. Corresponding points P_R (xR, y) of the image 121R are extracted. The parallax amount measurement circuit 100 calculates a distance between the feature point P_L (xL, y) and the corresponding point P_R (xR, y), for example, an absolute value of “xL−xR” as the parallax amount ΔP. In FIGS. 8 and 9, only one feature point and corresponding point are shown in order to avoid complicating the drawing. However, in actuality, a large number of feature points and corresponding points are extracted and the points between the points are extracted. Is calculated. In this manner, the parallax amount ΔP1 of the main subject Ob and the parallax amount ΔP2 of the background Bg are obtained by the parallax amount calculation process (S18).

  First, the first determination circuit 103 calculates a parallax amount difference ΔX (= | ΔP1−ΔP2 |) (S20). Next, the first determination circuit 103 performs a first determination process for determining whether or not the parallax amount difference ΔX is greater than or equal to the first threshold value TH1 (S22). The first threshold TH1 is stored in the ROM 37 in advance. When the parallax amount difference ΔX is greater than or equal to the first threshold value TH1, the CPU 35 assigns a three-dimensional display identifier to the images 121L and 121R (S24). Thereafter, the images 121L and 121R are recorded as image data in the memory card 29 together with the three-dimensional display identifier (S26).

  The first threshold value TH1 stored in the ROM 37 may be not only one but two or more. When there are a plurality of first threshold values TH1 stored in the ROM 37, a first determination process is performed based on the selected first threshold value TH1. An operation of selecting one first threshold TH1 from among the plurality of first thresholds TH1 can be performed by the operation unit 31. Therefore, the first determination process can be performed based on the desired first threshold value TH1 by operating the operation unit 31.

  In the first determination process (S22), when it is determined that the parallax amount difference ΔX is less than the first threshold TH1, the difference circuit 105 calculates the contrast difference ΔC between the images 120R and 121R for each pixel. Contrast difference calculation processing is performed (S30). The contrast difference ΔC is the contrast C1 of the pixel G121 (see FIG. 9) at the coordinates (i, j) among the pixels constituting the image 121R and the pixel G120 at the coordinates (i, j) among the pixels constituting the image 120R. This is a difference (C1−C0) from the contrast C0 of FIG. When the contrast difference ΔC of each pixel is arranged at the corresponding position of the pixel, a difference image ΔR as shown in FIG. 10 is obtained.

  Next, the shadow pixel number calculation circuit 106 performs a shadow pixel number calculation process (S32) for calculating the shadow pixel number SS. As shown in FIG. 11, the shadow pixel number SS is the total number of pixels having a contrast difference ΔC of less than 0 among the pixels constituting the difference image ΔR. By calculating the number of pixels where the contrast difference ΔC is less than 0, it is possible to calculate the number of shadow pixels SS resulting from flash photography.

  The second determination circuit 107 performs a second determination process for determining whether or not the obtained number of shadow pixels SS is greater than or equal to the second threshold TH2 (S34). The second threshold value TH2 is stored in the ROM 37 in advance. If the number of shadow pixels SS is equal to or greater than the second threshold TH2, a three-dimensional display identifier is assigned to the images 121L and 121R (S22). On the other hand, when the number of shadow pixels SS is less than the second threshold value TH2, a two-dimensional display identifier is assigned to the images 121L and 121R (S36). Thereafter, the images 121L and 121R are recorded as image data in the memory card 29 together with the assigned identifier (S26).

  The second threshold TH2 stored in the ROM 37 may be not only one but two or more. When there are a plurality of second threshold values TH2 stored in the ROM 37, a second determination process is performed based on the selected second threshold value TH2. An operation of selecting one second threshold value TH2 from the plurality of second threshold values TH2 can be performed by the operation unit 31. Therefore, the second determination process can be performed based on the desired second threshold value TH2 by operating the operation unit 31.

  Next, a flow for displaying image data on the LCD panel 20 will be described. As shown in FIG. 12, first, the image data recorded on the memory card 29 is read (S40). The CPU 35 determines whether or not the identifier given to the image data is a three-dimensional display identifier (S42). If the identifier assigned to the image data is a three-dimensional display identifier, the display controller 89 displays the image on the LCD panel 20 in the three-dimensional display mode (S44). On the other hand, when the identifier given to the image data is a two-dimensional display identifier, the display controller 89 displays the image on the LCD panel 20 in the two-dimensional display mode (S50). When the next image data is read, the process returns to S40 (S46). On the other hand, if the next image data is not read, the flow ends (S46).

  Since the present invention determines the suitability of the 3D display mode of the image data based on the size of the shadow of the subject included in the image data, the 3D display mode is more accurate than the method based only on the difference in parallax amount. Can be determined. For example, even when the background is plain as in the case of studio photography, according to the present invention, the perspective difference between the subject and the background can be evaluated based on the size of the shadow of the subject. The suitability of the mode can be determined with high accuracy. In addition, when the proportion of the subject in the entire image is large, the 3D display mode is preferable in terms of power. However, according to the present invention, the suitability of the 3D display mode is determined even in such a case. It can be done with high accuracy.

  In addition, since the present invention assigns an identifier based on the determination result to the image data, when reproducing the image data, the image data suitable for three-dimensional display is displayed in three dimensions, and the image is suitable for two-dimensional display. Data can be displayed in two dimensions. Therefore, according to the present invention, it is possible to automatically switch between the three-dimensional display and the two-dimensional display according to the characteristics of the image.

  Two pieces of image data to which a three-dimensional display identifier is assigned may be saved individually, or a combination of the pieces of image data may be saved. When storing a combination of two pieces of image data to which a three-dimensional display identifier is assigned as one piece of image data, the original two pieces of image data may be stored for backup. Only one of the two image data to which the two-dimensional display identifier is assigned may be stored, or both image data may be stored. When only one of the two image data is stored, the other may be stored for backup. Accordingly, the image can be displayed in a display mode different from the display mode determined to be appropriate by the present invention.

  In the above embodiment, the description has been given only in the case where all the parallax amounts of each subject are calculated in the parallax amount calculation processing (S18 in FIG. 5). Even in this case, the present invention can be applied. A flowchart in this case is shown in FIG. The description of the flowchart in FIG. 13 is performed only for parts different from the above embodiment, and the same parts as those in the above embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  When the release button 28 is fully pressed (S10), shooting preparation processing (S12), strobe shooting (S14), and non-strobe shooting (S16) are sequentially performed. In the parallax amount calculation process (S18), the parallax amounts of the strobe OFF images 121L and 121R are calculated. If the parallax amount of all subjects cannot be calculated in the parallax amount calculation process, both image data are saved, and the process ends. On the other hand, when there are a plurality of subjects for which the amount of parallax can be calculated and those for which the amount of parallax could not be calculated, a calculated value is set for the amount of parallax of the subject for which the amount of parallax has been calculated. A value (for example, a negative value) indicating that the parallax amount of the subject could not be calculated is set as the parallax amount of the subject whose parallax amount could not be calculated.

  Next, a calculation value determination process (S60) is performed to determine whether or not there is a parallax amount that could not be calculated among the parallax amounts of each subject calculated in the parallax amount calculation process. In the calculated value determination process, when it is determined that the parallax amount of each subject calculated in the parallax amount calculation process does not include a parallax amount that could not be calculated, a parallax amount difference ΔX is calculated ( S20). On the other hand, in the calculated value determination process, when it is determined that the parallax amount of each subject calculated in the parallax amount calculation process includes a parallax amount that could not be calculated, the contrast difference calculation process is performed. (S30).

  Thus, the parallax amount of each subject calculated in the parallax amount calculation processing is performed by performing the calculation value determination processing (S60) between the parallax amount calculation processing (S18) and the calculation of the parallax amount difference ΔX (S20). Therefore, even if it is determined that the parallax amount cannot be calculated, it is possible to determine the suitability of the three-dimensional display mode.

  In the above embodiment, the strobe OFF image is stored. However, the present invention is not limited to this, and the strobe ON image may be stored.

In the above embodiment, the second determination process is performed using the number of shadow pixels of the difference image ΔR. However, the present invention is not limited to this, and the number of shadow pixels of the difference image ΔR of the image obtained by the second imaging unit is used. Then, the second determination process may be performed. Further, the difference image [Delta] L, the difference image ΔR each shadow pixel number _SS L, after calculating the SS _R, shadow pixel number _SS L, using the greater one of SS _R, even if the second determination process Good.

  In the above embodiment, the difference image ΔR is used for calculating the number of shadow pixels. However, the present invention is not limited to this, and a strobe ON image may be used for calculating the number of shadow pixels. In this case, non-flash photography (S16 in FIG. 5) and contrast difference ΔC calculation (S30 in FIG. 5) can be omitted.

  In the above embodiment, two strobe OFF images are used as images used for calculating the parallax amount ΔP. However, the present invention is not limited to this, and two strobe ON images may be used as images used for calculating the parallax amount ΔP. Alternatively, the difference image ΔL and the difference image ΔR2 may be used.

  In the above embodiment, the difference ΔX in the amount of parallax between the main subject and the subject including the background is the difference in the amount of parallax between the main subject and the background.However, the present invention is not limited to this, and a plurality of main subjects In the including subject, the difference between the parallax amount of the first main subject and the parallax amount of the second main subject may be used.

DESCRIPTION OF SYMBOLS 11 Digital camera 12 Digital photo frame 12L 1st imaging | photography part 12R 2nd imaging | photography part 14L Optical system 14R Optical system 35 Main CPU
50R image sensor 50L image sensor 100 display mode determination unit 102 parallax amount calculation circuit 103 first determination circuit 105 difference circuit 106 shadow pixel number calculation circuit 107 second determination circuit

Claims (9)

Parallax amount calculation means for calculating a first parallax amount for the first subject and a second parallax amount for the second subject based on a plurality of image data obtained by simultaneously photographing the subject from different viewpoints. When,
When the difference between the first parallax amount and the second parallax amount is greater than or equal to a first threshold, the plurality of image data is determined to be appropriate for three-dimensional display, and the difference between the parallax amounts A first display mode determination unit that puts the determination of the plurality of image data on hold when it is less than a first threshold;
When the difference in parallax amount is less than the first threshold value, the number of shadow pixels calculating means for calculating the number of pixels forming the shadow of the subject among the pixels forming the image data;
When the calculated number of shadow pixels is equal to or greater than a second threshold, the plurality of image data are determined to be appropriate for three-dimensional display, and the calculated number of shadow pixels is the second number. A display mode determination apparatus comprising: a second display mode determination unit that determines that the plurality of pieces of image data are not appropriate for three-dimensional display when less than a threshold value.   The shadow pixel number calculating means calculates, for each pixel, the contrast difference of the image data when the subject is irradiated with light when the image data when the subject is not irradiated with light is used as a reference. 2. The display mode determination apparatus according to claim 1, wherein the total number of pixels having the contrast difference of less than 0 is set as the number of shadow pixels.   A three-dimensional display identifier is assigned to the plurality of image data determined to be appropriate for three-dimensional display, and two-dimensional display is performed for the plurality of image data determined to be inappropriate for three-dimensional display by the display mode determination device. The display mode determination apparatus according to claim 1, further comprising an identifier providing unit that assigns an identifier for use.   A display device that displays the plurality of image data including the three-dimensional display identifier as a three-dimensional image and displays the plurality of image data including the two-dimensional display identifier as a two-dimensional image is provided. The display mode determination apparatus according to claim 3.   When the three-dimensional display identifier is given to the plurality of image data, the plurality of image data is stored, and when the two-dimensional display identifier is given to the plurality of image data 5. A display mode determination apparatus according to claim 3, further comprising recording means for storing at least one of the plurality of image data.   6. The display mode determination apparatus according to claim 5, wherein the recording unit stores both the plurality of image data even when the two-dimensional display identifier is assigned to the plurality of image data. . A plurality of optical systems that form an image of the subject are provided to be exposed to the digital camera body,
An image sensor that captures the combined image as image data, and a display mode determination device according to any one of claims 2 to 6, are incorporated in the digital camera body,
The light-emitting device that emits the light to the subject is disposed at a position sandwiching the second optical system by the first optical system. A parallax amount calculating step of calculating a first parallax amount for the first subject and a second parallax amount for the second subject based on a plurality of image data obtained by simultaneously photographing the subject from different viewpoints. When,
When the difference between the first parallax amount and the second parallax amount is greater than or equal to a first threshold, the plurality of image data is determined to be appropriate for three-dimensional display, and the difference between the parallax amounts If it is less than the first threshold, a first display mode determination step for holding the determination of the plurality of image data;
When the difference in the amount of parallax is less than the first threshold value, a shadow pixel number calculating step for calculating the number of pixels forming the shadow of the subject among the pixels forming the image data;
When the calculated number of shadow pixels is equal to or greater than a second threshold, the plurality of image data are determined to be appropriate for three-dimensional display, and the calculated number of shadow pixels is the second number. A display mode determination method, comprising: a second display mode determination step of determining that the plurality of image data are not appropriate for three-dimensional display when the image data is less than the threshold value. A display mode determination method according to claim 8,
An irradiation photographing step of simultaneously photographing the subject irradiated with light from different viewpoints to generate irradiation image data;
A non-irradiation photographing step of simultaneously photographing the subject after the irradiation of light from different viewpoints and generating non-irradiation image data;
The irradiation photographing step and the non-irradiation photographing step are performed before the shadow pixel number calculating step,
In the shadow pixel number calculating step,
A contrast difference calculating step for calculating, for each pixel, a contrast difference of the irradiated image data when the non-irradiated image data is used as a reference;
And a pixel number step in which the number of pixels having a contrast difference of less than 0 is set as the number of shadow pixels.

Images