JP4895312B2 - Three-dimensional display device, method and program - Google Patents

Description

  The present invention relates to a three-dimensional display device and method for three-dimensionally displaying a plurality of images so as to be stereoscopically viewed, and a program for causing a computer to execute the three-dimensional display method.

  It is known that stereoscopic viewing can be performed using parallax by combining a plurality of images and displaying them three-dimensionally. In such a stereoscopic view, a plurality of images are acquired by photographing the same subject from different positions using a plurality of cameras, and the plurality of images are three-dimensionally obtained using the parallax of the subjects included in the plurality of images. It can be generated by displaying.

  Specifically, in the case of a method for realizing stereoscopic vision by the naked eye balance method, a plurality of images can be arranged side by side to perform three-dimensional display. In addition, a plurality of images are overlapped with different colors such as red and blue, or a plurality of images are overlapped with different polarization directions, and a plurality of images are combined to perform three-dimensional display. Can do. In this case, stereoscopic viewing can be performed by using image separation glasses such as red-blue glasses and polarizing glasses to fuse the three-dimensionally displayed image with the auto-focus function of the eyes (an anaglyph method, a polarizing filter method). ).

  Further, it is also possible to display a plurality of images on a three-dimensional display monitor that can be stereoscopically viewed and stereoscopically viewed without using polarized glasses or the like, as in the parallax barrier method and the lenticular method. In this case, three-dimensional display is performed by cutting a plurality of images into strips in the vertical direction and arranging them alternately. In addition, a method of performing three-dimensional display by alternately displaying the left and right images while changing the light beam direction of the left and right images by using image separation glasses or attaching an optical element to the liquid crystal has been proposed. (Scan backlight method).

  In addition, a compound eye camera having a plurality of photographing units that performs photographing for performing such three-dimensional display has been proposed. Such a compound eye camera includes a 3D display monitor, generates a 3D display image from images acquired by a plurality of imaging units, and displays the generated 3D display image on the 3D display monitor in a 3D manner. it can.

  In such a compound-eye camera, at the time of shooting, shooting conditions such as an F value and a shutter speed, the number of shots and shooting date and time, icons for representing camera shake prevention, flash on / off, person mode, etc., pictograms, etc. It is necessary to display the object. By confirming such an object, the photographer can recognize the shooting date and time, the number of shots, the shooting conditions at the time of shooting, and the like.

Here, when displaying such an object during three-dimensional display, a method of switching from three-dimensional display to two-dimensional display has been proposed (see Patent Document 1). According to the method described in Patent Document 1, the stereoscopic effect of an image disappears when an object is displayed, so that the object can be easily viewed.
JP 2003-9185 A

  By the way, the method described in Patent Document 1 stops the three-dimensional display of an image when displaying an object, so that the stereoscopic effect of the image is lost. For this reason, it is conceivable to display the object superimposed on the 3D display image without stopping the 3D display. However, since the three-dimensional display uses the optical illusion to display the image three-dimensionally, if an object is suddenly displayed on the image while the image is being displayed in three dimensions, the line of sight is displayed. Moving to an object hinders stereoscopic viewing of the image. Conversely, if the stereoscopic view of the image is continued, the object cannot be recognized.

  The present invention has been made in view of the above circumstances, and an object thereof is to allow an object to be recognized without hindering stereoscopic vision.

A first three-dimensional display device according to the present invention includes a display unit that three-dimensionally displays a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
3D processing means for performing 3D processing for 3D display on the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated, and the object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject. An object display control means for displaying is provided.

  The “parallax amount according to the parallax amount of the main subject” is not only the same parallax amount as that of the main subject, but also the stereoscopic effect of the object and the main subject It includes a parallax amount that is different from the parallax amount of the main subject to the extent that the stereoscopic effect appears to match. In addition, when the stereoscopic effect can be changed in stages by setting the parallax amount of the object in five steps, for example, the parallax amount at the stage closest to the parallax amount of the main subject is also “the main subject's parallax amount”. It is included in the “parallax amount according to the parallax amount”.

  In the first three-dimensional display device according to the present invention, the object display control means displays a reference image object serving as a reference among the plurality of images at a predetermined position in the reference image. Displaying the object for the other image in the image at a position away from the corresponding position corresponding to the predetermined position in the other image at a disparity amount corresponding to the disparity amount of the main subject. It may be a means for three-dimensional display.

A second three-dimensional display device according to the present invention includes a display unit that three-dimensionally displays a plurality of images acquired by photographing at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
3D processing means for performing 3D processing for 3D display on the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, an object for each of the plurality of images is displayed at a position away from the main subject included in each of the plurality of images. Thus, an object display control means for three-dimensionally displaying the object is provided.

  The “predetermined distance” may be a perfect match in a plurality of images, and when a three-dimensionally displayed object is stereoscopically viewed, the stereoscopic effect of the object and the stereoscopic effect of the main subject appear to match. It may be different.

  In the first and second three-dimensional display devices according to the present invention, the main subject detection means is a means for detecting a focus area of a reference image serving as a reference among the plurality of images as the main subject. Also good.

  In the first and second three-dimensional display devices according to the present invention, the main subject detection unit accepts designation of a desired subject in the plurality of images, and uses the designated subject as the main subject. It is good also as a means to detect.

  In the first and second three-dimensional display devices according to the present invention, the main subject detection means may be means for detecting a specific subject included in the plurality of images as the main subject.

A first three-dimensional display method according to the present invention includes a display unit that three-dimensionally displays a plurality of images obtained by photographing at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
A three-dimensional display method in a three-dimensional display device comprising a three-dimensional processing means for performing three-dimensional processing for three-dimensional display on the plurality of images,
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated,
The object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject.

A second three-dimensional display method according to the present invention includes a display unit that three-dimensionally displays a plurality of images acquired by photographing at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
A three-dimensional display method in a three-dimensional display device comprising a three-dimensional processing means for performing three-dimensional processing for three-dimensional display on the plurality of images,
When displaying an object overlaid on the three-dimensionally displayed image, an object for each of the plurality of images is displayed at a position away from the main subject included in each of the plurality of images. Thus, the object is three-dimensionally displayed.

An image processing apparatus according to the present invention is an image processing apparatus that performs processing for three-dimensional display on a plurality of images acquired by shooting at different positions.
Main subject detection means for detecting a main subject included in the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, the parallax amount of the main subject between the plurality of images is calculated, and the parallax amount corresponding to the parallax amount of the main subject is calculated. And a storage means for storing as a feature.

An image processing method according to the present invention is an image processing method for performing processing for three-dimensional display on a plurality of images acquired by shooting at different positions.
Detecting a main subject included in the plurality of images;
Calculating an amount of parallax of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
A parallax amount corresponding to the parallax amount of the main subject is stored as a parallax amount of the object.

  Note that the first and second three-dimensional display methods and the image processing method according to the present invention may be provided as programs for causing a computer to execute them.

  According to the first three-dimensional display device and method of the present invention, when displaying an object overlaid on a three-dimensionally displayed image, the parallax amount of the main subject is calculated between the plurality of images, and the parallax amount of the main subject is calculated. The object is three-dimensionally displayed with a parallax amount corresponding to the above. Thereby, since the object can be stereoscopically viewed together with the image, the object can be recognized without interfering with the stereoscopic view of the image.

  According to the second three-dimensional display device and method of the present invention, when an object is displayed overlaid on a three-dimensionally displayed image, an object for each of the plurality of images is included in each of the plurality of images. By displaying the object at a predetermined distance from the main subject, the object is displayed three-dimensionally. Thereby, since the object can be stereoscopically viewed together with the image, the object can be recognized without interfering with the stereoscopic view of the image. Further, since it is not necessary to calculate the parallax amount of the main subject, the object can be quickly displayed in three dimensions.

  In addition, by detecting the in-focus area of the reference image among a plurality of images as the main subject, the object can be displayed with the same stereoscopic effect as the most watched area in the three-dimensional display. Recognition becomes easier.

  In addition, by detecting a designated desired subject among a plurality of images as a main subject, the object can be displayed with the same stereoscopic effect as the designated subject when displayed in three dimensions. Is easier to recognize.

  In addition, by detecting a specific subject among a plurality of images as a main subject, an object can be displayed with the same stereoscopic effect as a specific subject that is most likely to be watched in three-dimensional display. , Recognition of the object becomes easier.

  According to the image processing apparatus and method of the present invention, when displaying an object overlaid on a three-dimensionally displayed image, a parallax amount of a main subject between a plurality of images is calculated, and according to the parallax amount of the main subject The parallax amount is stored as the parallax amount of the object. For this reason, when displaying a plurality of images three-dimensionally, the object can be stereoscopically viewed together with the image by displaying the object with the stored amount of parallax of the object. It becomes possible to recognize.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing an internal configuration of a compound eye camera to which a three-dimensional display device according to a first embodiment of the present invention is applied. As shown in FIG. 1, the compound-eye camera 1 according to the first embodiment includes two photographing units 21A and 21B, a photographing control unit 22, an image processing unit 23, a compression / decompression processing unit 24, a frame memory 25, and a media control unit 26. An internal memory 27 and a display control unit 28.

  FIG. 2 is a diagram illustrating the configuration of the photographing units 21A and 21B. As shown in FIG. 2, the photographing units 21A and 21B include lenses 10A and 10B, diaphragms 11A and 11B, shutters 12A and 12B, CCDs 13A and 13B, analog front ends (AFE) 14A and 14B, and A / D conversion units 15A, 15B is provided.

  The lenses 10 </ b> A and 10 </ b> B are configured by a plurality of functional lenses such as a focus lens for focusing on a subject and a zoom lens for realizing a zoom function, and focus data obtained by AF processing performed by the imaging control unit 22. Based on zoom data obtained by operating a zoom lever (not shown), the position is adjusted by a lens driving unit (not shown).

  The apertures 11A and 11B are adjusted in aperture diameter by an aperture drive unit (not shown) based on aperture value data obtained by AE processing performed by the imaging control unit 22.

  The shutters 12A and 12B are mechanical shutters, and are driven by a shutter driving unit (not shown) according to the shutter speed obtained by the AE process.

  The CCDs 13A and 13B have a photoelectric surface in which a large number of light receiving elements are two-dimensionally arranged, and subject light is imaged on the photoelectric surface and subjected to photoelectric conversion to obtain an analog imaging signal. In addition, color filters in which R, G, and B color filters are regularly arranged are arranged on the front surfaces of the CCDs 13A and 13B.

  The AFEs 14A and 14B perform processing for removing noise of the analog imaging signal and processing for adjusting the gain of the analog imaging signal (hereinafter referred to as analog processing) for the analog imaging signals output from the CCDs 13A and 13B.

  The A / D converters 15A and 15B convert the analog imaging signals subjected to analog processing by the AFEs 14A and 14B into digital signals. Note that an image represented by digital image data acquired by the photographing unit 21A is a first image G1, and an image represented by image data acquired by the photographing unit 21B is a second image G2.

  The imaging control unit 22 includes an AF processing unit and an AE processing unit (not shown). The AF processing unit determines the focus area and the focal positions of the lenses 10A and 10B based on the pre-images acquired by the shooting units 21A and 21B by half-pressing the release button included in the input / output unit 34, and the shooting is performed. Output to the sections 21A and 21B. The AE processing unit determines an aperture value and a shutter speed based on the pre-image, and outputs them to the photographing units 21A and 21B.

  Here, as a focus position detection method by AF processing, for example, a passive method that detects a focus position using a feature that the contrast of an image is high when a desired subject is in focus can be considered. . More specifically, the pre-image is divided into a plurality of AF areas, and an image in each AF area is subjected to a filtering process using a high-pass filter, and an AF evaluation value is calculated for each AF area. That is, the position corresponding to the AF area with the highest output value by the filter is detected as the in-focus position. In the first embodiment, the imaging control unit 22 outputs the AF area from which the in-focus position is obtained in the first image G1, that is, the in-focus area to the display control unit 28 described later as the main subject area. . For this reason, in the first embodiment, the imaging control unit 22 corresponds to a main subject detection unit.

  In addition, the shooting control unit 22 instructs the shooting units 21A and 21B to perform main shooting for acquiring the main images of the first and second images G1 and G2 by fully pressing the release button. Before the release button is operated, the shooting control unit 22 displays a through image having a smaller number of pixels than the first and second images G1 and G2 for allowing the shooting unit 21A to check the shooting range. Then, an instruction to sequentially acquire at a predetermined time interval (for example, 1/30 second interval) is given.

  The image processing unit 23 adjusts white balance, gradation correction, sharpness correction, and color correction on the digital image data of the first and second images G1 and G2 acquired by the imaging units 21A and 21B. Etc. are applied. It should be noted that reference numerals G1 and G2 before processing are also used for the first and second images after processing in the image processing unit 23.

  The compression / decompression processing unit 24 is processed by the image processing unit 23, and a three-dimensional display image generated from the main images of the first and second images G1 and G2 for three-dimensional display as will be described later. Is compressed in a compression format such as JPEG, and a three-dimensional image file for three-dimensional display is generated. The three-dimensional image file includes image data of the first and second images G1 and G2 and image data of a three-dimensional display image. In addition, a tag in which incidental information such as shooting date and time is stored is assigned to the image file based on the Exif format or the like.

  The frame memory 25 is used when various processes including the process performed by the image processing unit 23 described above are performed on the image data representing the first and second images G1 and G2 acquired by the imaging units 21A and 21B. This is a working memory.

  The media control unit 26 accesses the recording medium 29 and controls writing and reading of a 3D image file or the like.

  The internal memory 27 stores various constants set in the compound eye camera 1, a program executed by the CPU 33, and the like.

  The display control unit 28 two-dimensionally displays the first and second images G1 and G2 stored in the frame memory 25 at the time of shooting on the monitor 20 or the first and second images recorded on the recording medium 29. The images G1 and G2 are displayed two-dimensionally on the monitor 20. In addition, the display control unit 28 displays the first and second images G1 and G2 that have been subjected to the three-dimensional processing as described later on the monitor 20 in a three-dimensional manner or the three-dimensional image recorded in the recording medium 29. It is also possible to display the file on the monitor 20 three-dimensionally. Note that switching between the two-dimensional display and the three-dimensional display may be performed automatically, or may be performed by an instruction from the photographer using the input / output unit 34 described later. Here, when the three-dimensional display is performed, the through images of the first and second images G1 and G2 are three-dimensionally displayed on the monitor 20 until the release button is pressed.

  When the display mode is switched to the three-dimensional display, both the first and second images G1 and G2 are used for display as described later, and when the display mode is switched to the two-dimensional display, the first and second images are used. One of the second images G1 and G2 is used for display. In the present embodiment, the first image G1 is used for two-dimensional display.

  In addition, when the image is displayed on the monitor 20 three-dimensionally or two-dimensionally, the display control unit 28 displays the object superimposed on the image. Here, as objects, shooting conditions such as the F value and shutter speed, the number of shots and the shooting date / time, icons for camera shake prevention, flash on / off, person mode, and the like, and pictograms are used. The object data is stored in the internal memory 27.

  FIG. 3 is a diagram illustrating an example of an object. Here, a state where an object is superimposed on one of the first and second images G1 and G2 is shown. As shown in FIG. 3, the image has a flash emission icon in the upper left corner, the number of shots taken so far in the upper right corner, the F value and shutter speed in the lower left corner, and characters representing the shooting date in the lower right corner. Overlaid on the image.

  The objects to be displayed are not limited to those shown in FIG. 3. Characters for informing the photographer of situations such as when AF processing cannot be performed, icons for performing face detection, camera shake prevention processing For example, an icon for performing shooting, an icon for performing macro shooting, and the like can be used.

  Further, the compound eye camera 1 according to the present embodiment includes a three-dimensional processing unit 30. The three-dimensional processing unit 30 performs three-dimensional processing on the first and second images G1 and G2 in order to display the first and second images G1 and G2 on the monitor 20 in three dimensions.

  Here, as the three-dimensional display in the present embodiment, any known method can be used. For example, the first and second images G1 and G2 are displayed side by side and stereoscopically viewed by the naked eye balance method, or a lenticular lens is attached to the monitor 20 and the images G1 and G2 are placed at predetermined positions on the display surface of the monitor 20. By displaying, a lenticular method can be used in which the first and second images G1 and G2 are made incident on the left and right eyes to realize three-dimensional display. Further, the optical path of the backlight of the monitor 20 is optically separated so as to correspond to the left and right eyes, and the first and second images G1 and G2 are separated on the display surface of the monitor 20 to the left and right of the backlight. By alternately displaying in accordance with the above, it is possible to use a scan backlight system that realizes three-dimensional display.

  The three-dimensional processing unit 30 performs three-dimensional processing corresponding to the three-dimensional display method on the first and second images G1 and G2. For example, when the three-dimensional display method is based on the naked eye balance method, three-dimensional processing is performed by generating a three-dimensional display image in which the first and second images G1 and G2 are arranged side by side. When the three-dimensional display method is the lenticular method, three-dimensional display images are generated by cutting the first and second images G1 and G2 into strips in the vertical direction and arranging them alternately. Process. When the three-dimensional display method is the scan backlight method, the first and second images G1 and G2 are alternately output to the monitor 20 in accordance with the separation of the backlight of the monitor 20 to the left and right 3 Perform dimension processing. In the following description, the method of arranging the first and second images G1 and G2 is referred to as a first method, and the other methods are referred to as a second method.

  The monitor 20 is processed according to the method of three-dimensional processing performed by the three-dimensional processing unit 30. For example, when the three-dimensional display method is the lenticular method, a lenticular lens is attached to the display surface of the monitor 20, and when the scan backlight method is used, an optical element for changing the light beam direction of the left and right images. Is attached to the display surface of the monitor 20.

  Here, when three-dimensional display is performed, an object is superimposed on an image displayed on the monitor 20. In the first embodiment, the display control unit 28 calculates the parallax amount of the main subject included in the first and second images G1 and G2, and monitors the object with the parallax amount according to the parallax amount of the main subject. 20 is displayed three-dimensionally. Details of this process will be described later.

  CPU33 controls each part of compound eye camera 1 according to the signal from input-output part 34 containing a release button, a zoom lever, etc.

  The data bus 35 is connected to each part constituting the compound eye camera 1 and the CPU 33, and exchanges various data and various information in the compound eye camera 1.

  Next, processing performed in the first embodiment will be described. FIG. 4 is a flowchart showing processing performed in the first embodiment. Here, processing in the case where the through images of the first and second images G1 and G2 are three-dimensionally displayed by the first method will be described. Further, it is assumed that the display mode of the monitor 20 is switched to the three-dimensional display. When the compound eye camera 1 is switched to the photographing mode or the power is turned on in the photographing mode, the CPU 33 starts monitoring whether or not the release button is half-pressed (step ST1). If the determination is affirmative, the imaging control unit 22 performs AF processing (step ST2).

  Then, the display control unit 28 calculates the amount of parallax between the first and second images G1 and G2 of the main subject included in the first image G1 detected by the AF process (step ST3).

  FIG. 5 is a diagram for explaining calculation of the amount of parallax. As shown in FIG. 5, the display control unit 28 cuts out the in-focus area of the first image G1, that is, the main subject area A1 from the first image G1, and uses the area A1 as a template to perform template matching with the second image G2. To extract a region A2 corresponding to the region A1 on the second image G2. Then, the absolute value Δx of the difference in the number of pixels in the horizontal direction between the center pixel O2 of the second image G2 and the center pixel O2 ′ of the area A2 is calculated as the amount of parallax.

  Then, the display control unit 28 creates an object to be displayed on the first and second images G1 and G2 so that the parallax amount is Δx (step ST4). Next, the three-dimensional processing unit 30 generates a three-dimensional display image by performing a three-dimensional process in which the first and second images G1 and G2 are arranged (step ST5), and the display control unit 28 further performs three-dimensional display. The created objects are superimposed on the first and second images G1 and G2 included in the display image to create a three-dimensional display image in which the objects are superimposed (creating a three-dimensional display image with an object, step ST6). ) Three-dimensional display images on which the objects are superimposed are three-dimensionally displayed on the monitor 20 (step ST7), and the process returns.

  FIG. 6 is a diagram showing first and second images G1 and G2 on which objects are superimposed in the first embodiment. As shown in FIG. 6, in the first and second images G1 and G2, an icon indicating flash emission is superimposed on the first and second images G1 and G2 in the upper left corner, respectively. Further, the parallax amount of the object is the same as the parallax amount Δx of the central cylinder as the main subject.

  In this way, the first and second images G1, G2 on which the objects are superimposed are arranged side by side to create a three-dimensional display image. Here, since each object is superimposed on the first and second images G1 and G2 with a parallax corresponding to the parallax amount Δx, when the three-dimensional display image on which the objects are superimposed is displayed in three dimensions, In addition, the object can be stereoscopically viewed.

  For example, in an image including a cone, a cylinder, and a cube as shown in FIG. 6, when a three-dimensional display in which the stereoscopic effect is reduced in the order of a triangular prism, a cylinder, and a cube as shown in FIG. If it is a subject, the object is stereoscopically viewed with the same stereoscopic effect as the main subject. In FIG. 7, the horizontal direction indicates the depth when stereoscopic viewing is performed. In FIG. 7, the vertical line is the display surface 20 </ b> A of the monitor 20.

  Here, when the first and second images G1 and G2 are three-dimensionally processed by the second method, the created object is a three-dimensional image similar to the three-dimensional processing for the first and second images G1 and G2. Processing is performed and superimposed on the three-dimensional display image.

  When the release button is fully pressed, the CPU 33 acquires the main images of the first and second images G1 and G2, and the three-dimensional processing unit 30 performs the three-dimensional processing from the first and second images G1 and G2. A display image is generated, the compression / expansion processing unit 24 generates a three-dimensional image file including the first and second images G1 and G2 and the image data of the three-dimensional display image, and the media control unit 26 performs three-dimensional display. Processing for recording the image file on the recording medium 29 is performed.

  As described above, in the first embodiment, when an object is displayed overlaid on a three-dimensionally displayed image, the parallax amount Δx of the main subject between the plurality of images is calculated, and according to the parallax amount Δx of the main subject. The object is three-dimensionally displayed with the amount of parallax. Thereby, since the object can be stereoscopically viewed together with the image, the object can be recognized without interfering with the stereoscopic view of the image.

  In addition, since the in-focus area of the image G1 among the plurality of images is detected as the main subject, the object can be displayed with the same stereoscopic effect as the most watched area in the three-dimensional display. , Recognition of the object becomes easier.

  In the first embodiment, the object is created so as to have a parallax amount similar to the parallax amount Δx of the main subject. However, when the object displayed three-dimensionally is stereoscopically viewed, The object may be created so that the parallax amount of the main subject is different from the parallax amount of the main subject so that the sense of sensation and the stereoscopic effect of the main subject appear to match.

  In addition, when the stereoscopic effect can be changed in steps by setting the parallax amount of the object in steps, for example, in five steps, the object has a parallax amount that is closest to the parallax amount of the main subject. May be created. For example, by setting the parallax amount in five steps, the stereoscopic effect of the object can be set in five steps as shown in FIG. 8, but the stereoscopic effect obtained from the parallax amount of the main subject becomes the third stereoscopic effect from the left. If they are close, the object may be created so that the third parallax amount is obtained. In FIG. 8, the horizontal direction indicates the depth when stereoscopic viewing is performed.

  Next, a second embodiment of the present invention will be described. In the first embodiment, the object is created so as to have a parallax amount corresponding to the parallax amount Δx of the main subject. However, in the second embodiment, the first and second images G1, G2 are used. The object is displayed at the same distance from the main subject included in each of the above. Note that the configuration of the compound eye camera according to the second embodiment is the same as that of the compound eye camera according to the first embodiment, and thus detailed description thereof is omitted here.

  Next, processing performed in the second embodiment will be described. FIG. 9 is a flowchart showing processing performed in the second embodiment. Here, processing in the case where the through images of the first and second images G1 and G2 are three-dimensionally displayed by the first method will be described. Further, it is assumed that the display mode of the monitor 20 is switched to the three-dimensional display. When the compound eye camera 1 is switched to the photographing mode or the power is turned on in the photographing mode, the CPU 33 starts monitoring whether or not the release button is half-pressed (step ST11). If the determination is affirmative, the imaging control unit 22 performs AF processing (step ST12).

  Then, the display control unit 28 calculates the position on the second image G2 of the main subject included in the first image G1 detected by the AF process (step ST13). Note that the position of the main subject on the second image G2 is detected by template matching shown in FIG. As the position of the main subject, the pixel position at the center of the detected area is used.

  Then, the display control unit 28 overlaps the first and second images G1 and G2 so that the distances from the main subjects of the first and second images G1 and G2 become a predetermined distance L. An object to be displayed is created (step ST14). Next, the three-dimensional processing unit 30 generates a three-dimensional display image by performing a three-dimensional process of arranging the first and second images G1 and G2 (step ST15), and the display control unit 28 further performs three-dimensional display. The created objects are superimposed on the first and second images G1 and G2 included in the display image to create a three-dimensional display image in which the objects are superimposed (creating a three-dimensional display image with an object, step ST16). ) Three-dimensional display images on which the objects are superimposed are three-dimensionally displayed on the monitor 20 (step ST17), and the process returns.

  FIG. 10 is a diagram showing first and second images G1 and G2 on which objects are superimposed in the second embodiment. As shown in FIG. 10, the first and second images G1 and G2 are each overlaid with an icon indicating flash emission in the upper left corner. In addition, since the object is the same distance L from the central cylinder that is the main subject in each of the first and second images G1 and G2, the amount of parallax of the object is the same as that of the central cylinder that is the main subject. It is the same.

  In this way, the first and second images G1, G2 on which the objects are superimposed are arranged side by side to create a three-dimensional display image. Here, each object is superimposed on the first and second images G1 and G2 with parallax corresponding to the main subject included in the first and second images G1 and G2, respectively. When a three-dimensional display image is three-dimensionally displayed, an object can be stereoscopically viewed together with the image.

  Here, when the first and second images G1 and G2 are three-dimensionally processed by the second method, the generated object is a three-dimensional image similar to the three-dimensional processing for the first and second images G1 and G2. Processing is performed and superimposed on the three-dimensional display image.

  As described above, in the second embodiment, when an object is displayed on a three-dimensionally displayed image, an object for each of a plurality of images is determined from a main subject included in each of the plurality of images. The object is displayed three-dimensionally by being displayed at a position separated by a distance L. Thereby, since the object can also be stereoscopically viewed, the object can be recognized without disturbing the stereoscopic view of the image. Further, since it is not necessary to calculate the parallax amount of the main subject, the object can be quickly displayed in three dimensions.

  In the first and second embodiments, the focus area obtained by the AF process is detected as the main subject. However, one of the first and second images G1 and G2, for example, the first image is used. May be displayed on the monitor 20, and the main subject may be detected by accepting designation of the main subject by the photographer for the displayed first image G1. For example, as shown in FIG. 11, the main subject is displayed on the monitor 20 and the cursor K is displayed, and the cursor K is moved using the input / output unit 34 so that the photographer can select the desired subject. The detected subject may be detected as the main subject. In this case, the input / output unit 34 and the CPU 33 correspond to main subject detection means.

  The monitor 20 may be provided with a touch panel type input unit, and the main subject may be selected by touching the touch panel. Further, a finder may be provided in the compound-eye camera 1 to detect the line of sight of the photographer looking through the finder, and the subject being watched by the photographer may be detected as the main subject.

  Further, as in the compound eye camera 1A according to the third embodiment shown in FIG. 12, a face detection unit 36 that detects a face region from the first and second images G1 and G2 is provided, and the face detected by the face detection unit 36 The region may be used as a main subject. In this case, the face detection unit 36 corresponds to main subject detection means.

  Here, the face detection unit 36 uses a template matching method, a method using a face discriminator obtained by machine learning learning using a large number of sample images of a face, and the like. Then, a face is detected, and a predetermined area surrounding the detected face is detected as a face area.

  In the third embodiment, the face area is detected. However, a subject other than the face, such as a car and an animal face, may be detected as the main subject. Here, in the case where the detection target is a monkey face, when a flash is emitted at the time of shooting, a balloon described as “flash emission” may be displayed as an object as shown in FIG. Thereby, it is possible to obtain an effect that the main subject teaches the contents of the object.

  In the first to third embodiments, the case where the through images of the first and second images G1 and G2 before the actual photographing are displayed in three dimensions has been described. An object is displayed when the first and second images G1 and G2 immediately after are displayed three-dimensionally, and when the first and second images G1 and G2 recorded on the recording medium 29 are displayed three-dimensionally for confirmation. The present invention can also be applied when displaying.

  When applying the first embodiment, when recording the first and second images G1 and G2 on the recording medium 29, the parallax amounts of the first and second images G1 and G2 are recorded. Also good. In this case, when displaying the first and second images G1 and G2 recorded on the recording medium 29, the parallax amount recorded on the recording medium 29 is used without calculating the parallax amount. May be displayed three-dimensionally.

  In the above embodiment, the two-eye camera 1 is provided with the two photographing units 21A and 21B and performs the three-dimensional display using the two images G1 and G2. However, the three-or-more photographing units are provided. The present invention can also be applied to the case where three-dimensional display is performed using these images.

  In the above embodiment, the three-dimensional display device according to the present invention is applied to the compound eye camera 1, but a three-dimensional display device including the display control unit 28, the three-dimensional processing unit 30, and the monitor 20 is provided alone. You may make it do. In this case, a plurality of images acquired by photographing the same subject at a plurality of different positions are input to the three-dimensional display device, and the object is three-dimensionally displayed as in the first to third embodiments. .

  The embodiment of the present invention has been described above, but there is also a program that causes a computer to function as a unit corresponding to the display control unit 28 and the three-dimensional processing unit 30 and to perform processing as illustrated in FIGS. 4 and 9. This is one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

1 is a schematic block diagram showing an internal configuration of a compound eye camera to which a three-dimensional display device according to a first embodiment of the present invention is applied. Diagram showing the configuration of the shooting unit Diagram showing example object The flowchart which shows the process performed in 1st Embodiment. The figure for demonstrating calculation of the amount of parallax The figure which shows the image for three-dimensional displays on which the object was piled up in 1st Embodiment. A diagram for explaining the stereoscopic effect of a subject included in an image The figure which shows the state under the setting of the three-dimensional effect of an object in five steps The flowchart which shows the process performed in 2nd Embodiment. The figure which shows the image for three-dimensional displays on which the object was piled up in 2nd Embodiment. Illustration for explaining subject selection The schematic block diagram which shows the internal structure of the compound eye camera to which the three-dimensional display device by the 3rd Embodiment of this invention is applied. The figure which shows the state where the object of the balloon is displayed

Explanation of symbols

1, 1A Compound-eye camera 21A, 21B Imaging unit 22 Imaging control unit 28 Display control unit 30 Three-dimensional processing unit

Claims (19)

Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
3D processing means for performing 3D processing for 3D display on the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated, and the object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject. A three-dimensional display device comprising object display control means for displaying. Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a specific subject included in the plurality of images as a main subject;
3D processing means for performing 3D processing for 3D display on the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated, and the object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject. A three-dimensional display device comprising object display control means for displaying.   The object display control means displays a reference image object serving as a reference among the plurality of images at a predetermined position in the reference image, and displays another image object of the plurality of images as the other image. 2. The display unit according to claim 1, wherein the object is displayed in a three-dimensional manner by displaying the object at a position away from a corresponding position corresponding to the predetermined position in the image of the main subject by a parallax amount corresponding to the parallax amount of the main subject. Or the three-dimensional display device of 2. Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
3D processing means for performing 3D processing for 3D display on the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, an object for each of the plurality of images is displayed at a position separated from the main subject included in each of the plurality of images by a predetermined distance. A three-dimensional display device comprising: object display control means for three-dimensionally displaying the object.   The three-dimensional display device according to claim 4, wherein the main subject detection unit is a unit that detects, as the main subject, a focus area of a reference image serving as a reference among the plurality of images.   5. The three-dimensional display according to claim 4, wherein the main subject detection means is means for accepting designation of a desired subject in the plurality of images and detecting the designated subject as the main subject. apparatus.   5. The three-dimensional display device according to claim 4, wherein the main subject detection means is means for detecting a specific subject included in the plurality of images as the main subject. Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
A three-dimensional display method in a three-dimensional display device comprising a three-dimensional processing means for performing three-dimensional processing for three-dimensional display on the plurality of images,
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated,
A three-dimensional display method, wherein the object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject. Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a specific subject included in the plurality of images as a main subject;
A three-dimensional display method in a three-dimensional display device comprising a three-dimensional processing means for performing three-dimensional processing for three-dimensional display on the plurality of images,
When displaying an object overlaid on the three-dimensionally displayed image, a parallax amount of the main subject between the plurality of images is calculated,
A three-dimensional display method, wherein the object is three-dimensionally displayed with a parallax amount corresponding to the parallax amount of the main subject. Display means for three-dimensionally displaying a plurality of images acquired by shooting at different positions;
Main subject detection means for detecting a main subject included in the plurality of images;
A three-dimensional display method in a three-dimensional display device comprising a three-dimensional processing means for performing three-dimensional processing for three-dimensional display on the plurality of images,
When displaying an object overlaid on the three-dimensionally displayed image, an object for each of the plurality of images is displayed at a position away from the main subject included in each of the plurality of images. To display the object three-dimensionally. A program for causing a computer to execute a three-dimensional display method in a three-dimensional display device including a display unit that three-dimensionally displays a plurality of images acquired by photographing at different positions,
A procedure for performing three-dimensional processing for three-dimensional display on the plurality of images;
Detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
A procedure for calculating a parallax amount of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
A program causing a computer to execute a procedure for three-dimensionally displaying the object on the display unit with a parallax amount corresponding to the parallax amount of the main subject. A program for causing a computer to execute a three-dimensional display method in a three-dimensional display device including a display unit that three-dimensionally displays a plurality of images acquired by photographing at different positions,
A procedure for performing three-dimensional processing for three-dimensional display on the plurality of images;
Detecting a specific subject included in the plurality of images as a main subject;
A procedure for calculating a parallax amount of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
A program causing a computer to execute a procedure for three-dimensionally displaying the object on the display unit with a parallax amount corresponding to the parallax amount of the main subject. A program for causing a computer to execute a three-dimensional display method in a three-dimensional display device including a display unit that three-dimensionally displays a plurality of images acquired by photographing at different positions,
A procedure for performing three-dimensional processing for three-dimensional display on the plurality of images;
Detecting a main subject included in the plurality of images;
When displaying an object overlaid on the three-dimensionally displayed image, an object for each of the plurality of images is displayed at a position away from the main subject included in each of the plurality of images. To cause the computer to execute a procedure for three-dimensionally displaying the object on the display means . In an image processing apparatus that performs processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Main subject detection means for detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
When displaying an object overlaid on the three-dimensionally displayed image, the parallax amount of the main subject between the plurality of images is calculated, and the parallax amount corresponding to the parallax amount of the main subject is calculated. An image processing apparatus comprising storage means for storing the image as a storage means. In an image processing apparatus that performs processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Main subject detection means for detecting a specific subject included in the plurality of images as a main subject;
When displaying an object overlaid on the three-dimensionally displayed image, the parallax amount of the main subject between the plurality of images is calculated, and the parallax amount corresponding to the parallax amount of the main subject is calculated. An image processing apparatus comprising storage means for storing the image as a storage means. In an image processing method for performing processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
Calculating an amount of parallax of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
An image processing method, wherein a parallax amount corresponding to the parallax amount of the main subject is stored as a parallax amount of the object. In an image processing method for performing processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Detecting a specific subject included in the plurality of images as a main subject;
Calculating an amount of parallax of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
An image processing method, wherein a parallax amount corresponding to the parallax amount of the main subject is stored as a parallax amount of the object. In a program for causing a computer to execute an image processing method for performing processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Detecting a focus area of a reference image serving as a reference among the plurality of images as a main subject;
A procedure for calculating a parallax amount of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
A program for causing a computer to execute a procedure for storing a parallax amount corresponding to a parallax amount of the main subject as a parallax amount of the object. In a program for causing a computer to execute an image processing method for performing processing for three-dimensional display on a plurality of images acquired by shooting at different positions,
Detecting a specific subject included in the plurality of images as a main subject;
A procedure for calculating a parallax amount of the main subject between the plurality of images when displaying an object overlaid on the three-dimensionally displayed image;
A program for causing a computer to execute a procedure for storing a parallax amount corresponding to a parallax amount of the main subject as a parallax amount of the object.

Images