JP4852169B2 - Three-dimensional display device, method and program - Google Patents

Description

  The present invention relates to a three-dimensional display device and method for three-dimensionally displaying a plurality of images so as to be stereoscopically viewed, and a program for causing a computer to execute the three-dimensional display method.

  It is known that stereoscopic viewing can be performed using parallax by combining a plurality of images and displaying them three-dimensionally. In such a stereoscopic view, a plurality of images are acquired by photographing the same subject from different positions using a plurality of cameras, and the plurality of images are three-dimensionally obtained using the parallax of the subjects included in the plurality of images. This can be done by displaying.

  Specifically, in the case of a method for realizing stereoscopic vision by the naked eye parallel method, a plurality of images can be arranged side by side to perform three-dimensional display. In addition, a plurality of images are overlapped with different colors such as red and blue, or a plurality of images are overlapped with different polarization directions, and a plurality of images are combined to perform three-dimensional display. Can do. In this case, stereoscopic viewing can be performed by using image separation glasses such as red-blue glasses and polarizing glasses to fuse the three-dimensionally displayed image with the auto-focus function of the eyes (an anaglyph method, a polarizing filter method). ).

  Further, it is also possible to display a plurality of images on a three-dimensional display monitor that can be stereoscopically viewed and stereoscopically viewed without using polarized glasses or the like, as in the parallax barrier method and the lenticular method. In this case, three-dimensional display is performed by cutting a plurality of images into strips in the vertical direction and arranging them alternately. In addition, a method of performing three-dimensional display by alternately displaying the left and right images while changing the light beam direction of the left and right images by using image separation glasses or attaching an optical element to the liquid crystal has been proposed. (Scan backlight method).

  In addition, a compound eye camera having a plurality of photographing units that performs photographing for performing the three-dimensional display as described above has been proposed. Such a compound eye camera is provided with a 3D display monitor, generates a 3D image for 3D display from images acquired by a plurality of imaging units, and displays the generated 3D image on the 3D display monitor. Can do.

  Further, in such a compound eye camera, a menu for setting the camera, shooting conditions such as F value and shutter speed at the time of shooting, characters such as the number of shots and shooting date, camera shake prevention, flash on / off, It is necessary to display an icon for representing a person mode or the like and an object such as a pictograph. By confirming such an object, the photographer can recognize the shooting date and time, the number of shots, the shooting conditions at the time of shooting, and the like.

  Here, when displaying such an object during three-dimensional display, if the object is arranged so as to have parallax in each of a plurality of images and displayed three-dimensionally, not only the image but also the object is stereoscopically viewed. Can do. Here, depth information is given in advance to the object displayed in the image, the depth information of the specified object is compared with the depth information of the other object, and the specified object is displayed in front of the other object. As described above, a method for changing the depth information of a designated object has been proposed (see Patent Document 1). In addition, when a specified object is three-dimensionally displayed among a plurality of objects displayed two-dimensionally, the specified object and an object that is likely to overlap with the other object so that the specified object does not overlap with another object A method for changing the position, size, and amount of parallax has been proposed (see Patent Document 2).

JP-A-2005-65162 Japanese Patent Laid-Open No. 2005-122501

  By the way, when an object is displayed on a three-dimensional image displayed three-dimensionally, the object is displayed three-dimensionally so as to have a predetermined stereoscopic effect. However, when the stereoscopic effect of a certain part included in the three-dimensional image displayed in three dimensions is larger than the stereoscopic effect of the object, the object overlaps with that part, so that the object appears to be embedded in that part. In particular, if the object consists only of characters, or the background of the object is translucent or transparent, if the object overlaps a certain part of the 3D image, the object is located behind the part in the foreground. Even though it appears to exist, it becomes a very unnatural way of seeing through that part.

  Here, since the distance information is previously given to both the three-dimensional image and the object in the method described in Patent Document 1, when the distance information is not given to the image for three-dimensional display, I do not know how to change the three-dimensionality of the object. Further, the technique described in Patent Document 2 is to eliminate overlapping of objects when a part of objects included in an image is displayed in 3D, and is applied when the entire image is displayed in 3D. Can not.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to display an object so that there is no sense of incongruity due to the overlap of the 3D image and the object when the object is displayed during 3D display.

A three-dimensional display device according to the present invention includes an image acquisition unit that acquires a plurality of images having parallax when a subject is viewed from different viewpoints;
3D processing means for performing 3D processing for 3D display on the plurality of images and performing the 3D processing on an object displayed superimposed on the 3D image displayed in 3D ,
Various displays including a three-dimensional display of the three-dimensional image, that is, display means for performing at least a three-dimensional display of the three-dimensional image;
Distance information calculating means for calculating distance information of the three-dimensional image,
The three-dimensional processing means is configured to make the object relative to the three-dimensional image relative to the three-dimensional image based on the distance information so as to eliminate the overlap between the object and the three-dimensional image during the three-dimensional display. It is a means for changing the general position.

  Here, the three-dimensional processing for the plurality of images and the three-dimensional processing for the object may be performed simultaneously or separately. That is, after superimposing an object on a plurality of images, three-dimensional processing may be performed on the plurality of images and objects simultaneously, or after performing three-dimensional processing on each of the plurality of images and objects separately. An object may be overlaid on the three-dimensionally processed image.

  The “position in the three-dimensional space” includes not only a position in the depth direction when a three-dimensionally displayed image is stereoscopically viewed, but also a position in a two-dimensional direction orthogonal to the depth direction.

In the three-dimensional display device according to the present invention, the distance information calculation unit may be configured to use the distance information for each pixel in a reference area where the object is displayed in a reference image serving as a reference among the plurality of images. As a means to calculate
The 3D processing unit is configured to display the 3D image of the object on the 3D image so that the object is displayed in 3D at a position on the near side of the 3D image based on distance information in the reference area. It is good also as a means to change the relative position of the depth direction on a dimensional space.

  In the three-dimensional imaging apparatus according to the present invention, the distance information calculation unit may be a unit that calculates parallaxes of corresponding points corresponding to each other between the plurality of images as the distance information.

  In this case, the distance information calculation unit may be a unit that extracts a feature portion in the plurality of images and calculates a parallax of the corresponding point from the feature portion.

  Further, in the three-dimensional imaging apparatus according to the present invention, when the plurality of images are acquired by imaging, the distance information calculating unit is configured to acquire the plurality of images based on imaging conditions when the plurality of images are acquired. It may be a means for calculating the distance information.

  In the three-dimensional imaging device according to the present invention, the three-dimensional processing unit may be configured to have the parallax with respect to the object such that the parallax of the corresponding points calculated in the reference region is greater than or equal to a maximum parallax. It is good also as a means to perform a three-dimensional process.

  In the three-dimensional imaging apparatus according to the present invention, the three-dimensional processing unit is calculated in the reference region when the three-dimensional processing is performed on the object so that the object has a predetermined parallax. The three-dimensional processing may be performed on the plurality of images so that the maximum parallax among the parallaxes of the corresponding points is equal to or smaller than the predetermined parallax.

  In the three-dimensional imaging apparatus according to the present invention, the three-dimensional processing unit may be configured so that the object is displayed at a position where the overlap with the three-dimensional image is eliminated based on the distance information. It is good also as a means to change the position of the said object in the direction orthogonal to the depth direction.

  In the three-dimensional imaging apparatus according to the present invention, the distance information calculation unit may be a unit that calculates parallaxes of corresponding points corresponding to each other between the plurality of images as the distance information.

  In this case, the distance information calculation unit may be a unit that extracts a feature portion in the plurality of images and calculates a parallax of the corresponding point from the feature portion.

  In the three-dimensional imaging apparatus according to the present invention, the image acquisition unit may be a plurality of imaging units that acquire the plurality of images by imaging a subject from different viewpoints.

  In the three-dimensional imaging apparatus according to the present invention, the distance information calculation means and the three-dimensional processing are performed so that the distance information is calculated and the plurality of images and the object are three-dimensionally processed at predetermined time intervals. Control means for controlling the means may be further provided.

  In the three-dimensional imaging apparatus according to the present invention, when the optical system of the imaging unit is driven, the distance information is calculated so that the calculation of the distance information and the three-dimensional processing of the plurality of images and the object are performed. Control means for controlling the calculation means and the three-dimensional processing means may be further provided.

In the three-dimensional imaging apparatus according to the present invention, an imaging control unit that controls the imaging unit to image the subject at predetermined time intervals;
Evaluation value calculation means for calculating an evaluation value including at least one evaluation value of luminance and high frequency components of the image acquired by the photographing means at the predetermined time interval;
The distance information calculating means and the three-dimensional processing means so as to perform the calculation of the distance information and the three-dimensional processing of the plurality of images and the object when the evaluation value changes beyond a predetermined threshold value. Control means for controlling the above may be further provided.

The three-dimensional imaging method according to the present invention includes an image acquisition unit that acquires a plurality of images having parallax when the subject is viewed from different viewpoints;
3D processing means for performing 3D processing for 3D display on the plurality of images and performing the 3D processing on an object displayed superimposed on the 3D image displayed in 3D ,
A three-dimensional display method in a three-dimensional display device comprising display means for performing at least three-dimensional display of the three-dimensional image,
Calculating distance information of the three-dimensional image;
Changing the relative position of the object in the three-dimensional space with respect to the three-dimensional image based on the distance information so as to eliminate the overlap between the object and the three-dimensional image during the three-dimensional display. It is characterized by.

  In addition, you may provide as a program for making a computer perform the three-dimensional display method by this invention.

Another three-dimensional display device according to the present invention includes: an image acquisition unit that acquires a plurality of images having parallax when the subject is viewed from different viewpoints;
3D processing means for performing 3D processing for 3D display on the plurality of images and performing the 3D processing on an object displayed superimposed on the 3D image displayed in 3D ,
Display means for performing at least three-dimensional display of the three-dimensional image;
Distance information calculating means for calculating distance information of the three-dimensional image,
Based on the distance information, the three-dimensional processing means is configured to eliminate the state where a part or all of the object at the time of the three-dimensional display is in a positional relationship to be hidden in the three-dimensional image. It is a means to change the relative position in the three-dimensional space with respect to the said three-dimensional image.

  According to the present invention, the distance information of the three-dimensional image displayed in three dimensions is calculated, and the object 3 is displayed based on the calculated distance information so as to eliminate the overlap between the object and the three-dimensional image at the time of three-dimensional display. The relative position on the three-dimensional space with respect to the three-dimensional image is changed. For this reason, even if the distance information of the three-dimensional image displayed three-dimensionally is unknown, the object can be three-dimensionally displayed so as not to overlap with the three-dimensional image.

  In addition, in the reference image serving as a reference among the plurality of images, distance information is calculated for each pixel in the reference region where the object is displayed, and the front side of the three-dimensional image is calculated based on the distance information in the reference region. By changing the relative position in the depth direction in the three-dimensional space with respect to the three-dimensional image of the object so that the object is three-dimensionally displayed at the side position, the three-dimensional image does not overlap with the front side of the three-dimensional image. Thus, the object can be displayed in three dimensions.

  Also, by calculating the parallax of corresponding points corresponding to each other between a plurality of images as distance information, the distances at the corresponding points of the plurality of images can be accurately calculated.

  In this case, by extracting feature portions from a plurality of images and calculating the parallax of the corresponding points from the feature portions, the distance information can be calculated with a small amount of computation, so the processing time can be shortened.

  Further, in the case where a plurality of images are acquired by shooting, the distance information is calculated based on the corresponding points by calculating the distance information based on the shooting conditions when the plurality of images are acquired, and In comparison, the distance information can be calculated with a small amount of calculation, so that the processing time can be shortened.

  Further, based on the distance information, by changing the position of the object in a direction orthogonal to the depth direction in the three-dimensional space so that the object is displayed at a position where the overlap with the three-dimensional image is eliminated, the three-dimensional image and The object can be displayed without overlapping the three-dimensional image without changing the stereoscopic effect of the object.

  In addition, by calculating distance information and performing three-dimensional processing of a plurality of images and objects at a predetermined time interval, the stereoscopic effect of the three-dimensional image displayed in a three-dimensional manner is changed by moving the subject being photographed. Even in such a case, the object can be displayed so as not to overlap with the three-dimensional image.

  In addition, when the optical system of the photographing unit is driven, the distance information is calculated, and the three-dimensional processing of a plurality of images and objects is performed, so that the zoom and focus positions are changed, so that the three-dimensional display is performed. Even when the stereoscopic effect of the three-dimensional image is changed, the object can be three-dimensionally displayed so as not to overlap with the three-dimensional image.

  In addition, when an evaluation value including at least one evaluation value of luminance and high-frequency component of the image acquired by the photographing unit is calculated at a predetermined time interval, and the evaluation value changes beyond a predetermined threshold, the distance By calculating information and performing three-dimensional processing of a plurality of images and objects, the brightness and / or focal position of the image to be captured changes, and the stereoscopic effect of the three-dimensional image displayed in three dimensions is changed. Even in such a case, the object can be displayed so as not to overlap with the three-dimensional image.

1 is a schematic block diagram showing an internal configuration of a compound eye camera to which a three-dimensional display device according to a first embodiment of the present invention is applied. Diagram showing the configuration of the shooting unit Diagram for explaining 3D processing for menu (Part 1) Diagram for explaining the calculation of parallax The figure which shows the example of the 1st and 2nd image The figure which shows the three-dimensional effect of the image shown in FIG. The figure which shows the three-dimensional effect at the time of displaying a menu on the image shown in FIG. Diagram for explaining the arrangement of the reference area Diagram showing distance distribution in the reference area Diagram for explaining 3D processing for menu (2) The figure for demonstrating the three-dimensional process by enlarging the parallax of a menu The figure for demonstrating the three-dimensional process by making the parallax of a 1st and 2nd image small The flowchart which shows the process performed in 1st Embodiment. A diagram showing the state in which an object is displayed The figure which shows the state from which the edge was extracted from the image in a reference | standard area | region The figure which shows the state from which the face area was extracted from the image in a reference | standard area | region The flowchart which shows the process performed in 2nd Embodiment. The flowchart which shows the process performed in 3rd Embodiment The figure which shows the distance distribution on the three-dimensional image calculated in 4th Embodiment

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing an internal configuration of a compound eye camera to which a three-dimensional display device according to a first embodiment of the present invention is applied. As shown in FIG. 1, the compound-eye camera 1 according to the first embodiment includes two photographing units 21A and 21B, a photographing control unit 22, an image processing unit 23, a compression / decompression processing unit 24, a frame memory 25, and a media control unit 26. An internal memory 27 and a display control unit 28. The photographing units 21A and 21B are arranged so as to photograph a subject with a predetermined baseline length and convergence angle. The vertical positions of the photographing units 21A and 21B are the same.

  FIG. 2 is a diagram illustrating the configuration of the photographing units 21A and 21B. As shown in FIG. 2, the photographing units 21A and 21B include focus lenses 10A and 10B, zoom lenses 11A and 11B, diaphragms 12A and 12B, shutters 13A and 13B, CCDs 14A and 14B, analog front ends (AFE) 15A and 15B, and A / D converters 16A and 16B are provided. The photographing units 21A and 21B include focus lens driving units 17A and 17B that drive the focus lenses 10A and 10B and zoom lens driving units 18A and 18B that drive the zoom lenses 11A and 11B.

  The focus lenses 10A and 10B are for focusing on a subject, and can be moved in the optical axis direction by focus lens driving units 17A and 17B including a motor and a motor driver. The focus lens driving units 17A and 17B control the movement of the focus lenses 10A and 10B based on focusing data obtained by AF processing performed by the imaging control unit 22 described later.

  The zoom lenses 11A and 11B are for realizing a zoom function, and can be moved in the optical axis direction by zoom lens driving units 18A and 18B including a motor and a motor driver. The zoom lens driving units 18A and 18B control the movement of the zoom lenses 11A and 11B based on the zoom data obtained by the CPU 33 by operating the zoom lever included in the input unit 34.

  The apertures 12A and 12B are adjusted in aperture diameter by an aperture drive unit (not shown) based on aperture value data obtained by AE processing performed by the imaging control unit 22.

  The shutters 13A and 13B are mechanical shutters, and are driven by a shutter driving unit (not shown) according to the shutter speed obtained by the AE process.

  The CCDs 14A and 14B have a photoelectric surface in which a large number of light receiving elements are two-dimensionally arranged, and subject light is imaged on the photoelectric surface and subjected to photoelectric conversion to obtain an analog photographing signal. In addition, color filters in which R, G, and B color filters are regularly arranged are arranged on the front surfaces of the CCDs 14A and 14B.

  The AFEs 15A and 15B perform processing for removing noise of the analog photographing signal and processing for adjusting the gain of the analog photographing signal (hereinafter referred to as analog processing) for the analog photographing signals output from the CCDs 14A and 14B.

  The A / D converters 16A and 16B convert the analog photographing signals subjected to the analog processing by the AFEs 15A and 15B into digital signals. Note that an image represented by digital image data acquired by the photographing unit 21A is a first image G1, and an image represented by image data acquired by the photographing unit 21B is a second image G2.

  The imaging control unit 22 includes an AF processing unit and an AE processing unit (not shown). The AF processing unit determines the focusing area based on the pre-images acquired by the imaging units 21A and 21B by half-pressing the release button included in the input unit 34, and determines the focal positions of the lenses 10A and 10B. Output to 21A and 21B. The AE processing unit calculates the brightness of the pre-image as the luminance evaluation value, determines the aperture value and the shutter speed based on the luminance evaluation value, and outputs them to the imaging units 21A and 21B.

  Here, as a focus position detection method by AF processing, for example, a passive method that detects a focus position using a feature that the contrast of an image is high when a desired subject is in focus can be considered. . More specifically, the pre-image is divided into a plurality of AF areas, and the image in each AF area is subjected to a filtering process using a high-pass filter, and an AF evaluation value that is an evaluation value of a high-frequency component is obtained for each AF area. The AF area having the highest evaluation, that is, the highest output value from the filter is detected as the in-focus area.

  In addition, the shooting control unit 22 instructs the shooting units 21A and 21B to perform main shooting for acquiring the main images of the first and second images G1 and G2 by fully pressing the release button. Before the release button is operated, the shooting control unit 22 displays a through image having a smaller number of pixels than the main image for causing the shooting unit 21A to check the shooting range at a predetermined time interval (for example, 1 / Instruct to sequentially obtain at intervals of 30 seconds).

  The image processing unit 23 adjusts white balance, gradation correction, sharpness correction, and color correction on the digital image data of the first and second images G1 and G2 acquired by the imaging units 21A and 21B. Etc. are applied. It should be noted that reference numerals G1 and G2 before processing are also used for the first and second images after processing in the image processing unit 23.

  The compression / decompression processing unit 24 is processed by the image processing unit 23, and a three-dimensional display image generated from the main images of the first and second images G1 and G2 for three-dimensional display as will be described later. Is compressed in a compression format such as JPEG, and a three-dimensional image file for three-dimensional display is generated. The three-dimensional image file includes image data of the first and second images G1 and G2 and image data of a three-dimensional display image. In addition, a tag in which incidental information such as shooting date and time is stored is assigned to the image file based on the Exif format or the like.

  The frame memory 25 is used when various processes including the process performed by the image processing unit 23 described above are performed on the image data representing the first and second images G1 and G2 acquired by the imaging units 21A and 21B. This is a working memory.

  The media control unit 26 accesses the recording medium 29 and controls writing and reading of a 3D image file or the like.

  The internal memory 27 stores baseline lengths and convergence angles of the imaging units 21A and 21B, various constants set in the compound-eye camera 1, a program executed by the CPU 33, and the like. Further, as will be described later, when the menu is three-dimensionally displayed, information on the position of the reference region for arranging the menu in the first image G1 and information on the parallax D0 given to the menu are also stored. Note that the parallax D0 is set to a predetermined value in advance.

  The display control unit 28 two-dimensionally displays the first and second images G1 and G2 stored in the frame memory 25 at the time of shooting on the monitor 20 or the first and second images recorded on the recording medium 29. The images G1 and G2 are displayed two-dimensionally on the monitor 20. In addition, the display control unit 28 displays the first and second images G1 and G2 that have been subjected to the three-dimensional processing as described later on the monitor 20 in a three-dimensional manner or the three-dimensional image recorded in the recording medium 29. Can also be displayed three-dimensionally on the monitor 20. Note that switching between the two-dimensional display and the three-dimensional display may be automatically performed, or may be performed by an instruction from the photographer using the input unit 34 described later. Here, when the three-dimensional display is performed, the through images of the first and second images G1 and G2 are three-dimensionally displayed on the monitor 20 until the release button is pressed.

  When the display mode is switched to the three-dimensional display, both the first and second images G1 and G2 are used for display as described later, and when the display mode is switched to the two-dimensional display, the first and second images are used. One of the second images G1 and G2 is used for display. In the present embodiment, the first image G1 is used for two-dimensional display.

  Further, the compound eye camera 1 according to the present embodiment includes a three-dimensional processing unit 30. The three-dimensional processing unit 30 performs three-dimensional processing on the first and second images G1 and G2 in order to display the first and second images G1 and G2 on the monitor 20 in three dimensions. Here, as the three-dimensional display in the present embodiment, any known method can be used. For example, the first and second images G1, G2 are displayed side by side and stereoscopically viewed by the naked eye parallel method, or a lenticular lens is attached to the monitor 20, and the images G1, G2 are placed at predetermined positions on the display surface of the monitor 20. By displaying the lenticular method, it is possible to use a lenticular method in which the first and second images G1 and G2 are incident on the left and right eyes to realize three-dimensional display. Further, the optical path of the backlight of the monitor 20 is optically separated so as to correspond to the left and right eyes, and the first and second images G1 and G2 are separated on the display surface of the monitor 20 to the left and right of the backlight. By alternately displaying in accordance with the above, it is possible to use a scan backlight system that realizes three-dimensional display.

  The monitor 20 is processed according to the method of three-dimensional processing performed by the three-dimensional processing unit 30. For example, when the three-dimensional display method is the lenticular method, a lenticular lens is attached to the display surface of the monitor 20, and when the scan backlight method is used, an optical element for changing the light beam direction of the left and right images. Is attached to the display surface of the monitor 20.

  The three-dimensional processing unit 30 performs a three-dimensional process for three-dimensionally displaying a menu for giving various instructions to the compound eye camera 1 on the monitor 20. FIG. 3 is a diagram for explaining a three-dimensional process for a menu. Here, the contents of the first and second images G1 and G2 are omitted for explanation. As illustrated in FIG. 3, the three-dimensional processing unit 30 performs a three-dimensional process on the first and second images G1 and G2 by overlapping the menu M1 and the menu M2 so as to have the parallax D0. Thereby, both the first and second images G1, G2 and the menus M1, M2 can be displayed three-dimensionally. Here, the parallax D0 is set to a predetermined value so that the menu has a desired stereoscopic effect, but the menus M1 and M2 on the first and second images G1 and G2 are displayed. It is changed according to the stereoscopic effect in the display area. This change process will be described later.

  In addition, the compound eye camera 1 according to the present embodiment includes a distance information calculation unit 31. The distance information calculation unit 31 obtains a corresponding point with the second image G2 for each pixel in the reference region B1 in which the menu M1 on the first image G1 is arranged, and calculates the parallax between the corresponding points as the distance between the pixels. Calculate as information. FIG. 4 is a diagram for explaining the parallax calculation. As shown in FIG. 4, when a coordinate system with the lower left corner as the origin is set in the first image G1, the reference region B1 exists in the range of x0 to x1 in the x direction. The distance information calculation unit 31 obtains corresponding points for each pixel in the reference area B1 by performing block matching within the search range S0 set on the second image G2. If the maximum allowable parallax value is Rmax and the minimum allowable parallax value is Rmin, the x-direction range of the search range S0 is the range of x0-Rmax to x1-Rmin on the second image G2. Become. In FIG. 4, the minimum allowable parallax value is 0.

  Here, since the vertical positions of the photographing units 21A and 21B are the same, the distance information calculation unit 31 sets a block of a predetermined size centered on each pixel for each pixel in the reference region B1, and searches for it. The correlation value is calculated while moving the block only in the x direction within the allowable parallax range within the range S0, and the pixel on the search range S0 that has obtained the correlation value having the largest correlation is determined as the corresponding point of the target pixel. Block matching is performed as follows.

  The correlation may be calculated by the following equation (1). In Expression (1), d is an allowable parallax, and the size centered on the pixel (x, y) in the reference area B1 and the search range S0 while changing the parallax d in the range of Rmin to Rmax. The correlation value SAD is calculated by performing block matching by calculating the absolute value of the difference using a block of w × w. Here, when Expression (1) is used, d that minimizes the correlation value SAD is the parallax for the pixel (x, y).

Note that the correlation value SSD by the square of the difference may be calculated by the following equation (2). Here, when Expression (2) is used, d that minimizes the correlation value SSD is the parallax for the pixel (x, y). Further, the correlation value COR may be calculated by the following equation (3). Here, when the correlation value COR is calculated by the expression (3), d that maximizes the correlation value COR is the parallax for the pixel (x, y).

  Here, assuming that the first and second images G1 and G2 are images including two persons P1 and P2 that are in front of and behind each other with a mountain as a background, as shown in FIG. When the two images G1 and G2 are three-dimensionally displayed, the positional relationship between the mountains, the person P1, and the person P2 in the depth direction (that is, the z direction) is as follows. The person P1 exists between a certain mountain and the person P2. In FIG. 6 and the following description, the origin in the z direction in the three-dimensional space when three-dimensional display is performed is, for example, the position of the imaging surface of the CCDs 14A and 14B of the photographing units 21A and 21B.

  When the menus M1 and M2 are overlaid on the first and second images G1 and G2 as shown in FIG. 3 and displayed three-dimensionally, the three-dimensional menu is displayed at the position z0 in the depth direction. Assuming that the parallax D0 of the menus M1 and M2 is set, the menu exists between the persons P1 and P2 as shown in FIG. In this case, in particular, the menu M0 has a semi-transparent or transparent background, or consists only of characters (that is, only has a pixel value in the font of the character, and has no pixel value in the others). If this is the case, the menu M0 can be seen through the person P2 even though the menu M0 is visible behind the person P2 that appears on the front side.

  Here, as described above, the distance information calculation unit 31 calculates the parallax at each pixel in the reference region B1 as the distance information. Therefore, by using the calculated parallax, when the menu M1 is arranged in the reference area B1 of the first image G1 as shown in FIG. 8, the parallax distribution in the reference area B1 as shown in FIG. Distance distribution can be obtained. In FIG. 9, the background mountain is on the back side, and the stereoscopic effect increases in the order of the persons P1 and P2. FIG. 9 shows a distance distribution in a certain xz plane orthogonal to the y-axis.

  Therefore, the three-dimensional processing unit 30 obtains the distance distribution of the pixels in the reference area B1 based on the distance information calculated by the distance information calculation unit 31, and uses the parallax calculated for the pixel having the largest stereoscopic effect in the distance distribution. 3D processing of the menus M1 and M2 is performed so as to have a large parallax. When the distance distribution is obtained as shown in FIG. 9, the pixel having the largest stereoscopic effect is the pixel corresponding to the point O1 in the person P2. Here, among the parallaxes calculated for the pixels in the reference region B1, the parallax at the point O1 is the largest. Therefore, assuming that the parallax at the point O1 is the maximum parallax Dmax, the three-dimensional processing unit 30 causes the menus M1 and M2 to have a larger parallax Db than the maximum parallax Dmax corresponding to the point O1, as shown in FIG. Then, a three-dimensional process is performed in which the menus M1 and M2 are arranged so as to overlap the first and second images G1 and G2. The parallax Db is calculated by adding a predetermined value to the maximum parallax Dmax. Further, Db = Dmax may be satisfied. When the first and second images G1 and G2 and the menus M1 and M2 are displayed three-dimensionally by performing the three-dimensional processing on the menus M1 and M2 as described above, z0 in the depth direction as shown in FIG. The menu M0 is stereoscopically viewed at a position z1 in front of the position P1 and in front of the person P2 that is most visible.

  Note that not only the parallax of the menus M1 and M2 but also the parallax of the first and second images G1 and G2 are changed so that the menu M0 is stereoscopically viewed in front of the person P2 during three-dimensional display. It is also possible to do so. Here, if the parallax between the first and second images G1, G2 is reduced, the stereoscopic effect of the three-dimensional image is reduced. For this reason, without changing the predetermined parallax D0 for the menus M1 and M2, the maximum parallax Dmax of the pixel corresponding to the point O1 in the person P2 is a parallax smaller by a predetermined value than D0. In addition, a three-dimensional process in which the overall parallax of the first and second images G1 and G2 is reduced may be performed. Note that Dmax = D0 may be satisfied. Accordingly, as shown in FIG. 12, the person P2 can be stereoscopically viewed rearward from the position z0 where the menu M0 is displayed during the three-dimensional display. Note that the three-dimensional processing for reducing the parallax may be performed by shifting the horizontal positions of the first and second images G1 and G2, and the first and second images G1 and G2 are obtained by morphing. It may be by processing.

  Further, by changing both the parallax of the menus M1 and M2 and the parallax of the first and second images G1 and G2, the person P2 is stereoscopically viewed rearward from the menu M0 during the three-dimensional display. It is also possible to do.

  The CPU 33 controls each part of the compound eye camera 1 according to a signal from the input part 34 including a release button and the like.

  The data bus 35 is connected to each part constituting the compound eye camera 1 and the CPU 33, and exchanges various data and various information in the compound eye camera 1.

  Note that when photographing a subject, the subject may move or the compound eye camera 1 may be moved to change the photographing position, so the sense of distance of the subject always changes. For this reason, in this embodiment, the process which changes the three-dimensional effect of the menu M0 shall be performed at predetermined time intervals. For this reason, a timer 36 is connected to the CPU 33.

  Next, processing performed in the first embodiment will be described. FIG. 13 is a flowchart showing processing performed in the first embodiment. Note that the monitor 20 of the compound eye camera 1 has the first and second images obtained by the three-dimensional processing unit 30 performing three-dimensional processing on the first and second images G1 and G2 acquired by the photographing units 21A and 21B. It is assumed that the through images of the images G1 and G2 are displayed three-dimensionally. In addition, since the present invention has a feature in processing when displaying a menu, only processing when an instruction to display a menu during through image display will be described.

  The CPU 33 monitors whether or not an instruction for menu display has been given by the photographer (step ST1). If step ST1 is affirmed, the reference position B1 for displaying the menu M1 in the first image G1 and the current parallax are displayed. Information on Dr is acquired (step ST2). Note that the initial value of the current parallax Dr is D0. Then, the distance information calculation unit 31 sets the reference area B1 in the first image G1 (step ST3), and calculates the parallax of each pixel in the reference area B1 as distance information (step ST4).

  Next, the three-dimensional processing unit 30 acquires the maximum parallax Dmax corresponding to the parallax with the largest stereoscopic effect among the calculated parallaxes of the distance information (step ST5), and the maximum parallax Dmax is the current of the menus M1 and M2. It is determined whether it is larger than the parallax Dr (step ST6). When step ST6 is affirmed, the three-dimensional processing unit 30 changes the current parallax Dr of the menus M1 and M2 to the parallax Db larger than the maximum parallax Dmax (step ST7), and the menu M1, M1 so as to have the parallax Db. M2 is arranged on each of the first and second images G1 and G2 (step ST8), and three-dimensional processing is performed on the first and second images G1 and G2 and the menus M1 and M2 (step ST9). Then, the display control unit 28 three-dimensionally displays the three-dimensional image on which the menu M0 is superimposed on the monitor 20 (step ST10). If step ST6 is negative, the process proceeds to step ST8, where the menus M1 and M2 are arranged in the first and second images G1 and G2, respectively, so as to have the current parallax Dr. Do.

  Next, the CPU 33 starts counting by the timer 36 and starts monitoring whether or not a predetermined time has elapsed since the three-dimensional display was performed (step ST11). If step ST11 is affirmed, the process returns to step ST2, and the processes after step ST2 are repeated.

  As described above, in the present embodiment, the distance information of each pixel in the reference region B1 is calculated, and the three-dimensional images of the first and second images G1 and G2 are calculated based on the distance information in the reference region. The relative position in the depth direction in the three-dimensional space with respect to the three-dimensional image of the menu M0 is changed so that the menu M0 is displayed at the front side position. The menu M0 can be displayed three-dimensionally so as not to overlap.

  In addition, since the calculation of distance information and the three-dimensional processing of the first and second images and the menus M1 and M2 are performed at predetermined time intervals, the menu M0 is displayed and the contents of the menu M0 are selected. Even if the stereoscopic effect of the three-dimensional image displayed in three dimensions is changed by moving the subject being photographed or moving the photographing position while the image is being photographed, it does not overlap with the three-dimensional image. Thus, the menu M0 can be displayed three-dimensionally.

  Here, at the time of shooting, as shown in FIG. 14, icons other than the menu such as the icon indicating flash emission, the number of shots taken so far, the F value and the shutter speed, and characters indicating the shooting date are always displayed. However, such an object can also be displayed three-dimensionally by performing a three-dimensional process. In the present embodiment, the calculation of distance information and the three-dimensional processing of a plurality of images and objects are performed at predetermined time intervals, so that the subject being shot moves or the shooting position moves. Thus, even when the stereoscopic effect of the three-dimensional image displayed in three dimensions is changed, the object can be three-dimensionally displayed so as not to overlap with the three-dimensional image.

  In the first embodiment, the parallax is calculated for all the pixels in the reference area B1, but the feature part in the image in the reference area B1 is extracted, and the parallax is obtained only for the pixel corresponding to the feature part. May be calculated. Here, as the characteristic part, a specific subject such as an edge or a predetermined face included in the image can be used.

  FIG. 15 is a diagram illustrating a state in which an edge is extracted from an image in the reference region, and FIG. 16 is a diagram illustrating a state in which a face region is extracted from an image in the reference region. In FIG. 15, the extracted edge is indicated by a thick line. As shown in FIG. 15, the number of pixels corresponding to the edge in the reference area B1 is smaller than the total number of pixels in the reference area B1. Further, as shown in FIG. 16, the number of pixels in the face area surrounded by the rectangle in the reference area B1 is also smaller than the total number of pixels in the reference area B1. Therefore, the feature amount in the image in the reference region B1 is extracted, and the parallax is calculated only for the pixel corresponding to the feature portion, so that the amount of calculation can be reduced, and the processing speed can be increased.

  In addition, specific subjects such as edges and faces are often included in main subjects included in the image, and are often located closest to the camera at the time of shooting. For this reason, by calculating the parallax of the characteristic portion, it is possible to obtain the pixel having the largest stereoscopic effect that is included in the reference region B1.

  Accordingly, the three-dimensional processing is performed on the menus M1 and M2 so that the parallax Dr is larger than the maximum parallax Dmax of the pixel having the largest stereoscopic effect included in the reference region B1, so that 3 The menu M0 can be displayed three-dimensionally so as not to overlap with the three-dimensional image.

On the other hand, from the focal length, the aperture value, the focus position, and the allowable circle of confusion in the photographing units 21A and 21B when the first and second images G1 and G2 are acquired, the following equations (4) and (5) are used. The distance between the near side and the far side of the depth of field can be calculated. The focal length, the aperture value, and the focus position can be obtained from the set values at the time of shooting of the focus lens driving units 17A and 17B, the aperture driving unit (not shown), and the zoom lens driving units 18A and 18B. The permissible circle of confusion stored in advance in the internal memory 27 can be obtained from the specifications of the CCDs 14A and 14B.

Here, Lnear: distance on the near side of the depth of field, Lfar: distance on the far side of the depth of field, f: focal length, F: aperture value, L: focus position, and δ: allowable circle of confusion.

  Thereby, it can be seen that the subject in the reference area B1 is within the distance range on the near side and the far side of the calculated depth of field. Therefore, from the distance Lmin on the near side of the depth of field, the parallax Dmax in the pixel having the largest stereoscopic effect included in the reference region B1 can be calculated by the following equation (6). Therefore, the menu M0 can be displayed three-dimensionally so as not to overlap the three-dimensional image on the front side of the three-dimensional image by performing three-dimensional processing on the menus M1, M2 so that the parallax is larger than the parallax Dmax. it can.

Parallax Dmax = base length K × focal length f / distance Lmin (6)
Next, a second embodiment of the present invention will be described. The configuration of the compound eye camera to which the 3D display device according to the second embodiment is applied is the same as that of the compound eye camera to which the 3D display device according to the first embodiment is applied, and only the processing to be performed is different. Here, detailed description of the configuration is omitted. In the first embodiment, distance information is calculated at a predetermined time interval, and a plurality of images and objects are three-dimensionally processed. In the second embodiment, the optical components of the photographing units 21A and 21B are used. When the system, that is, the focus lenses 10A and 10B and the zoom lenses 11A and 11B are driven, the calculation of distance information and the three-dimensional processing of a plurality of images and objects are different from the first embodiment. .

  Next, processing performed in the second embodiment will be described. FIG. 17 is a flowchart showing processing performed in the second embodiment. The CPU 33 monitors whether or not the photographer has instructed the menu display (step ST21). If step ST21 is affirmed, the reference position B1 for displaying the menu M1 in the first image G1 and the current parallax are displayed. Information on Dr is acquired (step ST22). Note that the initial value of the current parallax Dr is D0. Then, the distance information calculation unit 31 sets the reference area B1 in the first image G1 (step ST23), and calculates the parallax of each pixel in the reference area B1 as distance information (step ST24).

  Next, the three-dimensional processing unit 30 acquires the maximum parallax Dmax corresponding to the parallax with the largest stereoscopic effect among the calculated parallaxes of the distance information (step ST25), and the maximum parallax Dmax is the current of the menus M1 and M2. It is determined whether it is larger than the parallax Dr (step ST26). When step ST26 is affirmed, the three-dimensional processing unit 30 changes the current parallax Dr of the menus M1 and M2 to the parallax Db larger than the maximum parallax Dmax (step ST27), and the menu M1, M1 so as to have the parallax Db. M2 is arranged in each of the first and second images G1 and G2 (step ST28), and three-dimensional processing is performed on the first and second images G1 and G2 and the menus M1 and M2 (step ST29). Then, the display control unit 28 three-dimensionally displays the three-dimensional image on which the menu M0 is superimposed on the monitor 20 (step ST30). If step ST26 is negative, the process proceeds to step ST28, and the menus M1 and M2 are arranged on the first and second images G1 and G2, respectively, so as to have the current parallax Dr. Do.

  Next, the CPU 33 starts monitoring whether or not the optical system of the photographing units 21A and 21B has been driven (step ST31). When step ST31 is affirmed, the process returns to step ST22, and the processes after step ST22 are repeated.

  Thus, in the second embodiment, when the optical systems of the photographing units 21A and 21B, that is, the focus lenses 10A and 10B and the zoom lenses 11A and 11B are driven, the distance information is calculated, and the first and first Since the three-dimensional processing of the two images G1 and G2 and the menus M1 and M2 is performed, the stereoscopic effect of the three-dimensional image displayed in three dimensions can be obtained by changing the zoom position and the focus position of the photographing units 21A and 21B. Even if it is changed, the menu M0 can be displayed three-dimensionally so as not to overlap the three-dimensional image.

  Next, a third embodiment of the present invention will be described. The configuration of the compound eye camera to which the 3D display device according to the third embodiment is applied is the same as that of the compound eye camera to which the 3D display device according to the first embodiment is applied, and only the processing to be performed is different. Here, detailed description of the configuration is omitted. In the first embodiment, calculation of distance information and three-dimensional processing of a plurality of images and objects are performed at predetermined time intervals. However, in the third embodiment, the shooting control unit 22 The luminance evaluation value and the AF evaluation value are calculated every time the through images of the first and second images G1 and G2 are acquired, and the distance information is calculated when at least one of the luminance evaluation value and the AF evaluation value changes. In addition, the third embodiment is different from the first embodiment in that three-dimensional processing of the first and second images G1 and menus M1 and M2 is performed.

  Next, processing performed in the third embodiment will be described. FIG. 18 is a flowchart showing processing performed in the third embodiment. The CPU 33 monitors whether or not an instruction to display a menu has been given by the photographer (step ST41). When step ST41 is affirmed, the shooting control unit 22 first and second images that are currently acquired are displayed. A luminance evaluation value and an AF evaluation value are calculated from the through images of G1 and G2 (evaluation value calculation, step ST42). Next, the CPU 33 acquires information on the reference position B1 for displaying the menu M1 in the first image G1 and the current parallax Dr (step ST43). Note that the initial value of the current parallax Dr is D0. Then, the distance information calculation unit 31 sets the reference area B1 in the first image G1 (step ST44), and calculates the parallax of each pixel in the reference area B1 as distance information (step ST45).

  Next, the three-dimensional processing unit 30 acquires the maximum parallax Dmax corresponding to the parallax with the largest stereoscopic effect among the calculated parallaxes of the distance information (step ST46), and the maximum parallax Dmax is the current of the menus M1 and M2. It is determined whether or not the parallax is larger than (step ST47). If step ST47 is affirmed, the three-dimensional processing unit 30 changes the current parallax Dr of the menus M1 and M2 to the parallax Db larger than the maximum parallax Dmax (step ST48), and the menu M1, M1 so as to have the parallax Db. M2 is arranged in each of the first and second images G1 and G2 (step ST49), and three-dimensional processing is performed on the first and second images G1 and G2 and the menus M1 and M2 (step ST50). Then, the display control unit 28 three-dimensionally displays the three-dimensional image on which the menu M0 is superimposed on the monitor 20 (step ST51). If step ST47 is negative, the process proceeds to step ST49, and the menus M1 and M2 are arranged in the first and second images G1 and G2, respectively, so as to have the current parallax Dr. Do.

  Next, the imaging control unit 22 calculates a luminance evaluation value and an AF evaluation value from the through images of the first and second images G1 and G2 that are currently acquired (evaluation value calculation, step ST52), and the CPU 33 determines the luminance. It is determined whether at least one of the evaluation value and the AF evaluation value has changed beyond a predetermined threshold (step ST53). If step ST53 is negative, the processing returns to step ST52. When step ST53 is affirmed, the process returns to step ST43, and the processes after step ST43 are repeated.

  As described above, in the third embodiment, when at least one of the luminance evaluation value and the AF evaluation value changes beyond a predetermined threshold, the distance information is calculated, and the first and second images G1 are calculated. , G2 and menus M1 and M2 are three-dimensionally processed, and the stereoscopic effect of the three-dimensional image displayed in three dimensions is changed by changing the brightness and / or focus position of the image to be captured. Even so, the object can be displayed so as not to overlap the three-dimensional image.

  In the third embodiment, at least one of the luminance evaluation value and the AF evaluation value is calculated as the evaluation value. However, other evaluation values such as the color of the image may be used.

  In the first to third embodiments, the menus M1 and M2 are subjected to three-dimensional processing so that the parallax Db is larger than the parallax Dmax of the pixel having the largest stereoscopic effect in the reference area B1. The three-dimensional processing may be performed on the menus M1 and M2 by changing the position of the menu M0 in the two-dimensional direction perpendicular to the depth direction without changing the stereoscopic effect of the menu M0. Hereinafter, this will be described as a fourth embodiment. The configuration of the compound eye camera to which the 3D display device according to the second embodiment is applied is the same as that of the compound eye camera to which the 3D display device according to the first embodiment is applied, and only the processing to be performed is different. Here, detailed description of the configuration is omitted.

  In the fourth embodiment, the distance information calculation unit 31 calculates the parallax of all the pixels in the first image G1 as distance information. FIG. 19 is a diagram showing the distance distribution on the three-dimensional image calculated in the fourth embodiment. FIG. 19 shows a distance distribution in an xz plane orthogonal to the y-axis. When the first and second images G1 and G2 are arranged with predetermined parallax at predetermined reference positions in the first and second images G1 and G2, the stereoscopic effect of a certain portion P0 in the three-dimensional image is obtained. If it is larger than the stereoscopic effect of the menu M0, the portion P0 and the menu M0 overlap as shown in FIG. For this reason, in the fourth embodiment, the three-dimensional processing unit 30 does not change the parallax so that the menu M0 is displayed in a portion where the stereoscopic effect is smaller than the stereoscopic effect of the menu M0. The arrangement position of M2 is changed. For example, as shown in FIG. 19, the arrangement positions of the menus M1 and M2 are changed so that the menu M0 is three-dimensionally displayed on the right side of the portion P0.

  In this way, by changing the arrangement positions of the menus M1 and M2 on the first and second images G1 and G2 so that the menu M0 is displayed at a position that does not overlap with the three-dimensional image, the three-dimensional image and the menu are displayed. The menu M0 can be displayed so as not to overlap the three-dimensional image without changing the stereoscopic effect of M0.

  In the first to fourth embodiments, the two-eye camera 1 is provided with the two photographing units 21A and 21B to perform the three-dimensional display using the two images G1 and G2. However, three or more photographings are performed. The present invention can also be applied to a case where a unit is provided and three-dimensional display is performed using three or more images.

  In the first to fourth embodiments, the three-dimensional display device according to the present invention is applied to the compound-eye camera 1, but the display control unit 28, the three-dimensional processing unit 30, the distance information calculation unit 31, and the monitor The three-dimensional display device having 20 may be provided alone. In this case, a plurality of images acquired by photographing the same subject at a plurality of different positions are input to the three-dimensional display device, and the menu M0 is displayed together with the images in the same manner as in the first to fourth embodiments. Is displayed. In this case, an interface for inputting an image to the apparatus corresponds to the image acquisition means.

  As described above, the embodiments of the present invention have been described. As shown in FIGS. 12, 17, and 18, the computer functions as a unit corresponding to the display control unit 28, the three-dimensional processing unit 30, and the distance information calculation unit 31. A program for performing a simple process is also one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

DESCRIPTION OF SYMBOLS 1 Compound eye camera 21A, 21B Image pick-up part 22 Image pick-up control part 28 Display control part 30 Three-dimensional processing part 31 Distance information calculation part

Claims (15)

3D processing for three-dimensional display is performed on a plurality of images having parallax obtained by photographing the subject from different viewpoints at predetermined time intervals and viewing the subject from different viewpoints. Three-dimensional processing means for performing the three-dimensional processing on an object displayed in a three-dimensionally displayed three-dimensional image;
Display means for performing at least three-dimensional display of the three-dimensional image;
Distance information calculating means for calculating distance information of the three-dimensional image;
Evaluation value calculation means for calculating an evaluation value including at least one evaluation value of luminance and high frequency components of the image acquired at the predetermined time interval;
The distance information calculating means and the three-dimensional processing means so as to perform the calculation of the distance information and the three-dimensional processing of the plurality of images and the object when the evaluation value changes beyond a predetermined threshold value. Control means for controlling
The three-dimensional processing means is configured to make the object relative to the three-dimensional image relative to the three-dimensional image based on the distance information so as to eliminate the overlap between the object and the three-dimensional image during the three-dimensional display. A three-dimensional display device characterized by being means for changing a specific position. The distance information calculating means is means for calculating the distance information for each pixel in a reference area where the object is displayed in a reference image serving as a reference among the plurality of images.
The three-dimensional processing means is configured to display the three-dimensional image of the object on the three-dimensional image so that the object is three-dimensionally displayed at a position on the near side of the three-dimensional image based on distance information in the reference region. 2. The three-dimensional display device according to claim 1, wherein the three-dimensional display device is a means for changing a relative position in the depth direction in a three-dimensional space.   The three-dimensional display device according to claim 1, wherein the distance information calculation unit is a unit that calculates a parallax of corresponding points corresponding to each other between the plurality of images as the distance information.   4. The three-dimensional display device according to claim 3, wherein the distance information calculation unit is a unit that extracts a feature portion in the plurality of images and calculates a parallax of the corresponding point from the feature portion.   The three-dimensional display device according to claim 3, wherein the distance information calculation unit is a unit that calculates a parallax of the corresponding point based on a shooting condition when the plurality of images are acquired.   The distance information calculation means calculates a distance on the near side of the depth of field from the shooting condition, and calculates a maximum parallax of the corresponding point from the distance on the near side, the base length and the focal length at the time of shooting. 6. The three-dimensional display device according to claim 5, wherein the three-dimensional display device is means.   The three-dimensional processing means is means for performing the three-dimensional processing on the object so as to have a parallax greater than or equal to a maximum parallax among the parallaxes of the corresponding points calculated in the reference region. The three-dimensional display device according to any one of claims 3 to 6.   When the three-dimensional processing means performs the three-dimensional processing on the object so that the object has a predetermined parallax, the maximum parallax among the parallaxes of the corresponding points calculated in the reference region is The three-dimensional display device according to claim 3, wherein the three-dimensional display device is a unit that performs the three-dimensional processing on the plurality of images so that the parallax is equal to or less than a predetermined parallax.   The three-dimensional processing means is configured to position the object in a direction orthogonal to the depth direction in the three-dimensional space so that the object is displayed at a position where the overlap with the three-dimensional image is eliminated based on the distance information. The three-dimensional display device according to claim 1, wherein the three-dimensional display device is a means for changing.   The three-dimensional display device according to claim 9, wherein the distance information calculation unit is a unit that calculates parallax of corresponding points corresponding to each other between the plurality of images as the distance information.   The three-dimensional display device according to claim 10, wherein the distance information calculation unit is a unit that extracts a feature portion in the plurality of images and calculates a parallax of the corresponding point from the feature portion.   Control means for controlling the distance information calculation means and the three-dimensional processing means so as to perform the calculation of the distance information and the three-dimensional processing of the plurality of images and the objects at the predetermined time interval. The three-dimensional display device according to claim 1, wherein: 3D processing for three-dimensional display is performed on a plurality of images having parallax obtained by photographing the subject from different viewpoints at predetermined time intervals and viewing the subject from different viewpoints. Three-dimensional processing means for performing the three-dimensional processing on an object displayed in a three-dimensionally displayed three-dimensional image;
A three-dimensional display method in a three-dimensional display device comprising display means for performing at least three-dimensional display of the three-dimensional image,
Calculating distance information of the three-dimensional image;
Changing the relative position of the object in the three-dimensional space with respect to the three-dimensional image based on the distance information so as to eliminate the overlap between the object and the three-dimensional image during the three-dimensional display. When,
Calculating an evaluation value including an evaluation value of at least one of luminance and high-frequency component of the image acquired at the predetermined time interval;
And a step of calculating the distance information and performing a three-dimensional process on the plurality of images and the object when the evaluation value changes beyond a predetermined threshold value. . 3D processing for three-dimensional display is performed on a plurality of images having parallax obtained by photographing the subject from different viewpoints at predetermined time intervals and viewing the subject from different viewpoints. Three-dimensional processing means for performing the three-dimensional processing on an object displayed in a three-dimensionally displayed three-dimensional image;
A program for causing a computer to execute a three-dimensional display method in a three-dimensional display device provided with display means for performing at least three-dimensional display of the three-dimensional image,
A procedure for calculating distance information of the three-dimensional image;
A procedure for changing a relative position of the object in the three-dimensional space with respect to the three-dimensional image based on the distance information so as to eliminate an overlap between the object and the three-dimensional image at the time of the three-dimensional display. When,
A procedure for calculating an evaluation value including an evaluation value of at least one of luminance and high-frequency component of the image acquired at the predetermined time interval;
A program for causing a computer to execute calculation of the distance information and a procedure of performing three-dimensional processing of the plurality of images and the object when the evaluation value changes beyond a predetermined threshold value. 3D processing for three-dimensional display is performed on a plurality of images having parallax obtained by photographing the subject from different viewpoints at predetermined time intervals and viewing the subject from different viewpoints. Three-dimensional processing means for performing the three-dimensional processing on an object displayed in a three-dimensionally displayed three-dimensional image;
Display means for performing at least three-dimensional display of the three-dimensional image;
Distance information calculating means for calculating distance information of the three-dimensional image;
Evaluation value calculation means for calculating an evaluation value including at least one evaluation value of luminance and high frequency components of the image acquired at the predetermined time interval;
The distance information calculating means and the three-dimensional processing means so as to perform the calculation of the distance information and the three-dimensional processing of the plurality of images and the object when the evaluation value changes beyond a predetermined threshold value. Control means for controlling
Based on the distance information, the three-dimensional processing means is configured to eliminate the state where a part or all of the object at the time of the three-dimensional display is in a positional relationship to be hidden in the three-dimensional image. A three-dimensional display device characterized by being means for changing a relative position in a three-dimensional space with respect to the three-dimensional image.

Images