JP4676862B2 - Stereoscopic image re-photographing device - Google Patents

Description

The present invention provides a stereoscopic image display method according to the integral method, relates to a stereoscopic image recaptured equipment that produces an image having a controlled depth of the stereoscopic image to be reproduced.

  In general, as one of three-dimensional image display methods that allow a user to view a three-dimensional image freely from any viewpoint, integral photography (hereinafter referred to as “integral photography”) using convex lens groups or pinhole groups arranged in a plane. The IP) method is known.

Here, the concept of the IP scheme will be described with reference to FIG. In the IP system, as shown in FIG. 6A, at the time of photographing, a photographing aperture group (for example, a lens array) 11a composed of a number of pinholes or convex lenses on the front surface (subject T side) of a film (photographing surface) 11b. The element image group t1 including an inverted image (element image) of the subject T reduced according to the size and number of pinholes or convex lenses is photographed on the film 11b. Then, after developing the film 11b into a photograph, as shown in FIG. 6 (b), the photograph 31a is irradiated with diffused light from the back of the photograph (display surface) 31a, so that the photographing aperture of FIG. 6 (a) is obtained. An image (reproduced image T _x ) is reproduced through the display aperture group 31b having the same configuration as the group 11a. At this time, the path of light from the subject T photographed on the film (photographing surface) 11b in FIG. 6A and the path of light emitted from the photograph (display surface) 31a in FIG. since the same path, the distance between the reproduction image T _x a display aperture group 31b is composed and the distance between the imaging opening group 11a and the object T and the same (distance L _0).
However, with this configuration, a so-called reverse vision phenomenon occurs in which the projections and depressions are reversed such that a convex object is reproduced as a concave shape in a reproduced image.

As a first method for solving this problem, there has been disclosed a technique for re-photographing a stereoscopic image generated by the IP method and displaying the captured image (see Non-Patent Document 1).
In this method, as shown in FIG. 7A, the subject T is photographed in the same manner as the IP method described in FIG. 6, and the photographed first element image group t1 is photographed as shown in FIG. 7B. Is displayed. Further, as shown in FIG. 7B, the image reproduced by the displayed first element image group t1 and display opening group 21b is re-photographed as a second element image group t2. Then, as shown in FIG. 7C, by displaying the re-captured image (second element image group t2), the unevenness of the subject T is reversed twice, and viewed from the observation direction. Thus, it is possible to reproduce an image (reproduced image T _x ) having the same uneven shape as the subject T behind the display surface.

At this time, in this method, as shown in FIG. 7B, the second imaging aperture group 22a composed of a large number of pinholes or convex lenses that perform the second imaging, and the first display that performs the first display. by adjusting the distance between the opening group 21b _{(L d),} moving the position of the reproduced image shown in FIG. 7 (c), the distance _{(L d)} at equal distances _(L s = -L _d) amount corresponding be able to.

Also, as a second method for solving the above problem, a method of converting an element image that is an inverted image into a point target image in advance is disclosed (see Patent Document 1 and Non-Patent Document 2).
As shown in FIG. 8, this method converts an inverted element image into an upright image by subjecting the element image to point object conversion. Specifically, as shown in FIG. 9, a lens constituting the photographing aperture group is a refractive index distribution lens (GRIN lens GL), and the length of the GRIN lens GL is 3/4 times the period P of the optical path meandering. By doing so, the inverted element image can be converted into an upright image.

Then, as shown in FIG. 10 (a), the first imaging aperture group (GRIN lens array) 11Ga composed of a number of GRIN lenses is provided on the front surface (subject T side) of the film (imaging surface) 11b. An element image group t1 including a plurality of standing element images is photographed. Then, as shown in FIG. 10B, by displaying an upright element image (element image group t1) on the photograph (display surface) 31a, the depth is not inverted through the second display opening group 31b. An image (reproduced image T _x ) can be reproduced.

In this method, as shown in FIG. 11A, the imaging surface 11b of the first imaging aperture group (GRIN lens array) 11Ga shown in FIG. 10A is opposite (subject T side). A large-aperture convex lens (depth control lens L) is arranged on the side to shoot a subject. In this state, as shown in FIG. 11B, each element image of the element image group t1 is rotated by 180 degrees with respect to the subject T. Therefore, as shown in FIG. By rotating the entire surface by 180 degrees, an image (reproduced image Tx) in the same direction as the subject T can be reproduced. In this method, the position (depth) of the reproduced image can be controlled by adjusting the position (L ₁ , L ₂ ) of the large-diameter convex lens (depth control lens L).
JP-A-10-150675 (paragraphs 0026 to 0038, FIG. 7) HEIves, “Optical properties of a Lippmann lenticulated sheet”, J. Opt. Soc. Am. 21, pp. 171-176 (1931). F. Okano, J. Arai, H. Hoshino and I. Yuyama, “Three-dimensional video system based on integral photography”, Opt. Eng., 38, 1072-1077 (1999)

  In the first method described above, the unevenness of the reproduced image is not reversed, and the distance between the first display aperture group for performing the first display and the second imaging aperture group for performing the second imaging is set. By adjusting the position, the position of the reproduced image can be adjusted. In the first method, there is a demand for enhancing the effect of rendering an image by reproducing a reproduced image of a distant subject closer. However, in this first method, since the position of the reproduced image is only translated in the depth direction, the reproduced image of the subject existing at infinity is reproduced at infinity. For this reason, there is a problem that the desire to reproduce a reproduced image of a distant subject closer cannot be fully satisfied.

  On the other hand, the second method described above is excellent in that the reproduced image can be reproduced closer by adjusting the position of the large-diameter convex lens even when the subject exists at infinity. ing. However, in the second method, an optical system such as a depth control lens is disposed on the subject side of the first photographing aperture group, and a reproduced image is adjusted between the optical system and the photographing aperture group (GRIN lens array). It is necessary to secure a distance to do. For this reason, the photographing apparatus adopting this method has a size that secures at least the distance. However, since the photographing device performs photographing at different places such as outdoors, further reduction in size and weight is desired.

The present invention has been made in view of the above-described problems, and without using a lens for controlling the depth of the imaging apparatus, an image captured by the imaging apparatus is used to reproduce a stereoscopic image. and an object thereof is to provide a stereoscopic image recaptured equipment capable of performing the depth control.

  The present invention was devised to achieve the above object. First, the stereoscopic image re-photographing apparatus according to claim 1 is a reproduced image in which the unevenness of the subject is reversed when displayed as a stereoscopic image. A three-dimensional image re-imaging device that re-photographs an element image group consisting of a plurality of element images as an image to be a reconstructed image in which the unevenness is further inverted, an element image group display unit, a display aperture group, A photographing aperture group and an elemental image group photographing unit are provided, and a depth control lens is provided between the display aperture group and the photographing aperture group.

In such a configuration, the stereoscopic image re-capturing apparatus displays a plurality of element images (element image groups) captured in advance by the element image group display unit. Corresponding to the position of the element image, light is emitted from the display surface by a display aperture group in which a plurality of apertures that are pinholes or lenses are arranged in a plane on the display surface side of the element image group display unit. Let the element image light pass through. Further, the stereoscopic image re-photographing device causes the light of the element image to pass through and be condensed by a photographing aperture group in which a plurality of apertures that are pinholes or lenses are arranged in a plane on the photographing surface side in the photographing means. Then, the element image group (element image group) is photographed by the element image group photographing unit. As a result, the captured elemental image becomes an image showing the same direction and unevenness as the subject at the time of reproduction. A pinhole, a convex lens, or a lens system composed of a plurality of convex lenses in the optical axis direction can be used for the openings in the display aperture group and the imaging aperture group.
In this way, the stereoscopic image re-photographing device re-photographs an element image that is a reproduced image in which the unevenness captured in advance is reversed, so that the unevenness is further reversed, and the same uneven shape as the subject is obtained. This is an image for displaying a stereoscopic image.

  Furthermore, the stereoscopic image re-imaging device includes a depth control lens between the display aperture group and the imaging aperture group, thereby moving the position of the reproduced image reproduced by the display aperture group in the optical axis direction of the depth control lens. be able to. Then, the stereoscopic image re-imaging device captures the light that has passed through the depth control lens by the imaging means, and the captured image has the depth moved in the optical axis direction of the depth control lens. As a result, an image re-captured by the stereoscopic image re-capturing device is reproduced by a normal stereoscopic image display device, and is generated and visually recognized at a position different from the actual subject position (depth controlled). Will be. For example, by using a convex lens as the depth control lens, an image captured by the stereoscopic image re-capturing device becomes an image that forms a reproduced image in front of the actual subject position.

  In the stereoscopic image re-photographing apparatus according to claim 2, the depth control lens is a convex lens, and the convex lens is disposed in contact with the display aperture group.

  In such a configuration, the stereoscopic image re-photographing device arranges a convex lens as a depth control lens in contact with the display aperture group, so that an image re-photographed by the stereoscopic image re-photographing device is a convex lens (more precisely, a convex lens). From the position of the principal point) to the focal length of the convex lens. When the convex lens is arranged away from the display aperture group, the subject may not be reproduced as a reproduced image depending on the focal length of the convex lens and the position of the subject. Regardless of the position, the reproduced image of the subject is reproduced.

The present invention has the following excellent effects.
According to the first aspect of the present invention, a stereoscopic image is displayed by the depth control lens provided between the display aperture group and the imaging aperture group by re-imaging the image obtained by imaging the subject as a plurality of element images. It is possible to generate an image in which the depth is controlled with respect to the image to be performed. As a result, the effect of rendering the image can be enhanced by reproducing the reproduced image of the distant subject closer. In addition, it is not necessary to provide a lens for controlling the depth of the photographing device or the display device, and the photographing device or the display device can be reduced in size and weight.
According to the second aspect of the present invention, by arranging the convex lens as the depth control lens in contact with the display aperture group, an image capable of reproducing the reproduced image of the subject regardless of the position of the subject is obtained. You can shoot.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Standing body image re-capturing device]
First, referring to FIG. 1, it described three-dimensional image re-capturing apparatus according to implementation embodiments of the present invention. FIG. 1 is a schematic diagram illustrating a configuration of a stereoscopic image re-photographing apparatus.
As shown in FIG. 1, the stereoscopic image re-photographing device 2 re-photographs an image (first element image group) obtained by photographing the subject as a plurality of element images, which is photographed by the stereoscopic image photographing device 1, and displays a stereoscopic image. The apparatus 3 generates an image (second element image group) in which the depth of a reproduced image displayed as a stereoscopic image is changed. Here, the stereoscopic image re-photographing apparatus 2 includes a display unit 21, a photographing unit 22, and a depth control lens L.

  The display means 21 displays an image (first element image group) captured by the stereoscopic image capturing device 1 as an inverted image for each element image, and includes an element image group display unit 21a, a display opening, and the like. And a group 21b.

  The element image group display unit 21a displays an image (first element image group) captured by the stereoscopic image capturing apparatus 1, and is a liquid crystal display, a plasma display, a CRT (Cathode Ray Tube), or the like.

The display opening group 21b generates an inverted image of the unevenness from the element image corresponding to the position of each element image displayed on the element image group display unit 21a, and includes a plurality of openings 21b ₁ , 21b ₂ ,..., 21b _n are arranged in a plane on the display surface of the element image group display unit 21a. The openings 21b ₁ , 21b ₂ ,..., 21b _n are pinholes or convex lenses, and here, a single convex lens is used. The light that has passed through the display aperture group 21b is incident on a depth control lens L described later.

  The photographing unit 22 receives light incident from the depth control lens L and photographs each element image, and includes a photographing aperture group 22a and an element image group photographing unit 22b.

Photographic aperture group 22a is, there is for focusing the light incident from the depth control lens L, a plurality of openings _{22a 1,} 22a 2, ..., a 22a _n, shooting later elements image group captured portion 22b A plurality of flat surfaces are arranged on the surface. The opening _{22a 1, 22a 2, ...,} 22a n is a pinhole or a convex lens, where is directed to the use of convex lens. The light that has passed through the photographing aperture group 22a is incident on the elemental image group photographing unit 22b.

The elemental image group photographing unit 22b receives and shoots the light collected through the photographing aperture group 22a and is a CCD (Charge Coupled Device), a photographic film, or the like. In the element image group captured portion 22b, the opening _{22a 1,} 22a 2 of the photographic aperture groups 22a, ..., the number of element images corresponding to 22a _n (second element image group) is to be captured. The element image group (second element image group) photographed by the element image group photographing unit 22b is displayed on the stereoscopic image display device 3, so that the subject is reproduced as a stereoscopic image. At this time, since the stereoscopic image generated by the display aperture group 21b (the unevenness is inverted) is reproduced again by the stereoscopic image display device 3, the unevenness of the depth is correctly reproduced.

The depth control lens L is disposed between the display unit 21 and the photographing unit 22, and controls the depth when the element image photographed on the photographing surface of the element image group photographing unit 22b is displayed as a stereoscopic image. is there. A convex lens can be used as the depth control lens L, and at least a lens having a diameter larger than that of the display means 21 and the photographing means 22 is used.
Since this depth control lens L refracts incident light, the light of the element image group displayed by the display means 21 is refracted according to the refractive index, the radius of curvature, etc. of the depth control lens L and is taken by the photographing means 22. Will be filmed. That is, individual element images photographed on the photographing surface of the photographing means 22 are photographed while being shifted in the optical axis direction of the depth control lens L. As a result, when the group of element images photographed by the photographing means 22 is displayed on the stereoscopic image display device 3, it is generated and visually recognized before the actual subject position. For example, when the refractive index of the depth control lens L is the same, the position of the depth control lens L is positioned closer to the display unit 21 so that the position of the reproduced image displayed as a stereoscopic image is closer to the observation direction. Can be played back. A method for controlling the depth of the stereoscopic image by the depth control lens L will be described in detail later with reference to FIG.

  By configuring the stereoscopic image re-imaging device 2 in this way, an image with a controlled depth can be generated from an image (element image group) for displaying a stereoscopic image. Further, according to this configuration, since it is not necessary to provide the depth control lens in the stereoscopic image capturing device 1, the stereoscopic image capturing device 1 can be reduced in size and weight.

[Details of depth control method]
Next, with reference to FIG. 2, a depth control method for a stereoscopic image using the stereoscopic image recapturing apparatus 2 described in FIG. 1 will be described. 2A and 2B are schematic diagrams for explaining a depth control method of a stereoscopic image according to the present invention, in which FIG. 2A shows a case where the subject is photographed, FIG. 2B shows a case where the subject is re-photographed, and FIG. Show.

Further, here, as shown in FIG. 2A, a photographing aperture group 11a in which a plurality of convex lenses are arranged in a plane, and an elemental image group photographing unit 11b that receives and photographs light that has passed through the photographing aperture group 11a. The stereoscopic image capturing apparatus 1 having the above functions captures the subject T as an image (first element image group t1).
Further, as shown in FIG. 2 (b), the stereoscopic image re-imaging device 2 displays the image (first element image group t1) photographed in FIG. 2 (a) and the image reproduced by the display aperture group 21b. It is assumed that an image (second element image group t2) subjected to depth control is generated by re-photographing.
Further, as shown in FIG. 2C, an element image group display unit 31a that displays the second element image group t2 generated in FIG. 2B, and a display aperture group 31b in which a plurality of convex lenses are arranged in a planar shape. It is assumed that the stereoscopic image display device 3 including the second element image group t2 is presented as a stereoscopic image.

Hereinafter, the depth control method of the stereoscopic image will be described together with the procedure from photographing the subject T to reproducing it as a stereoscopic image. This procedure is performed by two shooting / display steps when shooting and display are taken as one step.
[First shooting step]
First, as shown in FIG. 2 (a), the three-dimensional image photographing apparatus 1, from the principal point of the convex lens is a first photographic aperture group 11a, the distance L ₀ apart object T, the first element images t1 Shoot as.

[First display step]
Then, as illustrated in FIG. 2B, the stereoscopic image re-capturing device 2 displays an image (first element image group t1) captured by the stereoscopic image capturing device 1. At this time, when the depth control lens L is not present, the first element image group t1 displayed on the first element image group display unit 21a of the stereoscopic image re-imaging device 2 is the principal point of the first display aperture group 21b. Thus, the reproduced image T ₁ is formed at a position separated by a distance L ₀ from the image.

[Second shooting step]
Then, the stereoscopic image re-imaging device 2 photographs the light that has passed through the depth control lens L provided between the first display aperture group 21b and the second imaging aperture group 22a as the second element image group t2. . Note that the second element images t2 are the images that have passed through the depth control lens L, an image forming in front than the reproduced image T ₁ to be reproduced in a first display step that the (reconstructed image T ₂₎ It becomes.
Here, from the principal point of the depth control lens L, the distance to the reproduced image T ₁ from the principal point of L _a, the depth control lens L to be reproduced in a first display step, is reproduced by the second element images t2 Assuming that the distance to the reproduced image T ₂ is L _b and the focal length of the depth control lens L is f _d , each of L _a , L _b, and f _d satisfies the relationship of the following expression (1).

That is, from the depth control lens L, the distance L _b to the reproduced image T ₂ to be reproduced by the second element images t2 can be shown by the following equation (2).

Further, the difference L _t between the position of the reproduced image T ₁ reproduced by the first element image group t _{1 and} the position of the reproduced image T ₂ reproduced by the second element image group t 2 in the first display step is as follows: It can be shown by equation (3).

[Second display step]
Then, as illustrated in FIG. 2C, the stereoscopic image display device 3 displays an image (second element image group t2) captured by the stereoscopic image re-imaging device 2. At this time, in FIG. 2B, the reproduced image T ₃ shown in FIG. 2C is adjusted by adjusting the distance (L _d ) between the first display aperture group 21b and the second imaging aperture group 22a. Can be moved forward in the observation direction by a distance (L _s ) corresponding to the distance (L _d ). That is, the reproduced image of the subject viewed from the observation direction is displayed in front of the distance (L _s ) shown in the following expression (4) from the position of the actually photographed subject.

  As described above, in the stereoscopic image re-imaging device 2, by re-imaging the images (first element image group t1) captured by the stereoscopic image capturing device 1 and the images reproduced by the display aperture group 21b, Compared with the conventional method shown in FIG. 7, the depth can be controlled more greatly. Further, as compared with the conventional method shown in FIG. 11, it is not necessary to provide a depth control lens in the stereoscopic image capturing apparatus 1, so that the configuration of the stereoscopic image capturing apparatus 1 can be simplified.

Incidentally, in FIG. 2 (b), the from the principal point of the depth control lens L, the distance L _a to the reproduced image T ₁ to be reproduced in the first display step, becomes equal to the focal length f _d of the depth control lens L If the formula as shown in (2), from the depth control lens L, and the distance L _b to the reproduced image T ₂ to be reproduced by the second element images t2 infinity (∞), and the inability to depth control become. In this case, from the principal point of the depth control lens L in the first display step, the distance L _a to the reproduced image T ₁ is, as not equal to the focal length f _d of the depth control lens L, the position of the depth control lens L It can be avoided by adjusting.

(Preferred example of depth control method)
Next, with reference to FIG. 3, without being affected by the focal length f _d of the depth control lens L as described in Figure 2, a description is given depth control method capable of controlling the depth of the reproduced image. 3A and 3B are schematic diagrams for explaining a preferable method for controlling the depth of a stereoscopic image according to the present invention, in which FIG. 3A is a time when a subject is photographed, FIG. 3B is a time when re-photographing, and FIG. Respectively. Note that the stereoscopic image re-capturing apparatus 2B in FIG. 3 is that the depth control lens L in FIG. 3B is disposed in contact with the first display aperture group 21b, so that the stereoscopic image re-imaging in FIG. The configuration of the apparatus 2 is different. The stereoscopic image capturing apparatus 1 and the stereoscopic image display apparatus 3 are the same as those in FIG.

In the following, with regard to the depth control method of a stereoscopic image, the procedure will be described in order for two shooting / display steps in which shooting and display are performed as one step, similar to the steps described in FIG.
[First shooting step]
First, as shown in FIG. 3A, the stereoscopic image capturing apparatus 1 captures a subject as a first element image group through the first imaging aperture group 11a. At this time, there is a possibility that the position of the subject exists from the position T _{0 of} the first imaging aperture group 11a, which is the position closest to the stereoscopic image capturing apparatus 1, to the infinity T _∞, which is the farthest position. .

[First display step]
Then, as illustrated in FIG. 3B, the stereoscopic image re-capturing device 2 </ b> B displays an image (first element image group) captured by the stereoscopic image capturing device 1.
Here, as shown in FIG. 3B, since the stereoscopic image re-photographing apparatus 2B abuts the depth control lens L on the first display aperture group 21b, in FIG. A distance L ₀ from the first display aperture group 21b to the reproduced image T ₁ reproduced by the display aperture group 21b, and a reproduced image T ₂ reproduced from the depth control lens L through the depth control lens L. distance L _a to can be considered to be equal distance.

This is because when the distance L ₀ (that is, the distance L _a ) changes from immediately before the first display aperture group 21b to infinity, the depth control lens L is changed from the depth control lens L according to the equation (2). through distance L _b to reproduce image T ₂ to be reproduced is shown to vary from "0" to the focal length f _d.

[Second shooting step]
Therefore, as shown in FIG. 3B, the stereoscopic image re-imaging device 2B captures the image displayed in the first display step as a second element image group via the depth control lens L, the second element set of images taken, the main point of the depth control lens L from (distance "0"), the reproducible images by the focal length f _d.

[Second display step]
And as shown in FIG.3 (c), the stereoscopic image display apparatus 3 displays the image image | photographed with the stereoscopic image re-imaging apparatus 2B.
As a result, the reproduced image of the subject located on the first photographing aperture group 11a in FIG. 3A is reproduced from the second display aperture group 31b to the position in front of the observation direction in FIG. 3C. The At this time, the distance from the second display aperture group 31b to the reproduced image is equal to the distance L _{I, 0} from the second imaging aperture group 22a to the depth control lens L in FIG.

In addition, the reproduced image of the subject existing at infinity in FIG. 3A is reproduced from the second display aperture group 31b to the back in the observation direction in FIG. 3C. At this time, the distance from the second display aperture group 31b to the reproduced image is the distance L _I, from the second imaging aperture group 22a to the reproduced image reproduced through the depth control lens L in FIG _{. Equal} to _∞ .

As described above, in the stereoscopic image re-photographing apparatus 2B, the depth control lens L is disposed in contact with the first display aperture group 21b, so that a reproduced image of a subject existing from the distance “0” to infinity is obtained. and it can be reproduced within a focal length f _d of the depth control lens L.
Here, the stereoscopic image re-photographing apparatuses 2 and 2B perform depth control of the reproduced image by a lens system such as the depth control lens L, but the present invention performs depth control by an arithmetic circuit or the like equivalent to the optical system. It may be done.

[Steric image depth converter]
Next, a stereoscopic image depth conversion device according to an embodiment of the reference example will be described with reference to FIG. FIG. 4 is a block configuration diagram showing the configuration of the stereoscopic image depth conversion apparatus. The stereoscopic image depth conversion device 4 is assumed to be an equivalent device to the stereoscopic image re-photographing device 2B shown in FIG. 3, and refer to FIG. 5 as appropriate. The element image group (first element image group) input to the stereoscopic image depth conversion device 4 is a video signal captured by the CCD or the like in the stereoscopic image capturing device 1 (see FIG. 1).

  As shown in FIG. 4, the stereoscopic image depth conversion device 4 performs processing by the optical system in the stereoscopic image re-imaging device 2 </ b> B described in the first embodiment by calculation. Here, the stereoscopic image depth conversion device 4 includes a distribution unit 41, an element image conversion unit 42 for the element image, a depth conversion unit 43, an element image reproduction unit 44, and an addition unit 45.

The distribution unit 41 divides the input video signal into element images. Here, the distribution unit 41 outputs the light wave (g _{s, m} (x _s, x _{s, m)} of the m-th element image of the first element image group t1 displayed on the display surface of the element image group display unit 21a in the input video signal _{. m} , y _{s, m} )) is output to the element image conversion means 42 previously associated with the m-th element image. Here, the light wave is a complex number representing the amplitude and phase when a video signal is handled as a wave of light.

  Element image conversion means (first element image light wave conversion means) 42 converts the light wave of the element image into a light wave when passing through an element lens (virtual pinhole or lens) having a predetermined focal length. is there. Here, the element image conversion means 42 includes a light wave calculation means 42a and a phase shift means 42b.

The light wave calculating means 42a calculates the light wave incident on the element lens (virtual pinhole or lens) by approximating the light wave of the element image by Fresnel approximation. That is, the light wave calculation means 42a uses the general Fresnel approximation as a signal corresponding to the light wave reaching the m-th opening (element lens or the like) of the first display opening group 21b, and uses the following formula (5 ) To calculate the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each element image.

Here, x _{s, m} , x _{o, m} are the x coordinate of the center of the m-th element image in the entire image (first element image group) and the optical axis center of the m-th opening of the display aperture group 21b, respectively. X coordinate from. Y _{s, m} , _{yo, m} are respectively the y coordinate of the center of the mth element image in the entire image (first element image group) and the optical axis center of the mth opening of the display opening group 21b. Y coordinate. F ₁ is the focal length of the aperture (element lens) of the first display aperture group 21b, and k is the wave number 2π / λ (λ is the wavelength).
The light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each element image is output to the phase shift means 42b.

The phase shift unit 42b shifts the phase by the phase corresponding to the virtual element lens from the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) of the element image input from the light wave calculation unit 42a. The calculated light wave is calculated. That is, the phase shift means 42b shifts the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) by the phase corresponding to the element lens as shown in the following equation (6). Then, a signal (light wave R _{o, m} (x _{o, m} , y _{o, m} )) corresponding to the light wave emitted from the element lens is calculated.

The light wave (R _{o, m} (x _{o, m} , y _{o, m} )) for each element image is output to the depth conversion means 43.

  The depth conversion unit 43 converts the light wave converted by the element image conversion unit 42 into a light wave when passing through a virtual depth control lens L (convex lens) having a predetermined focal length. Here, the depth conversion means 43 includes a phase shift means 43a and a redistribution means 43b.

The phase shift unit 43a calculates a light wave obtained by shifting the phase of the light wave converted by the element image conversion unit 42 by a phase corresponding to a virtual convex lens. That is, the phase shift means 43a shifts the light wave (R _{o, m} (x _{o, m} , y _{o, m} )) by the phase corresponding to the depth control lens L as shown in the following equation (7). Thus, a signal corresponding to the light wave emitted from the depth control lens L (light wave R _{d, m} (x _{o, m} , _{yo, m} )) is calculated.

Here, x ₀ and y ₀ are distances from the optical center of the depth control lens L. Further, when the pitch of the apertures (element lenses) of the first display aperture group 21b is P, the relationship of x _o = x _{o, m} + mP, y _o = y _{o, m} + mP is established.
The light wave (R _{d, m} (x _{o, m} , y _{o, m} )) for each element image calculated by the phase shift means 43a is output to the redistribution means 43b.

The redistribution unit 43b uses the light wave (R _{d, m} (x _{o, m} , y _{o, m} )) for each element image input from the phase shift unit 43a, so that the apertures (elements) of the second imaging aperture group 22a The light wave incident on the lens is calculated. The light wave (R _{d, m} (x _{o, m} , y _{o, m} )) calculated by the phase shift means 43a not only reaches the mth element lens but also reaches the surrounding element lenses. It will be. Therefore, the redistribution unit 43b transmits the light wave (R _{d, m} (x _{o, m} , y _{o, m} )) for each element image input from the phase shift unit 43a to the aperture of the second imaging aperture group 22a. The light wave (R _{p, n, m} (x _{p, n} , y _{p, n} )) that has reached the nth element lens resulting from being distributed to (element lens) is expressed using a general Fresnel approximation. It calculates by the following formula | equation (8).

Here, x _{p, n} and y _{p, n} are the x coordinate and the y coordinate from the optical center of the aperture (element lens) of the nth imaging aperture group 22a, respectively. L _d is the distance between the depth control lens L and the second imaging aperture group 22a.
The light wave (R _{p, n, m} (x _{p, n} , y _{p, n} )) for each aperture of the photographing aperture group 22 a is output to the element image reproducing means 44.

  The element image reproduction means (second element image light wave conversion means) 44 is a light wave when the light wave converted by the depth conversion means 43 passes through an element lens (virtual pinhole or lens) having a predetermined focal length. It is to convert to. Here, the element image reproduction means 44 includes a phase shift means 44a and a light wave calculation means 44b.

The phase shift means 44a shifts the phase from the light wave (R _{p, n, m} (x _{p, n} , y _{p, n} )) converted by the depth conversion means 43 by the phase corresponding to the virtual element lens. The light wave is calculated. That is, the phase shift means 44a shifts the light wave (R _{p, n, m} (x _{p, n} , y _{p, n} )) by the phase corresponding to the element lens, as shown in the following formula (9). Thus, a signal (light wave R _{r, n, m} (x _{o, m} , _{yo, m} )) corresponding to the light wave emitted from the element lens is calculated.

Here, f ₂ is the focal length of the opening of the second photographing aperture group 22a (element lens).
The light wave (R _{r, n, m} (x _{p, n} , y _{p, n} )) for each element lens is output to the light wave calculation means 44b.

The light wave calculation means 44b calculates the light wave which is emitted from the element lens (virtual pinhole or lens) and reaches the photographing surface of the element image group photographing unit 22b by approximating the light wave of each element lens by Fresnel approximation. is there. That is, the light wave calculation means 44b calculates the light wave (R _{e, n, m} (x _{e, n} ) emitted from the nth element lens and reaching the imaging surface of the element image group imaging unit 22b by the following equation (10). , Y _{e, n} )).

Here, x _{e, n} and y _{e, n} are the x coordinate and y coordinate from the center of the element image, respectively. The light wave (R _{e, n, m} (x _{e, n} , y _{e, n} )) calculated by the light wave calculating unit 44 b is output to the adding unit 45.

The adding means 45 calculates the sum of the light wave power from the light wave for each element image output from the element image reproducing means 44, thereby obtaining a video signal subjected to depth conversion processing, that is, the second element image group t2. Is generated.
By the way, the power of the light wave (R _{e, n, m} (x _{e, n} , y _{e, n} )) reaching the imaging surface of the element image group imaging unit 22b output from the element image reproducing means 44 is the amplitude of light. Can be represented by the square of. Moreover, since the light wave emitted as each element image of the first element image group t1 is incoherent (the wavelength and phase are not constant), the phase of the light wave can be regarded as uncorrelated.

  Therefore, the adding means 45 calculates the total power of the light waves emitted as each element image (−M to M) of the first element image group t1 by the following equation (11), thereby obtaining the second element image. The power of one element image (n-th element image in the equation (11)) of the group t2 is obtained, and a video signal proportional to the power is output.

As a result, the stereoscopic image depth conversion device 4 is equivalent to the stereoscopic image re-photographing device 2B shown in FIG. 3, and an image (second element image group) in which the depth of the reproduced image displayed as a stereoscopic image is changed by calculation. ) Can be generated.
The present invention is not limited to this configuration. For example, here, as shown in FIG. 5, the stereoscopic image depth conversion device 4 is an apparatus equivalent to the stereoscopic image re-photographing device 2B in which the depth control lens L is disposed in contact with the first display aperture group 21b. Although realized, it can be realized as an apparatus equivalent to the stereoscopic image re-imaging apparatus 2 shown in FIG.
In this case, Fresnel approximation means (not shown) for calculating the light wave incident on the depth control lens L by approximating the light wave output from the element image conversion means 42 (phase shift means 42b) to the depth conversion means 43. May be provided before the phase shift means 43a.

Here, an example is shown in which the element lens is calculated as a convex lens, but in the case of a pinhole, the phase shift function (phase shift means 42b and 44a) can be omitted. In this case, the focal lengths f ₁ and f ₂ are the distances between the pinholes and the corresponding element images.

In addition, the stereoscopic image depth conversion device 4 can realize each of the above-described means by an arithmetic circuit, and can also be used for a central processing unit (CPU: Central Processing Unit), a main storage device (RAM: Random Access Memory), and the like. It can also be realized by operating a general computer provided with a program (stereoscopic image depth conversion program) that functions as each of the above-described means.
As described above, according to the present invention, the processing from the first display step to the second photographing step is performed by calculation instead of the optical system, so that the apparatus configuration is improved in addition to the effects in the stereoscopic image re-photographing apparatuses 2 and 2B. The effect which can be reduced in size is produced.

It is a schematic diagram showing the configuration of a three-dimensional image reconstruction capturing apparatus according to implementation embodiments of the present invention. It is a schematic diagram for demonstrating the depth control method of the three-dimensional image in this invention, Comprising: (a) at the time of the object imaging | photography in a stereoscopic image imaging device, (b) at the time of re-imaging by a stereoscopic image re-imaging device, (c). Indicates a stereoscopic image display time in the stereoscopic image display device. 3A and 3B are schematic diagrams for explaining a preferable method for controlling the depth of a stereoscopic image in the present invention, in which FIG. c) shows the time of stereoscopic image display in the stereoscopic image display device. It is a block block diagram which shows the structure of the stereo image depth conversion apparatus which concerns on embodiment of a reference example . It is a schematic diagram for demonstrating the method to control the depth of a stereo image by the calculation in embodiment of a reference example . It is explanatory drawing for demonstrating the concept of the conventional IP system, Comprising: (a) has shown the time of object photography, (b) has shown the time of a three-dimensional image display. It is a schematic diagram for demonstrating the conventional depth control method of a stereo image, (a) at the time of object imaging | photography, (b) at the time of re-imaging, (c) has shown the time of a stereoscopic image display. It is a figure which shows the point object conversion of the element image for canceling the inversion of the conventional depth. It is a figure which shows typically the point-symmetric transformation using the conventional GRIN lens. It is explanatory drawing for demonstrating the concept of the IP system using the conventional GRIN lens array, Comprising: (a) is at the time of object imaging | photography, (b) has shown the time of stereoscopic image display. It is a schematic diagram for demonstrating the depth control method of the stereo image using the conventional GRIN lens array, (a) at the time of object photography, (b) at the time of re-photographing, (c) at the time of a stereo image display. Show.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Stereoscopic image imaging device 2, 2B Stereoscopic image re-imaging device 21 Display means 21a Element image group display part 21b Display aperture group 22 Imaging means 22a Imaging aperture group 22b Element image group imaging part 3 Stereo image display apparatus 4 Stereo image depth conversion apparatus 41 Distributing means 42 Element image converting means (first element image light wave converting means)
43 Depth conversion means 44 Element image reproduction means (second element image light wave conversion means)
45 Adding means L Depth control lens

Claims (2)

3D image re-shooting that re-shoots an element image group consisting of a plurality of element images that are reconstructed images with the projections and depressions of the subject reversed when displayed as a stereoscopic image A device,
An element image group display unit for displaying the element image group;
A display aperture group in which openings, which are a plurality of pinholes or lenses corresponding to the position of each element image displayed on the element image group display section, are arranged in a plane on the display surface side of the element image group display section; ,
A plurality of pinholes or lens openings arranged at a position facing the display opening group, and a photographing aperture group arranged in a plane,
Provided with this imaging aperture group on the imaging surface side, comprising an elemental image group imaging unit for imaging the elemental image group,
A depth control lens is provided between the display aperture group and the imaging aperture group to control a depth when an element image captured by the element image group imaging unit is displayed as a stereoscopic image. 3D image re-shooting device.   The stereoscopic image re-imaging device according to claim 1, wherein the depth control lens is a convex lens, and the convex lens is disposed in contact with the display aperture group.

Images