JP5767500B2 - Stereoscopic image correction apparatus and program thereof, and stereoscopic image display apparatus - Google Patents

Description

  The present invention relates to a technique for generating an image for displaying a stereoscopic image and a technique for displaying a stereoscopic image, and in particular, generates an image for displaying an IP (Integral Photography) type stereoscopic image. The present invention relates to a stereoscopic image correction apparatus, a program thereof, and a stereoscopic image display apparatus for displaying a stereoscopic image.

  In general, among the methods for acquiring three-dimensional information of a subject through a lens array or a spatial filter, an IP method is known as a method for capturing and displaying a stereoscopic image using a minute optical element array.

  Here, normal stereoscopic image capturing based on the IP system will be described with reference to FIGS. 8A and 8B. The stereoscopic image capturing apparatus illustrated in FIG. 8A captures an object 111 from a capturing direction 114 indicated by an arrow through a lens group 112 including, for example, a convex lens. Here, the photographing direction 114 is a direction in which the stereoscopic image photographing device photographs the subject 111 arranged in front of the lens group 112 (leftward in FIG. 8A). At this time, an image of the object 111, for example, an image 115, is formed on the imaging plate 113 behind the lens group 112 (right side in FIG. 8A) as many as the number of convex lenses constituting the lens group 112. . Here, the imaging plate 113 is an information acquisition device configured to include a plurality of imaging elements arranged on a substrate. Each image sensor is, for example, a CCD (Charge Coupled Device) image sensor.

  FIG. 8B shows a stereoscopic image display device including a lens group 122 in which convex lenses are arranged in one plane and a display element 123, a stereoscopic image 121, an observation direction 124, and an image 125 of the lens group. . Here, the display element 123 displays an image 125 corresponding to the image 115 photographed by the photographing plate 113 of the stereoscopic image photographing device shown in FIG. The display element 123 is composed of an information display device such as a liquid crystal panel. As a result of the display from the stereoscopic image display device, as shown in FIG. 8B, a stereoscopic image 121 is generated at the same place as the place where the subject 111 exists. However, the stereoscopic image 121 is generated as a reverse view image. That is, as shown in FIG. 8A, when viewed from the shooting direction 114, the cylinder of the subject 111 is present in front of the prism. However, as shown in FIG. 8B, in the stereoscopic image 121 corresponding to the subject 111, a prism is generated in front of the cylinder as viewed from the observation direction 124.

  In FIGS. 8A and 8B, the lens group is displayed as a minute optical element array, and the operation has been described on the assumption that the three-dimensional information of the subject is acquired and displayed using this lens group. As a minute optical element array, a minute aperture array (spatial filter) may be used.

In FIG. 8A, the size (height) of the prism of the subject 111 is xc, the distance from the lens group 112 to the prism of the subject 111 is zc, and the distance from the lens group 112 to the surface on which the image 115 is photographed. Let dc be the pitch of the convex lens that constitutes the lens group 112, and pc be the size (height) of the image of the subject that is generated by the convex lens that constitutes the lens group 112. 8B, the size (height) of the prisms of the stereoscopic image 121 is xr, the distance from the lens group 122 to the prisms of the stereoscopic image 121 is zr, and the surface on which the image 125 is displayed from the lens group 122. distance d _{r up,} pr a convex lens pitch of the lens group 122, the size of each image displayed on the display device 123 (height) and kr. In this case, the relationship between xc and xr is expressed by Equation (101) by Non-Patent Document 1.

From the relationship of the above formulas (102a) to (102c), the pitch pc of the convex lenses constituting the lens group 112 of the stereoscopic image capturing apparatus shown in FIG. 8A and the stereoscopic image shown in FIG. It is derived that the ratio xr / xc of the size of the stereoscopic image 121 with respect to the subject 111 can be controlled by making the pitch pr of the convex lenses constituting the lens group 122 of the display device different. Similarly, the size kc image 115 of the subject photographed, by varying the magnitude kr image 125 to be displayed, controls the magnitude of the ratio xr / _{x c} of the three-dimensional image 121 with respect to the subject 111 It is guided that we can do it.

  Conventionally, in order to solve the problem of generating a reverse-view image in which the depth is inverted compared to the subject, the information acquired by the stereoscopic image capturing apparatus shown in FIG. A stereoscopic image depth conversion device that inputs the information after the input to the stereoscopic image display device shown in FIG. 8B and finally generates a stereoscopic image with the correct depth is disclosed (see Patent Document 1).

  A stereoscopic image depth conversion device disclosed in Patent Document 1 will be described with reference to FIGS. 9 (a) and 9 (b). FIG. 9A shows the image 131 acquired in FIG. 8A, the first virtual lens array 132, the virtually generated stereoscopic image 133, and the second virtual lens array 134. , An image 135 (virtually generated stereoscopic image 133) generated by the second virtual lens array 134. Here, the image 135 is not optically generated, but is generated by calculation processing of the stereoscopic image depth conversion device.

  FIG. 9B shows an image 141 generated by the arithmetic processing in FIG. 9A (that is, the image 135 in FIG. 9A), a lens group 142 in which convex lenses are arranged in a single plane, and a three-dimensional image. 143, the viewing direction 144. As shown in FIG. 9B, for example, as a result of the operation of the stereoscopic image display apparatus of FIG. 8B, the cylinder is observed in front of the prism as viewed from the observation direction 144. The stereoscopic image 143 is equivalent to the depth relationship of the subject in FIG.

  If a method described in Patent Document 1 is combined with a method for controlling the ratio of the size of a stereoscopic image derived from the relationship of the expressions (102a) to (102c) described above, a problem of generating a reverse-view image It is possible to generate a stereoscopic image with the correct depth while solving the above. However, in this case, in order to change the ratio xr / xc of the size of the stereoscopic image 121 with respect to the subject 111, the optical system apparatus must be changed according to the ratio xr / xc. There are difficulties. Therefore, conventionally, there has been disclosed a stereoscopic image conversion device that controls the ratio of the reproduced image that is the same as the unevenness of the subject using arithmetic processing without depending on an optical device such as a stereoscopic image capturing device or a stereoscopic image display device. (See Patent Document 2).

JP 2007-114483 A JP 2009-272922 A

J. Optical Society of America A, Vol. 21, pp.951-958, 2004

  In this way, by using the stereoscopic image conversion method as described in Patent Document 1 or Patent Document 2, the stereoscopic information acquired by the stereoscopic image capturing device is displayed on the stereoscopic image display device using arithmetic processing. When it is done, it can be converted to a desired reproduced image.

However, as pointed out in Non-Patent Document 1, if there is a relative displacement between the lens group of the stereoscopic image capturing device and the lens group of the stereoscopic image display device, the image quality of the reproduced image is reduced. It will cause deterioration.
For example, the position of the convex lens 112b1 constituting the lens unit 122 of the stereoscopic image display device shown in three-dimensional imaging apparatus lens group 112 and 10 of (b), the lens group 112 shown in FIG. 10 (a), the lens In the group 122, the position of the convex lens 122b1 provided corresponding to the convex lens 122b1 is shifted. In such a case, when a stereoscopic image obtained by photographing a subject through the lens group 112 with the stereoscopic image capturing device is displayed as a reproduced image through the lens group 122 with the stereoscopic image display device, it is emitted from the convex lens 122b1 of the stereoscopic image display device. The direction of the emitted light beam 121b1 deviates from the direction of the light beam emitted from the other convex lens, so that the stereoscopic image 121 is blurred or blurred, and the image quality of the stereoscopic image 121 is deteriorated. Become.

This image quality degradation may occur even when a stereoscopic image having a correct depth is generated by arithmetic processing, as in the method described in Patent Document 1. For example, FIG. 11 (a), the as shown in FIG. 11 (b), the first, second virtual lens array 132, the relative positional deviation between the lens group 142b has occurred In this case, the traveling direction of the light beam 143b1 emitted from the lens 142b1 constituting the lens group 142b shown in FIG. 11B is deviated from the imaging position of the stereoscopic image 143 composed of the light beams emitted from other lenses. As a result, the image quality of the stereoscopic image is deteriorated.
Further, this image quality degradation may also occur when a stereoscopic image changed from the size of the subject is generated by arithmetic processing, as in the method described in Patent Document 2.

  In view of such circumstances, the present invention is effective when there is a relative misalignment between the lens group of the stereoscopic image capturing apparatus used for photographing the subject and the lens group of the stereoscopic image display apparatus used for displaying the subject. An image that becomes a reconstructed image in which the relative positional deviation is corrected when the stereoscopic information acquired by the stereoscopic image photographing device is displayed as a stereoscopic image using arithmetic processing without depending on the optical device. It is an object of the present invention to provide a stereoscopic image correction apparatus that can be converted into information, a program thereof, and a stereoscopic image display apparatus.

The present invention has been developed to achieve the above-described object. First, the stereoscopic image correction apparatus according to claim 1 is obtained by photographing a subject through a first element optical lens group. element images corrected, a stereoscopic image correction device for generating a second component image group to be displayed as a stereoscopic image through the second lens component group, and the first arrangement information acquisition unit, a distribution unit, the first An element image conversion unit, an addition unit, a second arrangement information acquisition unit, a redistribution unit, and a second element image conversion unit are provided.

According to such a configuration, the stereoscopic image correction apparatus acquires the arrangement information of the respective first element optical lenses constituting the first element optical lens group by the first arrangement information acquisition unit.
Next, the stereoscopic image correction device distributes the first element image group for each element image according to the arrangement information of each first element optical lens acquired by the first arrangement information acquisition means by the distribution means . As this, when distributing the first element images, by using the arrangement information of each of the first lens component constituting the first lens component group of the three-dimensional image photographing apparatus, the first element image group Can be accurately distributed for each element image belonging to the element image area of each first element optical lens.

Further, the stereoscopic image correction apparatus separates light wave information representing each elemental image distributed by the distributing unit as light waves by the first elemental image converting unit at a predetermined distance from the image plane of the elemental image. Then, it is converted into light wave information representing a wave of light that has passed through a first virtual aperture group composed of an optical element or a spatial filter for each element image virtually arranged and propagated to a predetermined distance. . Each element image input to the first element image conversion means is divided into the first element image group according to the element image area of each first element optical lens constituting the first element optical lens group. It was obtained by distributing. Thus, the first elemental image converting means, the light wave information of each element image distributed by the distribution means, by converting the lightwave information passing through the first virtual aperture groups, the light wave of each element image information, though, performs arithmetic processing for converting the lightwave information representing the wave of light passing through the first lens component groups.
Further, the stereoscopic image correction apparatus adds the light wave information of each element image converted by the first element image conversion unit by the adding unit by the distributed number. Therefore, each element image of the first element image group is changed by the first element image conversion means, and the intermediate generation stage state of the element image group is obtained.

Still further, the stereoscopic image correction device acquires the arrangement information of each second element optical lens constituting the second element optical lens group by the second arrangement information acquisition unit.
Next, the stereoscopic image correction apparatus uses the redistribution unit to add the light wave information added by the addition unit according to the arrangement information of each second element optical lens acquired by the second arrangement information acquisition unit, to the second element image. Redistribution is performed for each element image constituting the group.

Then, the stereoscopic image correction apparatus uses the second element image conversion unit to optically distribute the light wave information of each element image distributed by the redistribution unit on the image surface of the element image. The light wave information is converted into light wave information representing a wave of light that has propagated through a second virtual aperture group composed of an element or a spatial filter until the image is formed. Therefore, by passing through the second virtual aperture group, as a result of the change in each element image, the state of the re-photographed element image is obtained. Here, elements lightwave information element image input to the second element image converting means, the re-distribution means described above, the light wave information are added by the adding means element image in accordance with the arrangement information of the second lens component group Redistributed for each image. Therefore, the light image information of the element image is converted by the second element image conversion means into light wave information representing the wave of light that has passed through the second virtual aperture group, and is captured by the stereoscopic image capturing device. Arithmetic processing is executed as if the first element image group was optically displayed on the second element optical lens of the stereoscopic image display device and re-photographed.

Then, the stereoscopic image correction apparatus generates the second element image group by combining the light wave information of the element images converted by the second element image conversion unit by the redistributed number by the combining unit. As a result, it is possible to generate a second element image group that is a reproduced image with good image quality without blurring or blurring when displayed as a stereoscopic image.

The stereoscopic image correction program according to claim 2 corrects the first element image group acquired by photographing the subject through the first element optical lens group, and displays it as a stereoscopic image through the second element optical lens group. In order to generate the second element image group, the computer is connected to a first arrangement information acquisition unit, a distribution unit, a first element image conversion unit, an addition unit, a second arrangement information acquisition unit, a redistribution unit, and a second element image conversion. It was made to function as a means and a coupling means.

According to such a configuration, the stereoscopic image correction program acquires the arrangement information of the respective first element optical lenses constituting the first element optical lens group by the first arrangement information acquisition unit.
Next, the stereoscopic image correction program distributes the first element image group for each element image according to the arrangement information of each first element optical lens acquired by the first arrangement information acquisition means by the distribution means.
In addition, the stereoscopic image correction program separates light wave information representing each element image distributed by the distribution unit as light waves by the first element image conversion unit at a predetermined distance from the image plane of the element image. Then, it is converted into light wave information representing a wave of light that has passed through a first virtual aperture group composed of an optical element or a spatial filter for each element image virtually arranged and propagated to a predetermined distance. .
Further, the stereoscopic image correction program adds the light wave information of each elemental image converted by the first elemental image converting unit by the adding unit by the distributed number.
Furthermore, the stereoscopic image correction program acquires the arrangement information of each second element optical lens constituting the second element optical lens group by the second arrangement information acquisition unit.
Next, the stereoscopic image correction program uses the redistribution means to add the light wave information added by the addition means according to the arrangement information of each second element optical lens acquired by the second arrangement information acquisition means, to the second element image. Redistribution is performed for each element image constituting the group.
Then, the stereoscopic image correction program is configured to calculate the optical wave information for each element image obtained by virtually arranging the light wave information of each element image distributed by the redistribution means on the image plane of the element image by the second element image conversion means. The light wave information is converted into light wave information representing a wave of light that has propagated through a second virtual aperture group composed of an element or a spatial filter until the image is formed.
Then, the stereoscopic image correction program generates a second element image group by combining the light wave information of the element images converted by the second element image converting unit by the redistributed number by the combining unit.

The stereoscopic image display apparatus according to claim 3 corrects the first element image group acquired by photographing the subject through the first element optical lens group, and displays the corrected image as a stereoscopic image through the second element optical lens group. A stereoscopic image display apparatus for generating a second element image group to be displayed and displaying the second element image group as a stereoscopic image, wherein the first arrangement information acquisition means, the distribution means, the first element image conversion means, And adding means, second arrangement information acquisition means, redistribution means, second element image conversion means, combining means, display elements, and a second element optical lens group.

According to such a configuration, the stereoscopic image display device acquires the arrangement information of the respective first element optical lenses constituting the first element optical lens group by the first arrangement information acquisition unit.
Next, in the stereoscopic image display device, the distribution unit distributes the first element image group for each element image according to the arrangement information of each first element optical lens acquired by the first arrangement information acquisition unit.
Further, the stereoscopic image display apparatus separates the light wave information representing each element image distributed by the distribution unit as a light wave by the first element image conversion unit at a predetermined distance from the image plane of the element image. Then, it is converted into light wave information representing a wave of light that has passed through a first virtual aperture group composed of an optical element or a spatial filter for each element image virtually arranged and propagated to a predetermined distance. .
Further, the stereoscopic image display device adds the light wave information of each element image converted by the first element image conversion unit by the number of the distribution by the adding unit.
Still further, the stereoscopic image display device acquires the arrangement information of each second element optical lens constituting the second element optical lens group by the second arrangement information acquisition unit.
Next, the stereoscopic image display device uses the redistribution unit to add the light wave information added by the addition unit according to the arrangement information of each second element optical lens acquired by the second arrangement information acquisition unit, to the second element image. Redistribution is performed for each element image constituting the group.
Subsequently, the stereoscopic image display device uses the second element image conversion unit to add the light wave information of each element image distributed by the redistribution unit to each element image virtually arranged on the image plane of the element image. The light wave information is converted into light wave information representing a wave of light that has propagated through a second virtual aperture group constituted by an optical element or a spatial filter until the image is formed.
Then, the stereoscopic image display device generates the second element image group by combining the light wave information of the element images converted by the second element image conversion unit by the redistributed number by the combining unit.
Then, the stereoscopic image display device displays the second element image group generated by the combining unit by the display element, and the second element image group output from the display element by the second element optical lens group. Pass through to form a reconstructed image. According to this, it is possible to display a reproduced image with good image quality without blurring or blurring.

The present invention has the following effects.
According to the first aspect of the present invention, the image quality is displayed when a stereoscopic image is displayed due to the relative displacement between the first element optical lens group of the stereoscopic image capturing device and the second element optical lens group of the stereoscopic image display device. By correcting the first element image group that becomes a reproduced image with deteriorated in accordance with the relative positional deviation, the second element image group that becomes a reproduced image with good image quality when displayed as a three-dimensional image is obtained. Can be generated.
According to the second aspect of the present invention, the computer in which the stereoscopic image correction program is installed can achieve the same effects as the stereoscopic image correction apparatus by realizing each function based on this program. .
According to the third aspect of the present invention, the stereoscopic image display device is converted into a stereoscopic image by a relative positional shift between the first element optical lens group of the stereoscopic image photographing device and the second element optical lens group of the stereoscopic image display device. By correcting the first element image group, which is a reproduced image with degraded image quality when displayed, according to this relative positional shift, a reproduced image with good image quality is displayed when displayed as a stereoscopic image. A second element image group can be generated and this reproduced image can be displayed.

It is a block diagram which shows typically an example of the stereo image correction apparatus which concerns on embodiment of this invention. (A) is a figure which shows a part of structure of the 1st element optical lens group of a stereo image imaging device, (b) shows a part of structure of the 2nd element optical lens group of a stereo image display apparatus. FIG. It is explanatory drawing which shows typically an example of the virtual opening group assumed by the arithmetic processing of the stereo image correction apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically an example of the integration range used by the arithmetic processing of the stereo image correction apparatus which concerns on embodiment of this invention. (A) is explanatory drawing which shows a mode that a to-be-photographed object is image | photographed with a stereo image imaging device typically, (b) concerns on embodiment of this invention from the 1st element image group acquired by (a). It is explanatory drawing which shows typically a mode that it displays as a reproduced image with a stereo image display apparatus, after producing | generating a 2nd element image group with a stereo image correction apparatus. It is explanatory drawing which shows typically the other example of the virtual aperture group assumed by the arithmetic processing of the stereo image correction apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically the further another example of the virtual opening group assumed by the arithmetic processing of the stereo image correction apparatus which concerns on embodiment of this invention. It is explanatory drawing which shows typically imaging | photography and reproduction | regeneration of the stereo image by the conventional IP system, (a) has shown the stereo image imaging device, (b) has each shown the stereo image display apparatus. In FIG. 8, it is explanatory drawing which shows typically the case where the relative position shift has arisen between the lens group of a stereo image imaging device, and the lens group of a stereo image display apparatus. It is explanatory drawing which shows the method of avoiding the conventional reverse vision typically, (a) is the stereo image depth conversion apparatus which converts the information acquired by imaging | photography, (b) displays the converted image. A stereoscopic image display device is shown. In FIG. 10, it is explanatory drawing which shows typically the case where the relative position shift has arisen between the lens group of a stereo image imaging device, and the lens group of a stereo image display apparatus.

[Outline of configuration of stereoscopic image correction apparatus]
First, an outline of the configuration of the stereoscopic image correction apparatus according to the present embodiment will be described with reference to FIGS. 1 and 2. The stereoscopic image correction apparatus 10 according to the present embodiment includes a first element image group acquired by photographing a subject through the first element optical lens group illustrated in FIG. In the stereoscopic image display device 30, the light wave is converted into a second element image group used for displaying as a reproduced image after passing through the second element optical lens group shown in FIG.

  Here, as shown in FIG. 2A, the first element optical lens group of the stereoscopic image capturing apparatus 20 is configured by arranging a plurality of first element optical lenses on the same plane. As shown in FIG. 2B, the second element optical lens group of the stereoscopic image display device 30 is configured by arranging a plurality of second element optical lenses on the same plane. Here, as shown in FIGS. 2A and 2B, the first element optical lens group is configured by arranging the first element optical lenses without gaps, while the second element optical lens group is A gap δ is formed between the second element optical lens arranged in the first row and the second element optical lens arranged in the second row, so that the second element optical lens shown in FIG. There is a relative displacement between the one-element optical lens group and the second-element optical lens group shown in FIG.

As described above, the stereoscopic image correction apparatus 10 of the present embodiment has a relative positional shift between the first element optical lens group of the stereoscopic image capturing apparatus 20 and the second element optical lens group of the stereoscopic image display apparatus 30. When the image is displayed as a stereoscopic image, the light wave of the first element image group that becomes a deteriorated reproduced image is corrected according to this relative positional shift, and when displayed as a stereoscopic image, A second element image group that is a reproduced image with good image quality is generated.
More specifically, the stereoscopic image correction apparatus 10 passes through a first element optical lens group in which a plurality of first element optical lenses forming an image of a subject are arranged on the same plane in the stereoscopic image photographing apparatus 20. The light wave of the first element image group acquired by photographing the subject is passed through the second element optical lens group in which a plurality of second element optical lenses are arranged on the same plane in the stereoscopic image display device 30. This is converted into a second element image group used for display as a reproduced image.

  The first element image group input to the stereoscopic image correction apparatus 10 is a video signal captured by an imaging element such as a CCD in the stereoscopic image capturing apparatus 20. Then, the second element image group generated by the stereoscopic image correction apparatus 10 is displayed on an information display device such as a liquid crystal panel in the stereoscopic image display apparatus 30. As the stereoscopic image capturing device 20 and the stereoscopic image display device 30, for example, the configurations illustrated in FIGS. 8A and 8B can be employed.

  The stereoscopic image correction apparatus 10 includes, for example, a calculation device such as a CPU (Central Processing Unit), a storage device (storage means) such as a memory and a hard disk, and an input device that detects input of information from the outside such as a mouse and a keyboard. An interface device that transmits and receives various types of information via a communication line such as a LAN (Local Area Network), a computer having a display device such as an LCD (Liquid Crystal Display), and a program installed in the computer Is done. The stereoscopic image correction apparatus 10 includes a first arrangement information acquisition unit 11 and a second arrangement shown in FIG. 1 as a result of the hardware device and software cooperating to control the hardware resources by a program. The arrangement information acquisition unit 12, the distribution unit 13, the first element image conversion unit 14, the addition unit 15, the redistribution unit 16, the second element image conversion unit 17, and the combination unit 18 are realized.

  The first arrangement information acquisition unit 11 acquires the arrangement information of each first element optical lens constituting the first element optical lens group of the stereoscopic image capturing device 20. Further, the second arrangement information acquisition unit 12 acquires arrangement information of each second element optical lens constituting the second element optical lens group of the stereoscopic image display device 30. The arrangement information here is, for example, an element image area for each first element optical lens (second element optical lens). However, in the distribution means 13 (redistribution means 16) to be described later, when the size of the first element optical lens (second element optical lens) is known, the position information of the first element optical lens (second element optical lens) It may be.

  The distribution unit 13 outputs the light wave of the first element image group for each element image according to the arrangement information of each first element optical lens constituting the first element optical lens group acquired by the first arrangement information acquisition unit 11. To distribute. Here, the distributed element image is identified by m (−M ≦ m ≦ M). The first element image conversion means 14 functions for each distributed (2M + 1) element images. The distributed element image m is then added by the adding means 15. Then, the redistribution unit 16 distributes the added element image again in accordance with the arrangement information of the respective second element optical lenses constituting the second element optical lens group acquired by the second arrangement information acquisition unit 12. . At this time, the redistributed element images are identified by n (−N ≦ n ≦ N). The second element image converting means 17 functions for each (2N + 1) element images that have been redistributed. The redistributed element images n are then combined by the combining means 18 to generate a second element image group.

[Outline of virtual aperture group assumed in calculation processing of stereoscopic image correction apparatus]
Next, an outline of a virtual aperture group assumed in the calculation process of the stereoscopic image correction apparatus 10 of the present embodiment will be described with reference to FIG. 3 (refer to FIG. 1 as appropriate). The first element image conversion means 14 shown in FIG. 1 propagates the light wave of the element image distributed by the distribution means 13 through the first virtual opening group 41 to the second virtual opening group 42. Convert to light wave. Here, the light wave is a complex number representing the amplitude and phase when a video signal is handled as a wave of light. In addition, the elements (openings) constituting the aperture group mean optical elements such as lenses and spatial filters such as pinholes. Hereinafter, the aperture is represented by a convex lens (element lens). Further, the first virtual opening group 41 and the second virtual opening group 42 are separated by a predetermined distance L. Further, if the first virtual aperture group 41 is a virtual lens array, the predetermined distance L can be the focal length of the element lens. Note that the first and second virtual aperture groups 41 and 42 do not actually exist, and are assumed for the stereoscopic image correction apparatus 10 to perform arithmetic processing.

In FIG. 3, the light waves of the first element image group are emitted from the first element image surface 43, and the planar first and second virtual opening groups 41 and 42 whose thickness direction is shown in FIG. , And enters the second element image plane 44 from the left to the right. In addition, k ₁ indicates an area of the m-th element image in the first element image group, and d ₁ indicates from the first element image group (first element image plane 43) to the first virtual opening group 41. Indicates distance. Similarly, k ₂ represents the areas of the n th element image in the second elemental image group, to d ₂ and the second element image group from the second virtual aperture group 42 (second element image plane 44) Indicates the distance.

In FIG. 3, p ₁ indicates the pitch of the element lenses that form the first virtual aperture group 41, and p ₂ indicates the pitch of the element lenses that form the second virtual aperture group 42. In the present embodiment, the pitch ratio Φ represented by the following formula (1) is set as Φ = 1. That is, the pitches of the element lenses (openings) of the first and second virtual aperture groups 41 and 42 are the same. When the pitch of the element lenses (apertures) in the first virtual aperture group 41 or the second virtual aperture group 42 varies, the average value of pitches between the element lenses (apertures) or other values by representing the statistics, it shall be deemed to be a pitch p ₂ of the pitch p ₁ or the second virtual opening group 42 of the first virtual aperture group 41.

[Detailed Configuration of Stereoscopic Image Correction Device]
<First arrangement information acquisition means>
The first arrangement information acquisition unit 11 acquires the arrangement information of each first element optical lens constituting the first element optical lens group (see FIG. 2A) of the stereoscopic image capturing device 20.
For example, the first arrangement information acquisition unit 11 detects the first arrangement using the technique described in Non-Patent Document 3 (Non-Patent Document 3: Proc. SPIE, vol.6803, pp.68030G1-68030G9, 2008). Element image regions of the respective first element optical lenses constituting the element optical lens group are acquired as arrangement information. According to this technique, parallel light is incident on the first element optical lens group of the stereoscopic image capturing device 20, and the position of the point image generated by each first element optical lens is detected by the centroid method. The element image area of the first element optical lens is acquired as arrangement information. The first arrangement information acquisition unit 11 may detect the arrangement information itself as described above, or may acquire the arrangement information detected by another device using the method described above.
For example, when arrangement information detected by a means (not shown) using the above-described method is stored in advance as metadata in a storage means (not shown), the first arrangement information acquisition means 11 stores a memory (not shown) at an appropriate timing. The arrangement information may be read from the means.

  Further, for example, when the stereoscopic image capturing device 20 is a camera of a broadcast station, the broadcast station side acquires in advance arrangement information in each camera, and at the time of distribution of the broadcast program, the camera that captured the broadcast program The arrangement information may be multiplexed and transmitted. For example, when the stereoscopic image display device 30 is a television receiver and the stereoscopic image correction device 10 is incorporated in the television receiver, when the stereoscopic image display device 30 receives a broadcast program, the broadcast program is displayed. The first arrangement information acquisition unit 11 may acquire the arrangement information multiplexed in the first and second arrangement information.

Further, for example, the arrangement of the first element optical lenses constituting the first element optical lens group of the stereoscopic image photographing apparatus 20 that has photographed the moving picture or still picture in the data format of the moving picture or still picture on the broadcasting station side. The information may be embedded, and the first arrangement information acquisition unit 11 may acquire the arrangement information downloaded together with the moving image or the still image when the moving image or the still image is downloaded in the stereoscopic image display device 30.
The first arrangement information acquisition unit 11 outputs the arrangement information acquired in this way to the distribution unit 13.

<Second arrangement information acquisition means>
The second arrangement information acquisition unit 12 acquires arrangement information of each second element optical lens constituting the second element optical lens group (see FIG. 2B) of the stereoscopic image display device 30.
The second arrangement information acquisition means 12 is, for example, the same as described in the first arrangement information acquisition means 11, Non-Patent Document 3 (Non-Patent Document 3: Proc. SPIE, vol.6803, pp.68030G1-68030G9, 2008), element image regions of the respective second element optical lenses constituting the second element optical lens group are acquired as arrangement information. The second arrangement information acquisition unit 12 may detect the arrangement information itself as described above, or may acquire the arrangement information detected by another apparatus using the method described above. For example, when arrangement information detected by means (not shown) using the method described above is stored in advance as metadata in a storage means (not shown), the second arrangement information acquisition means 12 stores information (not shown) at an appropriate timing. The arrangement information may be read from the means.

Further, for example, the arrangement information of the second element optical lens constituting the second element optical lens group is measured in advance for each stereoscopic image display device 30 in the manufacturer, and this arrangement information is stored in the own device for each stereoscopic image display device 30. When the information is stored in advance in the storage means (not shown), the second arrangement information acquisition means 12 may read the arrangement information from the storage means (not shown) at an appropriate timing. Alternatively, when the manufacturer of the stereoscopic image display device 30 has the arrangement information for each of the stereoscopic image display devices 30, and the downloading of the arrangement information is permitted, the second arrangement information acquisition unit 12 determines the appropriate timing. Then, the arrangement information of the own device may be downloaded.
The second arrangement information acquisition unit 12 outputs the arrangement information of each second element optical lens constituting the second element optical lens group acquired in this way to the redistribution unit 16.

  1 illustrates the case where the first arrangement information acquisition unit 11 and the second arrangement information acquisition unit 12 each acquire arrangement information from the outside. However, the present invention is not limited to this, and as described above, It goes without saying that the 1 arrangement information acquisition unit 11 may directly acquire the arrangement information from the stereoscopic image capturing device 20 or the second arrangement information acquisition unit 12 from the stereoscopic image display device 30.

<Distributing means>
The distribution means 13 distributes the input video signal (the light wave of the first element image group) in element image units. The input video signal (the light wave emitted from the first element image plane 43) corresponds to a bundle of light waves of each element image constituting the first element image group, and this is expressed as a first element image group t1. I will do it. The distribution unit 13 distributes the first element image group t1 in the input video signal for each element image according to the arrangement information acquired by the first arrangement information acquisition unit 11. That is, the distribution unit 13 determines, for example, an area corresponding to the element image area of the m-th first element lens acquired by the first arrangement information acquisition unit 11 from the entire input image (first element image group t1). Extracted as the m-th element image. According to this, since the first element image group t1 is distributed for each element image in accordance with the arrangement information of each first element optical lens constituting the first element optical lens group, the entire input image (first element image group t1) is distributed. ) Can be accurately distributed for each element image captured by each first element optical lens.

Then, the distributing unit 13 converts the m-th element image light wave (gs _{, m} (x _{s, m} , y _{s, m} )) of the first element image group t1 in the input video signal to the m-th element image. It outputs to the 1st element image conversion means 14 matched previously with the element image. Here, x _{s, m} represents the x coordinate when the center of the m-th element image in the entire input image (first element image group) is the origin. Similarly, y _{s, m} indicates the y coordinate when the center of the m-th element image in the entire input image (first element image group) is the origin.

<First element image conversion means>
As shown in FIG. 1, the first element image conversion means 14 includes a light wave calculation means 14a, a phase shift means 14b, and a light wave calculation means 14c.

<< Light wave calculating means 14a >>
The light wave calculation means 14a calculates the light wave incident on the element lens constituting the first virtual aperture group 41 (see FIG. 3) from the light wave of the element image based on Fresnel approximation. is there. That is, the light wave calculation means 14a uses a general Fresnel approximation as a signal corresponding to the light wave reaching the m-th element lens of the first virtual aperture group 41, and calculates the element according to the following equation (2). A light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each image is calculated.

Here, x _{o, m} is the x coordinate when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Similarly, y _{o, m} is the y coordinate when the optical axis center of the mth aperture of the element lens group of the first virtual aperture group 41 is the origin. F ₁ indicates the focal length of the element lens of the first virtual aperture group 41. K is the wave number 2π / λ (λ is the wavelength of the light wave). The light wave (R _{i, m} (x _{o, m} , y _{o, m} )) for each element image is output to the phase shift means 14b.

<< Phase shift means 14b >>
The phase shift means 14b corresponds to the element lens of the first virtual aperture group 41 by converting the phase from the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) of the input element image. The light wave shifted by the phase is calculated. That is, the phase shift unit 14b converts the light wave (R _{i, m} (x _{o, m} , y _{o, m} )) into an element of the first virtual aperture group 41 as shown in the following equation (3). By shifting the phase corresponding to the lens, a signal (R _{o, m} (x _{o, m} , y _{o, m} )) corresponding to the light wave emitted from the element lens of the first virtual aperture group 41 is obtained. Calculate. This light wave (R _{o, m} (x _{o, m} , y _{o, m} )) is output to the light wave calculation means 14c.

<< Light wave calculating means 14c >>
The light wave calculation means 14c is a light wave emitted from the element lens of the first virtual aperture group 41, that is, a light wave (R _{o, m} (x _{o, m} , y _o ) of the element image input from the phase shift means 14b. _, M 2) is Fresnel approximated to calculate the light wave incident on the element lens of the second virtual aperture group 42 (see FIG. 3). In other words, the light wave calculation unit 14c uses a general Fresnel approximation as a signal corresponding to the light wave reaching the element lens of the second virtual aperture group 42, and uses the following formula (4) for each element image. A light wave (R _{d, m} (x _{d, m} , y _{d, m} )) is calculated. The light wave (R _{d, m} (x _{d, m} , y _{d, m} )) for each element image is output to the adding means 15.

In Expression (4), x _{d, m} is the incident surface of the second virtual aperture group 42 when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Is the x coordinate. Similarly _{, yd, m} is the y coordinate on the incident surface of the second virtual aperture group 42 when the optical axis center of the m-th element lens of the first virtual aperture group 41 is the origin. Indicates. L is the distance between the first virtual opening group 41 and the second virtual opening group 42. Further, the range in which the integral calculation is performed is set to be equivalent to the range in which the m-th element image (element image (m)) expands. An example of a range w in which the m-th element image in the first element image group extends is shown in FIG. In this case, a range w in which the m-th element image in the first element image group extends is expressed by Expression (5). Incidentally, k ₁ represents a region of the m-th element image in the first elemental image group, d ₁ is from the first element image group (first element image plane 43) to the first virtual opening group 41 Indicates distance.

  Note that, as described above, the light wave of the element image input to the first element image conversion unit 14 converts the first element image group t1 acquired by the stereoscopic image capturing device 20 into the first element optical element by the distribution unit 13. This is obtained by distributing each element image in accordance with the arrangement information of the first element optical lens constituting the lens group. For this reason, the first element image conversion means 14 converts, for example, the light wave of the m-th element image into a light wave corresponding to the light wave that has passed through the m-th opening of the first virtual opening group. This corresponds to converting the light wave of the m-th element image into a light wave that has passed through the m-th first element optical lens of the first element optical lens group.

<Adding means>
The adding means 15 adds the light waves of the respective element images converted by the first element image converting means 14 by the number distributed by the distributing means 13 on the incident surface of the second virtual aperture group 42. It is. The adding means 15 calculates the state in which the stereoscopic image is virtually reproduced by adding the light waves of the respective element images by the number distributed by the distributing means 13. The adding unit 15 adds light waves corresponding to the field angles of the element lenses of the first virtual aperture group 41. The adding means 15 emits a light wave (R _{d, m} (x _d, x, ₎ emitted from the element lens of the first virtual aperture group 42 and reaching the entrance surface of the element lens of the second virtual aperture group 42 _{. m} , y _{d, m} )) are added within the region of the second virtual aperture group. That is, as a signal corresponding to the light wave on the incident surface of the second virtual aperture group 42, the light wave (R _p ( x _p , y _p )) is calculated.

Here, x _p is an x coordinate when the center of the second virtual aperture group 42 is the origin, and similarly, y _p is the center of the second virtual aperture group 42 as the origin. This is the y coordinate. This light wave (R _p (x _p , y _p )) is output to the redistribution means 16.

<Redistribution means>
The redistribution means 16 redistributes the light wave added by the addition means 15 for each element image constituting the second element image group. The redistribution unit 16 uses the second arrangement information of the second element optical lens group acquired by the second arrangement information acquisition unit 12 using the light wave (R _p (x _p , y _p )) input from the addition unit 15. According to the above, it is divided into element image units. That is, for example, when the element image area of the nth second element lens is input from the second arrangement information acquisition unit 12, the redistribution unit 16 uses the nth second light from the light wave added by the addition unit 15. Light waves belonging to the area corresponding to the element image area of the element lens are extracted as the nth element image.

As a light wave for each element image obtained by redistributing the light wave (R _p (x _p , y _p )), a light wave corresponding to the n-th element lens constituting the second virtual aperture group 42 is represented by (R _{p, n} ( _{xp, n} , yp _{, n} )). Here, the redistribution means 16 converts the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) input to each element lens constituting the second virtual aperture group 42 into the second Is output to the second element image conversion means 17 that is associated in advance with the nth element lens that lies across the virtual aperture group 42.

<Second element image conversion means>
The second element image conversion means 17 converts the light wave of the element image distributed by the redistribution means 16 into a light wave that has propagated through the second virtual aperture group 42 until the image is formed. It is. In other words, the second element image conversion means 17 converts the light waves of the respective element images redistributed for each element image based on the arrangement information of the second element optical lenses by the redistribution means 16 to the respective second element optical lenses. Is converted into a light wave that has been propagated by a distance until it forms an image after passing through.

The second element image conversion means 17 redistributes the light wave input from the redistribution means 16 to the aperture region of the element lens (focal length f ₂ ) of the second virtual aperture group 42, and then It converts into a light wave that passes through the lens (focal length f ₂ ), and further propagates the light wave by the focal length f _{2 of the} element lens. The second element image conversion unit 17 functions for each (2N + 1) element images, for example, to propagate a bundle of light waves of each element image. The second element image group is composed of a bundle of light waves of the element images that propagate. This is expressed as a second element image group t2. That is, the second element image conversion unit 17 calculates the second element image group t2. As shown in FIG. 1, the second element image conversion means 17 includes a phase shift means 17a and a light wave calculation means 17b.

<< Phase shift means 17a >>
The phase shift unit 17 a converts the phase from the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) input from the redistribution unit 16 to the element lens of the second virtual aperture group 42. The light wave shifted by the corresponding phase is calculated. That is, the phase shift means 17a converts the light wave (R _{p, n} (x _{p, n} , y _{p, n} )) into an element of the second virtual aperture group 42 as shown in the following equation (7). By shifting the phase corresponding to the lens, a signal (R _{r, n} (x _{p, n} , y _{p, n} )) corresponding to the light wave emitted from the element lens of the second virtual aperture group 42 is obtained. Calculate.

Here, _{xp, n} is the x coordinate when the optical axis center of the nth element lens of the second virtual aperture group 42 is the origin. Similarly, yp _{, n} is the y coordinate when the center of the optical axis of the nth element lens of the second virtual aperture group 42 is the origin. F ₂ is the focal length of the element lens of the second virtual aperture group 42. The light wave (R _{r, n} (x _{p, n} , y _{p, n} )) for each element image is output to the light wave calculation means 17b.

<< Light wave calculating means 17b >>
The light wave calculating means 17b emits the light wave of each element lens of the second virtual aperture group 42 from the element lens of the second virtual aperture group 42 by performing Fresnel approximation, and the second element image plane The light wave reaching 44 (see FIG. 4) is calculated. That is, the light wave calculation means 17b calculates the light wave (R _{e, E)} emitted from the nth element lens of the second virtual aperture group 42 and reaching the second element image plane 44 by the following equation (8) _{. n} (x _{e, n} , y _{e, n} )) is calculated.

Here, x _{e, n} is an x coordinate from the center of the n-th element image in the entire output image (second element image group). Similarly, y _{e, n} is the y coordinate from the center of the nth element image in the entire output image. The light wave (R _{e, n} (x _{e, n} , y _{e, n} )) calculated by the light wave calculating unit 17 b is output to the coupling unit 18.

  As described above, the light wave of the element image input to the second element image conversion means 17 is changed from the light wave of the element image input from the addition means 15 to the second element optical lens group by the redistribution means 16. This is obtained by distributing for each element image in accordance with the arrangement information of the second element optical lens constituting. For this reason, the second element image conversion means 17 converts, for example, the light wave of the m-th element image into a light wave corresponding to the light wave that has passed through the m-th opening of the second virtual opening group. This corresponds to the process of displaying the light wave of the m-th element image on the m-th second element optical lens of the second element optical lens group and re-photographing.

<Coupling means>
The combining means 18 generates a second element image group by combining the light waves converted by the second element image converting means 17 by the number redistributed by the redistributing means 16. This combining means 18 calculates the sum of the powers of the light waves from the light waves for each element image output from the second element image conversion means 17, that is, a video signal in which the size of the stereoscopic image is converted, that is, Then, a video signal to be the second element image group t2 is generated. The second element image group t2 obtained by the combining unit 18 is output to the stereoscopic image display device 30 (see FIG. 1).

Here, the power of the light wave (R _{e, n} (x _{e, n} , y _{e, n} )) reaching the second element image plane 44 output from the second element image conversion means 17 is the light amplitude. It can be expressed as a square. Moreover, since the light wave emitted as each element image of the first element image group t1 is incoherent (the wavelength and phase are not constant), the phase of the light wave can be regarded as uncorrelated. Therefore, as shown in the following formula (9), the combining unit 18 uses the light wave of each element image reaching the second element image plane 44 (for example, if the element image is the n-th element image, the light wave is ( _{Re, n} (x _{e, n} , y _{e, n} )) is calculated, and the sum thereof is calculated to obtain a video signal of the entire second element image group t2.

The second element image group t2 is output to the stereoscopic image display device 30.
As shown in FIG. 5B, when the second element image group t2 output from the combining unit 18 of the stereoscopic image correction apparatus 10 is input to the stereoscopic image display apparatus 30, the stereoscopic image display apparatus 30 To display the second element image group t2. Then, the light wave of the second element image group t2 displayed on the display element passes through the second element optical lens group disposed in front of the display element, so that the light wave of the second element image group t2 converges. A reproduced image is generated at the same position where the subject exists.

Here, in the present embodiment, the pitch ratio Φ of the first and second virtual aperture groups 41 and 42 is Φ = 1. The size of the object 111 and x _c, when the size of the three-dimensional image 121 to be reproduced as x _r, the relationship between them is as the equation (101). That is, in the above equation (101), when α = γ, the relationship between the size x _c of the subject 111 and the size x _r of the reproduced stereoscopic image 121 is expressed by the following equation (10). It will be.

Therefore, the pitch p _c of the element lens in the stereoscopic image capturing device 20 (see FIG. 8A), and the pitch p _r of the element lens in the stereoscopic image display device 30 (see FIG. 1) and Are equal to each other, the size x _{c of the} subject (see FIG. 8A) is equal to the size x _r of the reproduced stereoscopic image (see FIG. 8B).

  Further, here, as shown in FIGS. 5A and 5B, between the second element optical lens group of the stereoscopic image display device 30 and the first element optical lens group of the stereoscopic image photographing device 20. A relative misalignment has occurred. However, the stereoscopic image correction apparatus 10 performs the above-described calculation process, and corrects the light wave of the first element image group t1 acquired by the stereoscopic image capturing apparatus 20 according to the relative positional deviation. As a result of the generation of the two-element image group t2, when the second element image group t2 is displayed as a reproduced image on the stereoscopic image display device 30, as shown in FIG. 5B, good image quality without blurring or blurring is obtained. 3D image 121.

  That is, the stereoscopic image correction apparatus 10 uses the first element optical group constituting the first element optical lens group from the first element image group acquired by photographing the subject through the first element optical lens group of the stereoscopic image capturing apparatus 20. A virtual stereoscopic image is created by accurately distributing each element image area of the lens, and propagating and adding the light waves of the respective element images, and this virtually created stereoscopic image is converted into the second element optical lens. Since the second element image group is generated by redistributing based on the position information of the second element optical lenses constituting the group and propagating and adding the light waves of the respective element images, When the two-element image group is displayed as a three-dimensional image through the second element optical lens, a three-dimensional image 121 with good image quality without blurring or blurring can be obtained.

  Thus, according to the stereoscopic image correction apparatus 10 of the present embodiment, the relative position between the first element optical lens group of the stereoscopic image photographing apparatus 20 and the second element optical lens group of the stereoscopic image display apparatus 30. The first element image group t1 acquired by the stereoscopic image capturing device 20 is corrected according to the shift, and the second element image group t2 with good image quality can be generated when displayed as a stereoscopic image.

  Although the present embodiment has been described above, the present invention is not limited to this, and can be implemented in various ways without changing the gist thereof.

For example, in this embodiment, as shown in FIG. 4, the pitch p _{1 of the} element lenses that form the first virtual aperture group 41 and the element lenses that form the second virtual aperture group 42. Although the pitch ratio Φ of the pitch p ₂ is set as Φ = 1, the present invention is not limited to this. For example, as shown in FIG. 6, the pitches p _{1 of the} element lenses forming the first virtual aperture group 41 and the pitches p ₃ of the element lenses forming the second virtual aperture group 42 are set. The ratio Φ may be Φ ≠ 1. That is, the pitches of the element lenses (openings) of the first and second virtual aperture groups 41 and 42 may be varied. In FIG. 6, the state of Φ <1 is shown, but Φ> 1 may be set. In this way, by setting the pitch ratio Φ represented by the above (1) in advance, the ratio of the size of the reproduced image can be controlled to a desired value.

  As shown in FIG. 6, when the pitches of the element lenses (apertures) of the first and second virtual aperture groups 41 and 42 are made different, the light waves of the element image are two virtual waves having different aperture pitches. Therefore, the length of the area of the first element image group is different from the length of the area of the second element image group. Therefore, when the second element image group is displayed as a stereoscopic image, a reproduced image having a different image size from that when the first element image group is displayed as a stereoscopic image can be displayed.

Here, when having different pitches of the first and second virtual element lens of aperture groups 41, 42 (opening), the size x _c of the subject 111, the size x of the three-dimensional image 121 to be reproduced The relationship with _r is expressed by the following equation (11).

That is, in FIG. 1, the pitch p _c of the element lenses in the three-dimensional image photographing apparatus 20, as the pitch p _r of the element lenses in the stereoscopic image display device 30 was assumed equal, the first virtual opening group 41 the pitch p ₁ of the element lenses, by controlling the pitch p ₂ of the second virtual element lens aperture group 42, the magnitude x _r of the three-dimensional image 121 to be reproduced to the size x _c of the subject 111 Can be changed.

For convenience of explanation, the case where α and γ are equal in the above-described equation (101) has been described in the above-described equations (10) and (11). However, in the present invention, the above-described equation (101) is described. ) May be α ≠ γ. Similarly, the pitch p _c of the element lenses in the three-dimensional image photographing apparatus 20, but the pitch p _r of the element lenses in the stereoscopic image display apparatus 30 has been described equal, it may be p _c ≠ p _r.

  Furthermore, in the present embodiment, the pitch of the first element optical lens in the first element optical lens group of the stereoscopic image capturing device 20 and the pitch of the second element optical lens in the second element optical lens group of the stereoscopic image display device 30 are described. However, if they are different, for example, the pitch of the second virtual aperture group 42 may be changed in accordance with the pitch of the first virtual aperture group 41. Good.

For example, when the screen size of the stereoscopic image capturing apparatus 20 is 20 inches and the screen size of the stereoscopic image capturing apparatus 20 is 50 inches, the number of first element optical lenses in the first element optical lens group and the second element optical lens group , The pitch of the first element optical lens in the first element optical lens group is 1 mm, and the pitch of the second element optical lens in the second element optical lens group is 2. .5mm.
When the screen sizes of the stereoscopic image capturing device 20 and the stereoscopic image display device 30 do not match in this way, the first element image group acquired by the stereoscopic image capturing device 20 is displayed on the stereoscopic image display device 30 as it is. As a result, the image cannot be displayed in an appropriate size that matches the screen size of the stereoscopic image display device 30, and the reproduced image is reduced or enlarged only partially.

  Therefore, the pitch of the second virtual aperture group 42 is changed in accordance with the pitch of the first virtual aperture group 41, that is, the second element optical lens in the second element optical lens group is the first element optical. A second virtual aperture group 42 arranged with the same pitch and number as the pitch and number of the first element optical lens in the lens group is created, and the light waves of the first element image group are converted into the first and second virtual waves. The second element image group is generated by converting the light wave that has passed through the aperture groups 41 and 42. As a result, it is possible to generate a second element image group that has a good image quality when displayed as a three-dimensional image and becomes a reproduced image with a proportion corresponding to the screen size of the three-dimensional image display device 30.

  Further, in the present embodiment, the formula using the Fresnel approximation is shown in the calculations of the formulas (2), (4), and (8) described above, but it is based on the Huygens-Fresnel principle. The calculation may be performed using integration or integration by Fraunhofer diffraction.

In the present embodiment, the focal lengths f ₁ of the element lenses of the first virtual aperture group 41 in the calculations of the expressions (2), (3), (7), and (8) described above, the focal length f ₂ of the second virtual element lens opening group 42 were used, respectively, but is not limited thereto. In other words, the focal length may be different for each element lens constituting the first virtual aperture group 41 or may be different for each element lens constituting the second virtual aperture group 42. .

  In the present embodiment, the first element image group is a plurality of element images that are reproduced images in which the unevenness of the subject is reversed when displayed as a stereoscopic image. However, the present invention is not limited to this. For example, when the first element image group is displayed as a stereoscopic image, a plurality of element images that have the same reproduced image as the unevenness of the subject may be used. In this case, the first element image group is obtained by inverting the individual element images that constitute the element image group that is the same reproduced image as the unevenness of the subject when displayed as a three-dimensional image by means not shown in the drawing so as to be point-symmetrically reversed. The first element image group is generated and output to the distribution unit of the stereoscopic image correction apparatus 10.

In this embodiment, the relationship between the number and density of the element lenses constituting the first virtual aperture group 41 and the number and density of the element lenses constituting the second virtual aperture group 42 is described. Although not, these relationships are arbitrary. For example, as shown in FIG. 7, element lenses constituting the first virtual aperture group 51 are arranged in a roughly small number, and a large number of element lenses constituting the second virtual aperture group 42 are densely arranged. May be arranged. In this case, the pitch of the element lenses constituting the first virtual opening group 51 is p _4, the pitch p ₁ of the element lenses constituting the first virtual aperture group 41 shown in FIG. 3 Is different.

  In addition, the stereoscopic image correction apparatus 10 is realized by controlling the hardware resources described above by the cooperation of the hardware apparatus and software as described above. This program (stereoscopic image correction program) can be provided via a communication line, or can be written on a recording medium such as a CD-ROM and distributed. In addition, the stereoscopic image correction apparatus 10 can also realize each of the above-described units with an arithmetic circuit.

In the present embodiment, the apertures constituting the first and second virtual aperture groups 41 and 42 have been described as element lenses, but may be replaced with a spatial filter such as a pinhole. In this case, the distances corresponding to the focal lengths f ₁ and f ₂ of the element lenses can be arbitrary distances.

  Further, in the stereoscopic image correction apparatus 10 of the present embodiment, wave optical calculation is applied to the propagation of the light wave, but the present invention is not limited to this. For example, a geometric optical calculation as described in Patent Document 2 (Patent Document 2: Japanese Unexamined Patent Application Publication No. 2009-21708) may be applied.

  Further, the stereoscopic image correction apparatus 10 can be configured by being incorporated in the stereoscopic image photographing apparatus 20 or the stereoscopic image display apparatus 30. For example, when the stereoscopic image capturing device 20 is a camera of a broadcasting station and the stereoscopic image display device 30 is a television receiver that receives and displays a stereoscopic image provided from the broadcasting station, the stereoscopic image correcting device 10 Is preferably incorporated into the stereoscopic image display device 30. In such a case, when the stereoscopic image correction apparatus 10 is incorporated into the stereoscopic image capturing apparatus 20, the broadcast station side determines the arrangement information acquired from a huge number of television receivers and the arrangement information of the cameras. Since a stereoscopic image corresponding to each television receiver must be generated, the burden on the broadcasting station increases. On the other hand, when the stereoscopic image correction apparatus 10 is incorporated in the stereoscopic image display apparatus 30, a huge number of television receivers each acquire a stereoscopic image and arrangement information from a broadcasting station, and the arrangement information and Since the stereoscopic image may be corrected according to the arrangement information of the device, convenience is enhanced.

DESCRIPTION OF SYMBOLS 10 Stereoscopic image correction apparatus 11 1st arrangement | positioning information acquisition means 12 2nd arrangement | positioning information acquisition means 13 Distribution means 14 1st element image conversion means 14a Light wave calculation means 14b Phase shift means 14c Light wave calculation means 15 Addition means 16 Redistribution means 17 1st Two-element image conversion means 17a Phase shift means 17b Light wave calculation means 18 Coupling means 20 Stereo image photographing device 30 Stereo image display device

Claims (3)

A stereoscopic image correction apparatus that corrects a first element image group acquired by photographing a subject through a first element optical lens group and generates a second element image group that is displayed as a stereoscopic image through the second element optical lens group Because
First arrangement information acquisition means for acquiring arrangement information of each first element optical lens constituting the first element optical lens group;
Distributing means for distributing the first element image group for each element image according to the arrangement information of the respective first element optical lenses acquired by the first arrangement information acquiring means,
An optical element for each element image in which light wave information representing each element image distributed by the distribution means as a wave of light is virtually arranged at a predetermined distance from the image plane of the element image, or First element image conversion means for converting light wave information representing a wave of light that has passed through a first virtual aperture group constituted by a spatial filter and propagated to a predetermined distance ;
Adding means for adding light wave information of each element image converted by the first element image converting means by the distributed number;
Second arrangement information acquisition means for acquiring arrangement information of each second element optical lens constituting the second element optical lens group;
In accordance with the arrangement information of the respective second element optical lenses acquired by the second arrangement information acquisition means, the light wave information added by the adding means is redistributed for each element image constituting the second element image group. Redistribution means to
A second virtual aperture composed of an optical element or a spatial filter for each element image in which the light wave information of each element image distributed by the redistribution means is virtually arranged on the image surface of the element image. Second element image conversion means for converting into light wave information representing a wave of light propagated by a distance until passing through the group to form an image;
3D image correction comprising: combining means for generating the second element image group by combining the light wave information converted by the second element image converting means by the redistributed number. apparatus. A computer for correcting a first element image group obtained by photographing a subject through the first element optical lens group and generating a second element image group displayed as a stereoscopic image through the second element optical lens group. The
First arrangement information acquisition means for acquiring arrangement information of each of the first element optical lenses constituting the first element optical lens group;
Distributing means for distributing the first element image group for each element image according to the arrangement information of the respective first element optical lenses acquired by the first arrangement information acquiring means,
An optical element for each element image in which light wave information representing each element image distributed by the distribution means as a wave of light is virtually arranged at a predetermined distance from the image plane of the element image, or First element image conversion means for converting light wave information representing a wave of light that has passed through a first virtual aperture group constituted by a spatial filter and propagated to a predetermined distance ;
Adding means for adding light wave information of each element image converted by the first element image converting means by the distributed number;
Second arrangement information acquisition means for acquiring arrangement information of each second element optical lens constituting the second element optical lens group;
In accordance with the arrangement information of the respective second element optical lenses acquired by the second arrangement information acquisition means, the light wave information added by the adding means is redistributed for each element image constituting the second element image group. Redistribution means,
A second virtual aperture composed of an optical element or a spatial filter for each element image in which the light wave information of each element image distributed by the redistribution means is virtually arranged on the image surface of the element image. A second element image converting means for converting light wave information representing a wave of light propagated by a distance until passing through the group to form an image;
A stereoscopic image correction program for causing light wave information converted by the second element image conversion means to function as a combining means for generating the second element image group by combining the redistributed number of light wave information . The first element image group acquired by photographing the subject through the first element optical lens group is corrected, and a second element image group that is displayed as a stereoscopic image is generated through the second element optical lens group. A stereoscopic image display device displaying a group of element images as a stereoscopic image,
First arrangement information acquisition means for acquiring arrangement information of each first element optical lens constituting the first element optical lens group;
Distributing means for distributing the first element image group for each element image according to the arrangement information of the respective first element optical lenses acquired by the first arrangement information acquiring means,
An optical element for each element image in which light wave information representing each element image distributed by the distribution means as a wave of light is virtually arranged at a predetermined distance from the image plane of the element image, or First element image conversion means for converting light wave information representing a wave of light that has passed through a first virtual aperture group constituted by a spatial filter and propagated to a predetermined distance ;
Adding means for adding light wave information of each element image converted by the first element image converting means by the distributed number;
Second arrangement information acquisition means for acquiring arrangement information of each second element optical lens constituting the second element optical lens group;
In accordance with the arrangement information of the respective second element optical lenses acquired by the second arrangement information acquisition means, the light wave information added by the adding means is redistributed for each element image constituting the second element image group. Redistribution means to
A second virtual aperture composed of an optical element or a spatial filter for each element image in which the light wave information of each element image distributed by the redistribution means is virtually arranged on the image surface of the element image. Second element image conversion means for converting into light wave information representing a wave of light propagated by a distance until passing through the group to form an image;
Combining means for generating the second element image group by combining the light wave information converted by the second element image converting means by the redistributed number;
A display element for displaying the second element image group generated by the combining means;
A stereoscopic image display device comprising: the second element optical lens group that transmits light waves of an element image output from the display element.

Images