JP6076083B2 - Stereoscopic image correction apparatus and program thereof - Google Patents

Description

  The present invention relates to a stereoscopic image correction apparatus for correcting an element image group obtained by photographing a subject by an integral photography method and a program therefor.

  Conventionally, as one of the stereoscopic image display methods capable of freely viewing stereoscopic images from an arbitrary viewpoint, integral photography (Integral Photography) using a group of convex lenses or pinholes arranged in a planar shape is possible. Hereinafter, the IP) method is known.

  Hereinafter, normal stereoscopic image capturing and stereoscopic image display based on the IP method will be described with reference to FIGS. 17 and 18. As shown in FIG. 17, the stereoscopic image capturing apparatus 910 includes a lens group 912 in which convex lenses are arranged on the same plane, and an imaging plate 913. FIG. 17 illustrates a subject 911, a photographing direction 914 of the stereoscopic image photographing device 910, and an element image 915 of the subject 911 formed by the lens group 912. When the subject 911 is viewed from the shooting direction 914, the cylinder is in front of the prism.

  The stereoscopic image photographing device 910 photographs a subject 911 through a lens group 912. Then, the same number of element images 915 of the subject 911 as the convex lenses that form the lens group 912 are generated on the imaging plate 913.

  As shown in FIG. 18, the stereoscopic image display device 920 includes a lens group 922 in which convex lenses are arranged on the same plane, and a display element 923. FIG. 18 illustrates a stereoscopic image 921, an observation direction 924 of the observer 926, an element image 925, and an observer 926. The display element 923 displays an element image 925 corresponding to the element image 915 photographed by the imaging plate 913 of the stereoscopic image photographing device 910.

  As a result, as shown in FIG. 18, the stereoscopic image 921 is generated such that the distance from the display element 923 is equal to the distance between the subject 911 and the imaging plate 913 in FIG. At this time, in the stereoscopic image 921 corresponding to the subject 911, when viewed from the observation direction 924, the prism is in front of the cylinder. That is, in normal stereoscopic image shooting based on the IP method, a reverse-view image (stereoscopic image 921) with an inverted depth is generated as compared with the subject 911 in FIG.

17 and 18, the optical element array is described as the lens group 912 in which minute convex lenses are arranged. However, the optical element array may be an aperture array (spatial filter) in which minute pinholes are arranged. Good.
Further, although the display element 923 has been described as displaying the element image 925 captured by the imaging plate 913, an image generated by a computer (not shown) may be displayed.

  Therefore, an invention for solving the above-described problem of reverse vision has been proposed (for example, Patent Document 1). The invention described in Patent Document 1 performs arithmetic processing on the information acquired by the stereoscopic image capturing device 910 in FIG. 17 and inputs the information after the arithmetic processing to the stereoscopic image display device 920 in FIG. A stereoscopic image 921 having a correct depth is generated.

  Hereinafter, with reference to FIGS. 19 and 20, an image depth conversion device 930 described in Patent Document 1 will be described. As shown in FIG. 19, the image depth conversion device 930 receives the element image 931 captured by the stereoscopic image capturing device 910 in FIG. 17, and this element image 931 is virtually formed through the first virtual lens array 932. The obtained stereoscopic image 933 is obtained by arithmetic processing. Then, the image depth conversion device 930 obtains an element image 935 in which the stereoscopic image 933 is virtually formed through the second virtual lens array 934 by arithmetic processing.

  As shown in FIG. 20, the stereoscopic image display device 940 has the same configuration as the stereoscopic image display device 920 of FIG. 18, and an element image 941 (generated by the image depth conversion device 930 of FIG. That is, the element image 935) of FIG. 19 is displayed. As a result, when the stereoscopic image 943 corresponding to the element image 941 is viewed from the observation direction 944, the cylinder is in front of the prism and the depth is equivalent to the subject 911 in FIG.

JP 2007-114483 A

  A stereoscopic image displayed by the conventional IP method corresponds to an optical aerial image, unlike a stereoscopic image obtained by a display using only binocular parallax. Therefore, in the conventional IP system, the state in which the observer sees the stereoscopic image has the same visibility as the state in which the real object is viewed.

  Here, when looking at the actual object, if the object is placed too close to the eye, or if the place where the object is viewed frequently changes in the depth direction, eye strain may occur. That is, when viewing a stereoscopic image displayed by the conventional IP method, eye strain may occur as in the case of viewing the actual subject.

  Therefore, an object of the present invention is to provide a stereoscopic image correction apparatus and a program thereof that can correct the depth of a stereoscopic image within an appropriate range for an observer.

  In view of the above problems, the stereoscopic image correction apparatus according to the first invention of the present application includes a depth range of a stereoscopic image when an elemental image group captured by the stereoscopic image capturing apparatus by the IP method is displayed on the stereoscopic image display apparatus, and A stereoscopic image correction apparatus that corrects a depth time change that is a change amount of a depth range, and includes a delay unit, a depth range calculation unit, a depth range determination unit, a depth time change calculation unit, and a depth time change determination unit. And a distance plane image generation means, a volume pixel distribution storage means, a depth range correction means, and a correction element image group generation means.

  According to this configuration, the stereoscopic image correction apparatus receives the element image group by the delay unit, delays the input element image group by a preset delay time, and outputs the delay element image group.

In addition, the stereoscopic image correction apparatus uses a depth range calculation unit to calculate a display depth range and a delayed display depth range when the element image group and the delay element image group are each displayed on the stereoscopic image display device by a predetermined depth range calculation method. Calculated by
This display depth range indicates a range in which a stereoscopic image is displayed in the depth direction when reproducing a stereoscopic video, that is, a depth distance of the stereoscopic image.

Then, the stereoscopic image correction apparatus determines whether or not the display depth range exceeds a preset reference depth range (for example, a depth range appropriate for the observer) by the depth range determination unit. This reference depth range may be a depth range unique to each observer, or may be a depth range common to all observers.
Here, when the display depth range exceeds the reference depth range and there is a possibility of causing eye strain when reproducing a stereoscopic image, the element image group is corrected by the following procedure.

  First, when the display depth range exceeds the reference depth range by the distance plane image generation unit, the stereoscopic image correction apparatus uses the virtual element optics having the same pitch and focal length of the element optical system of the stereoscopic image display apparatus for the element image group. A distance plane image showing a pixel distribution for each preset distance plane having a different distance from the virtual element optical system group by performing ray tracing through the virtual element optical system group in which the system is two-dimensionally arranged. Generate from element images. Then, the stereoscopic image correction device causes the distance plane image generation unit to store the generated distance plane image in the volume pixel distribution storage unit as a volume pixel distribution on the stereoscopic image space for displaying the stereoscopic image in association with the distance. .

  In this way, the same element optical system as that of the stereoscopic image display device is virtually arranged in the space, and the element image group is virtually expanded on the optical path, so that the generated volume pixel distribution is the stereoscopic image. A stereoscopic image space in which the display device displays a stereoscopic image is virtually shown.

  Next, when the display depth range exceeds the reference depth range by the depth range correction unit, the stereoscopic image correction device converts the coordinates of the volume pixel distribution in the element image group in the depth direction based on the ratio of the display depth range to the reference depth range. To correct. As described above, the corrected volume pixel distribution is obtained by compressing the stereoscopic image space displaying the stereoscopic image in the depth direction.

  Finally, the stereoscopic image correction apparatus performs ray tracing via the virtual element optical system group for each distance plane of the volume pixel distribution corrected by the depth range correction unit by the correction element image group generation unit, A corrected element image group is generated. As a result, the corrected elemental image group does not display a stereoscopic image at a position that is too close to the eyes.

Subsequently, the stereoscopic image correction apparatus calculates a display depth time change which is a change amount between the display depth range and the delayed display depth range by the depth time change calculation unit.
The display depth time change indicates how much the position of the stereoscopic image changes in the depth direction when the stereoscopic video is reproduced.

Further, the stereoscopic image correction apparatus determines whether or not the display depth time change exceeds a preset reference depth time change (for example, a depth time change appropriate for the observer) by the depth time change determination unit.
Here, when the display depth time change exceeds the reference depth time change and there is a possibility of causing eye strain at the time of reproduction of the stereoscopic video, the delay element image group is corrected by the following procedure.

  First, when the display depth time change exceeds the reference depth time change by the distance plane image generation unit, the stereoscopic image correction device performs ray tracing on the delay element image group via the virtual element optical system group. Then, a distance plane image is generated from the delay element image group, and is associated with the distance and stored in the volume pixel distribution storage unit as a volume pixel distribution on the stereoscopic image space for displaying the stereoscopic image.

  Next, when the display depth time change exceeds the reference depth time change by the depth range correction unit, the stereoscopic image correction device uses the difference between the display depth time change and the reference depth time change to calculate the volume pixel distribution in the delay element image group. Is corrected in the depth direction. Finally, the stereoscopic image correction apparatus performs ray tracing via the virtual element optical system group for each distance plane of the volume pixel distribution corrected by the depth range correction unit by the correction element image group generation unit, A corrected delay element image group is generated. Thus, the corrected delay element image group does not frequently change the position of the stereoscopic image in the depth direction.

In the stereoscopic image correction device according to the first invention of the present application, only the depth range may be corrected (second invention of the present application).
In the three-dimensional image correction apparatus according to the first invention of the present application, only the depth time change may be corrected (the third invention of the present application).

  Further, in the stereoscopic image correction apparatus according to the fourth invention of the present application, the depth range determination means sets in advance a unique individual depth range for each observer as the reference depth range, and whether the display depth range exceeds the individual depth range. And the depth range calculation means further calculates the depth range when the corrected elemental image group is displayed on the stereoscopic image display device by the depth range calculation method, and the common depth range common to the observer Is further provided with depth range warning means for determining whether or not the depth range calculated from the corrected elemental image group exceeds the common depth range, and warning if the depth range exceeds the common depth range It is characterized by that.

  According to such a configuration, a stereoscopic image can be reproduced within the optimum depth range for each observer, and an observer can be warned if there is a possibility of causing eye strain when reproducing the stereoscopic image.

  Further, in the stereoscopic image correction apparatus according to the fifth invention of the present application, the depth time change determining means presets an individual depth time change specific to each observer as the reference depth time change, and the display depth time change is the individual depth time change. The depth range calculation means further determines the depth range when the corrected delay element image group is displayed on the stereoscopic image display device by the depth range calculation method, and the depth time change The calculation means further calculates a depth time change that is a change amount between the display depth range and the corrected delay element image group, and a common depth time change common to the observer is preset and corrected. Determine whether the depth time change calculated from the later delay element image group exceeds the common depth time change, and warn if the depth time change exceeds the common depth time change That depth time change warning means, and further comprising a.

  According to such a configuration, a stereoscopic image can be reproduced within an optimum depth time change for an individual observer, and an observer can be warned when there is a possibility of causing eye strain when reproducing the stereoscopic image. .

  Here, the stereoscopic image correction apparatus according to the first invention of the present application is for causing a computer having hardware resources such as a CPU (Central Processing Unit) and storage means (for example, a memory and a hard disk) to operate cooperatively as the above-described means. It can also be realized by the three-dimensional image correction program (this sixth invention). This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

The present invention has the following excellent effects.
According to the first and sixth inventions of the present application, the stereoscopic image correction apparatus corrects the depth (depth range and depth time change) of the stereoscopic image within an appropriate range for the observer. Is not displayed, and the position of the stereoscopic image does not frequently change in the depth direction, and eye strain can be suppressed.

According to the second invention of the present application, since the stereoscopic image correction apparatus corrects the depth (depth range) of the stereoscopic image within an appropriate range for the observer, the stereoscopic image may be displayed at an excessively close position of the eyes. In addition, eye strain can be suppressed.
According to the third invention of the present application, since the stereoscopic image correction apparatus corrects the depth (change in depth time) of the stereoscopic image within an appropriate range for the observer, the position of the stereoscopic image may change frequently in the depth direction. In addition, eye strain can be suppressed.

According to the fourth aspect of the present invention, a stereoscopic image can be reproduced within an optimum depth range for each observer, and if there is a possibility of causing eye strain when reproducing this stereoscopic image, the observer is warned. it can.
According to the fifth invention of the present application, a stereoscopic image can be reproduced within an optimum depth time change for each observer, and warning is given to the observer when there is a possibility of causing eye strain when reproducing the stereoscopic image. Can do.

It is a block diagram which shows the structure of the stereo image correction apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structure of the three-dimensional image processing means of FIG. It is explanatory drawing for demonstrating the production | generation of the distance plane image in the distance plane image generation means of FIG. It is a block diagram which shows the structure of the distance plane image generation means of FIG. It is explanatory drawing for demonstrating the distance plane image which the distance plane image generation means of FIG. 2 produced | generated. It is explanatory drawing for demonstrating the volume pixel distribution memorize | stored in the volume pixel distribution memory | storage means of FIG. (A) And (b) is explanatory drawing for demonstrating correction | amendment of the depth range in the depth range correction | amendment means of FIG. It is explanatory drawing for demonstrating the concept which produces | generates the correction element image group in the correction element image group generation means of FIG. It is a block diagram which shows the structure of the correction | amendment element image group production | generation means of FIG. It is a block diagram which shows the structure of the stereo image correction apparatus which concerns on 2nd Embodiment of this invention. It is explanatory drawing for demonstrating correction | amendment of the depth time change in the three-dimensional image correction apparatus of FIG. It is explanatory drawing for demonstrating the concept which produces | generates the correction | amendment element image group in the stereo image correction apparatus of FIG. It is explanatory drawing for demonstrating the depth time change correct | amended with the stereo image correction apparatus of FIG. 11, (a) shows the depth time change before correction | amendment, (b) shows the depth time change after correction | amendment. 12 is a flowchart illustrating an operation of the stereoscopic image correction apparatus in FIG. 11. 12 is a flowchart illustrating an operation of the stereoscopic image correction apparatus in FIG. 11. It is a block diagram which shows the structure of the stereo image correction apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining imaging | photography of a to-be-photographed object in the conventional IP system. It is a figure explaining the display of a three-dimensional image in the conventional IP system. It is a figure explaining cancellation of depth inversion in the conventional IP system. It is a figure explaining the display of the element image group which the depth inversion canceled in the conventional IP system.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

(First embodiment)
[Configuration of stereoscopic image correction apparatus]
With reference to FIG. 1, the structure of the three-dimensional image correction apparatus 1 which concerns on 1st Embodiment of this invention is demonstrated.
The stereoscopic image correction apparatus 1 corrects the depth range of the element image group captured by the stereoscopic image capturing apparatus 100 when displaying a stereoscopic image by the IP method on the stereoscopic image display apparatus 200.

  The stereoscopic image correction apparatus 1 receives an element image group that is a video signal captured by the stereoscopic image capturing apparatus 100 using the IP method, and outputs a corrected element image group (correction element image group). The correction element image group is used as an element image group when displaying a stereoscopic image in the stereoscopic image display apparatus 200 using the IP method.

The stereoscopic image photographing device 100 is a photographing device using a general IP method for photographing a subject as an element image group, and is similar to the stereoscopic image photographing device 910 shown in FIG. 17, for example. The stereoscopic image display device 200 is a general display device that displays a group of element images as a stereoscopic image, and is similar to the stereoscopic image display device 920 shown in FIG. 18, for example.
The element image group input to the stereoscopic image correction apparatus 1 does not have to be a video signal obtained by capturing an actual subject, and the stereoscopic image capturing apparatus 100 is schematically illustrated by a computer (not shown) such as computer graphics. It may be generated by reproducing.

  In addition, the stereoscopic image correction apparatus 1 generates standard depth range information, reference depth range information, and element image group generation as information necessary for correcting the element image group together with the element image group video signal from the stereoscopic image capturing apparatus 100. Information and standard display information are input. However, the standard display device information may be set in the stereoscopic image correction device 1 in advance.

The standard depth range information is information indicating a preset depth range (standard depth range) when a stereoscopic image is displayed on a standard display device (not shown).
The reference depth range information is information indicating a preset depth range (reference depth range) appropriate for the observer. In the present embodiment, the reference depth range is a depth range (common depth range) common to all observers.

The element image group generation information is information indicating the characteristics and arrangement of an element optical system (hereinafter referred to as an element lens) that is an optical element included in the stereoscopic image capturing apparatus 100.
Specifically, the element image group generation information includes the distance from the element lens group of the stereoscopic image capturing device 100 to the imaging surface of the imaging plate (focal length of the element lens), and the center distance between the element lenses constituting the element lens group ( Pitch), the size of the elemental image of the subject imaged by the imaging plate, and the arrangement position of the element lens.

In the stereoscopic image correction apparatus 1, display device information is set in advance as information related to the stereoscopic image display device 200.
This display device information is information indicating the characteristics and arrangement of the element lenses included in the stereoscopic image display device 200.
Specifically, the display device information includes the distance from the element lens group of the stereoscopic image display device 200 to the display surface of the display element (focal length of the element lens), and the center interval (pitch) of the element lenses constituting the element lens group. And the size of the elemental image of the subject displayed on the display element and the arrangement position of the element lens.

The standard display device information is information indicating the characteristics and arrangement of the element lenses included in the standard display device.
Specifically, the standard display device information includes the distance from the element lens group of the standard display device to the display surface of the display element (focal length of the element lens), and the center interval (pitch) of the element lenses constituting the element lens group. ), The size of the element image of the subject displayed on the display element, and the arrangement position of the element lens.

  As illustrated in FIG. 1, the stereoscopic image correction device 1 is configured to generate an element image group based on standard depth range information, reference depth range information, element image group generation information, display device information, and standard display device information. Display state calculation means 10 and stereoscopic image processing means 20 are provided.

  The display state calculation unit 10 calculates a state (depth range) in which the element image group is displayed in the stereoscopic image display device 200, and includes a depth range calculation unit 11 and a depth range determination unit 13.

  The depth range calculation unit 11 calculates a display depth range when the element image group is displayed on the stereoscopic image display device 200 by a predetermined depth range calculation method. Specifically, the depth range calculation unit 11 calculates a display depth range using the following formulas (1) and (2), and generates display depth range information indicating the display depth range.

Specifically, the depth range calculation unit 11, of the standard depth range, a point closest to the observer depth position of (nearest point) by substituting Z _c of the formula (1), in the three-dimensional image display device 200 The closest point of the generated stereoscopic image is calculated. Further, the depth range calculation unit 11, of the standard depth range, the point far from the nearest to the observer the depth position of the (farthest point) by substituting Z _c of the formula (1), is generated by the stereoscopic image display apparatus 200 The farthest point of the stereoscopic image is calculated. Then, the depth range calculation unit 11 calculates a range from the nearest point to the farthest point of the stereoscopic image generated by the stereoscopic image display apparatus 200 as the display depth range.

Here, the distance from the element lens of the stereoscopic image display apparatus 200 to the stereoscopic image is Z _r. The distance from the lens when the stereoscopic image is displayed by the standard display until the stereoscopic image is Z _c.

The distance from the display surface of the display device in a standard display until the element lens is d _c, the pitch P _c of the element lens group, the size of the element image displayed by the display device K _c It is. These d _c , P _c , and k _c can be acquired from the standard display device information.

In the stereoscopic image display apparatus 200, the distance from the display surface of the display element to the element lens group is _dr , the pitch of the element lens group is _Pr , and the size of the element image displayed by the display element is K. _r . These d _r , P _r , and k _r can be acquired from the display device information.

Since the depth range calculation method described above is described in the following references, detailed description thereof will be omitted.
References: J.Arai, M.Okui, M.Kobayashi, and F.Okano: "Geometrical effects of positional errors in integral photography", J. Opt. Soc. Am. A, Vol. 21, pp.951-958 , 2004

The depth range determination unit 13 determines whether or not the display depth range calculated by the depth range calculation unit 11 exceeds the reference depth range.
When the display depth range exceeds the reference depth range, the depth range determination unit 13 generates a command signal for correcting the element image group.
On the other hand, when the display depth range does not exceed the reference depth range, the depth range determination unit 13 generates a command signal indicating that the element image group is not corrected.

  The command signal generated by the depth range determination unit 13 is output to the stereoscopic image processing unit 20 together with the display depth range information, the standard depth range information, the reference depth range information, the element image group generation information, and the display device information. Is done.

The stereoscopic image processing means 20 performs processing (depth range correction) on the video signal of the element image group based on the command signal.
When a command signal for correcting the element image group is input, the stereoscopic image processing unit 20 corrects the element image group.
On the other hand, when a command signal indicating that the element image group is not corrected is input, the stereoscopic image processing unit 20 outputs the same to the stereoscopic image display apparatus 200 without correcting the depth range.

  As shown in FIG. 2, the stereoscopic image processing unit 20 includes a distance plane image generation unit 30, a volume pixel distribution storage unit 40, a depth range correction unit 50, and a correction element image group generation unit 60.

  The distance plane image generation means 30 is a virtual element lens (virtual element optical system) in which each pixel of the input element image group has the same pitch and focal length of the element lens of the stereoscopic image display device 200 indicated by the optical element information. By performing optical path tracking through a configured virtual element lens group (virtual element optical system group), and assigning a pixel value of each pixel to each preset distance plane that is different in distance from the virtual element lens group A plane image for each distance (distance plane image) is generated.

The distance plane image generation unit 30 generates a distance plane image for each different distance and writes it in the volume pixel distribution storage unit 40 in association with the distance.
That is, the distance plane image generation means 30 corresponds to the pixel distribution when the element image group is displayed as a stereoscopic image by generating different plane images depending on a plurality of distances from the element image group constituted by one plane. Reproduce the space (stereoscopic image space).
This distance plane image generation means 30 generates a distance plane image at a plurality of predetermined distances, writes the volume pixel distribution in the volume pixel distribution storage means 40, and then indicates that the volume pixel distribution has been generated. The means 50 is notified.

Here, with reference to FIG. 3, the distance plane image which the distance plane image generation means 30 produces | generates for every distance is demonstrated.
As shown in FIG. 3, the distance plane image generation means 30 virtually corresponds to the image surface of the element image group G by the same distance as the distance _dr between the element lens group and the display surface in the stereoscopic image display device 200. By performing optical path tracking through the virtual element lens group V arranged at a distance, the pixel values of each pixel of the element image group G are assigned for each arbitrary distance L ₁ , L ₂ ,..., L _n. , planar image _{t 1,} t 2, ..., to generate a _{t n.}

Here, the distances L ₁ , L ₂ ,..., L _n are a plurality of predetermined distances, and the number and interval can be arbitrarily determined. If a large number of distance planes are set, the effect of correction can be increased accordingly, but the amount of calculation increases. Therefore, the number and interval of the distance planes are preferably determined according to the number of pixels of the element image group G and the like.

As shown in FIG. 3, the distance plane image generation means 30 has an arbitrary distance from a pixel g of a certain element image to a straight line passing through the center of the corresponding virtual element lens (virtual aperture [virtual pinhole]) V _L. The pixel value of the pixel g is assigned to the pixel on the coordinates intersecting with the plane. That is, the distance planar image generating unit 30, the pixel values of the pixels g of the elemental image, a planar image _t 1, ..., pixel _g 1 at the intersection of the _{t n,} ..., the pixel value of _{g n.}
Thus, as shown in FIG. 3, the pixel values of the pixels of the image region k _r of an element image, spread through the corresponding virtual element lenses _{V L (w 1, ...,} w n) distance plane with Assigned to. For example, a range w ₁ of the image region k _r of the element image is enlarged at a distance plane distance L ₁ is expressed by the following equation (3).

As for the range w _n of the image area k _r of the element image that reaches the distance L _n farthest from the virtual element lens group V, a can be obtained as for formula (3).
That is, the planar image (volume pixel distribution) that forms the solid generated by the distance planar image generation unit 30 has a virtual element lens group from the distance L ₁ closest to the virtual element lens group V in the z direction that is the depth direction. Generated in the range of the distance L _n farthest from V, and in the xy direction, the maximum spread range to which the pixel values of the element image group are allocated at the distance L _n farthest from the virtual element lens group V, that is, the element image group It is generated in a range larger by w _n / 2 in the vertical and horizontal directions than the size.

Here, the configuration of the distance plane image generating means 30 will be described with reference to FIG.
In order to generate a volume pixel distribution including a plurality of distance plane images described with reference to FIG. 3, the distance plane image generation unit 30 includes a division unit 31, an element image conversion unit 33, and a combination unit 35.

  The dividing unit 31 divides an element image group input as a video signal into images in element image units based on display device information. That is, the dividing unit 31 divides the element image group for each element image corresponding to the arrangement position of the element lens in the optical element information.

Note that the dividing unit 31 divides the element image group by the size of the element image handled by the stereoscopic image display apparatus 200. When the stereoscopic image capturing apparatus 100 and the stereoscopic image display apparatus 200 have different element image sizes (size, resolution), the dividing unit 31 matches the element image size of the stereoscopic image display apparatus 200. Image conversion processing such as down-conversion and up-conversion is performed.
The dividing unit 31 outputs a light wave for each divided element image to the element image converting unit 33.

  The element image converting unit 33 converts the element image input from the dividing unit 31 into a plane image at a preset distance by performing optical path tracking. Here, the element image conversion means 33 includes a pixel allocation means 33a.

The pixel assigning unit 33a assigns the pixel value of each pixel to the coordinates at which a straight line passing from the pixel to the center of the virtual element lens intersects a preset distance plane for each pixel of the element image. Thereby, a distance plane image for each element image is generated.
The pixel assigning unit 33a outputs the distance plane image generated for each element image to the combining unit 35.

The combining unit 35 combines the distance plane image for each element image on the preset distance plane generated by the element image conversion unit 33 (pixel allocation unit 33a) by the element image of the element image group on the distance plane. It is. That is, the combining unit 35 generates a distance plane image corresponding to the element image group by assigning the pixel value of the distance plane image generated for each element image on the preset distance plane.
Note that, when the pixel values of different element images are assigned to the same coordinates of the distance plane image, the combining unit 35 adds and averages the pixel values assigned to the same coordinates, thereby obtaining a pixel of the coordinates. Value.

Here, the planar light intensity distribution t will be described with reference to FIG.
In FIG. 5, the x-axis is the horizontal direction, the y-axis is the vertical direction, and the z-axis is the depth direction. Consider a case where two objects, a cylindrical body and a triangular pyramid, are taken as subjects, and the bottom surfaces of these cylindrical bodies and the triangular pyramid are photographed toward the z axis. In this case, in the distance plane image t, the pixel distribution range in which the pixel value exceeds zero represents the cylindrical three-dimensional image α and the triangular pyramid three-dimensional image β viewed from the z-axis.

Then, the combining unit 35 writes the distance plane image generated corresponding to the element image group in the volume pixel distribution storage unit 40 (see FIG. 2) in association with the distance.
The distance plane image generation unit 30 described above sequentially resets the distance from the virtual element lens group, and generates plane images corresponding to a plurality of distance planes as described with reference to FIG. That is, the distance planar image generating unit 30, as shown in FIG. 3, the distance from the virtual element lens group _{V L 1, L 2, ...} , planar image _{t 1,} t 2 corresponding to _{L n,} ..., _{t n} Is generated.
The plane images t ₁ , t ₂ ,..., T _n generated by the distance plane image generation means 30 are arranged for each distance as shown in FIG. Can do.

As a result, the volume pixel distribution storage means 40 stores a volume pixel distribution composed of planar images on a plurality of distance planes. This volume pixel distribution corresponds to a state in which pixels are distributed in a stereoscopic image space when an element image group is displayed on a stereoscopic image display device.
Returning to FIG. 2, the description of the configuration of the stereoscopic image processing means 20 will be continued.

  The volume pixel distribution storage unit 40 stores the distance plane image generated by the distance plane image generation unit 30 as a volume pixel distribution in association with the distance, and is a general storage device such as a hard disk. This volume pixel distribution is corrected by the depth range correction unit 50 and is referred to by the correction element image group generation unit 60.

  The depth range correction unit 50 corrects the coordinates of the volume pixel distribution stored in the volume pixel distribution storage unit 40 in the depth direction using the display depth range information and the reference depth range information.

This volume pixel distribution is a set of light intensities assigned to each coordinate in the stereoscopic image space before correction. Therefore, in order to calculate the volume pixel distribution (corrected volume pixel distribution) in which the depth range is corrected, as shown in FIG. 7A, the coordinates (X _r ) of the stereoscopic image space where the stereoscopic image is generated are corrected. , Y _r , Z ′ _r ) may be assigned to the corrected coordinates (X _c , Y _c , Z ′ _c ) of the stereoscopic image space in which the stereoscopic image is generated, as shown in FIG. .
7A illustrates the coordinate axes of the stereoscopic image space before correction as x _r , y _r , and z _r, and FIG. 7B illustrates the coordinate axes of the stereoscopic image space after correction as x _c , y _c , and so on. Illustrated as _zc .

Here, the ratio between the display depth range β and the reference depth range α is Ω (where Ω> 1). In this case, the relationship between the coordinates before correction (X _r , Y _r , Z ′ _r ) and the coordinates after correction (X _c , Y _c , Z ′ _c ) is expressed by the following equations (4) to (6). ). That is, the depth range correction unit 50 compresses the volume pixel distribution in the depth direction with the ratio Ω between the display depth range β and the reference depth range α using Expressions (4) to (6).

  Here, the depth range correction unit 50 writes the corrected volume pixel distribution in which the coordinate position of the volume pixel distribution is corrected to the volume pixel distribution storage unit 40 and notifies the correction element image group generation unit 60 that the correction is completed.

  The correction element image group generation unit 60 uses the virtual volume of the corrected volume pixel distribution corrected by the depth range correction unit 50 as a virtual element lens (virtual element lens having the same pitch and focal length of the element lens of the stereoscopic image display device 200 indicated by the display device information. Optical path tracking is performed via a virtual element lens group (virtual element optical system group) configured by an element optical system, and an element image group at a position corresponding to the display surface of the stereoscopic image display device 200 is generated. .

  That is, as shown in FIG. 8, the correction element image group generation unit 60 calculates the pixel value of each pixel in the corrected volume pixel distribution (for example, the pixel value of the point [pixel] a) from the pixel to the virtual element lens. Assigned as the pixel value of a pixel (for example, a point [pixel] a ′) on the image that intersects a straight line passing through the center of the group V. In FIG. 8, only pixel values at a certain point are assigned on the image, but assignment is performed over the entire volume pixel distribution after correction. As a result, a corrected element image group (corrected element image group G ′) is generated.

  Here, the configuration of the correction element image group generation means 60 will be described with reference to FIG. 9 (refer to FIG. 2 as appropriate). As shown in FIG. 9, the correction element image group generation unit 60 includes an element image reverse conversion unit 61 and a connection unit 63.

The element image reverse conversion means 61 generates an element image corresponding to the arrangement position of the element lens of the stereoscopic image display device 200 indicated by the display device information from the corrected volume pixel distribution.
That is, the virtual Element image inverting means 61, the element image converting unit 33 in the reverse direction, from the target pixel to determine the pixel value of the image area k _r elements image described in FIG. 3, corresponding to the element images The pixel value of the planar image t where the straight lines passing through the center of the lens V _L intersect is set as the pixel value of the target pixel of the element image. In the distance plane image generation means 30, the distances L ₁ , L ₂ ,..., L _n are predetermined, but in the element image inverse conversion means 61, the pixel values in the corrected volume pixel distribution The distance (z coordinate) to which is assigned becomes the distance plane that is the object for performing the inverse transformation.
Here, the element image reverse conversion unit 61 includes a pixel integration unit 61a and an interpolation unit 61b.

The pixel integration unit 61a integrates the planar images for each distance indicated by the corrected volume pixel distribution on the image plane (rear focal plane) of the element image group. That is, the pixel integration unit 61a calculates the pixel value of each pixel of the element image on the image plane of the element image group for each plane image indicated by the corrected volume pixel distribution from the pixel to the center of the virtual element lens _VL . The element image is generated by assigning the pixel value of the pixel on the planar image where the straight lines passing through.
Note that there may be a plurality of distance plane images corresponding to each pixel of the element image. In that case, the pixel integration unit 61a calculates an average value of the pixel values of the corresponding pixels of the corresponding plurality of distance plane images, and sets the average value as the corresponding pixel value of the element image.
For example, in FIG. 3, when the planar image t for each distance is an image constituting the corrected volume pixel distribution, the pixel integration unit 61 a calculates the pixel value of the pixel g of the element image from the pixel g to the virtual element. Calculated as an average value of the pixel values of the pixels g ₁ , g ₂ ,..., G _n of a plurality of planar images t ₁ , t ₂ ,..., T _{n where} the straight lines passing through the center of the lens V _L intersect.
The pixel integration unit 61a outputs the generated plurality of element images to the interpolation unit 61b.

The interpolating unit 61b interpolates pixel values of pixels to which no pixel value is assigned in the element image generated by the pixel integrating unit 61a.
The element image input to the interpolating means 61b is obtained by assigning the pixel value for each distance plane, which is the corrected volume pixel distribution, to the pixel value of the element image, and the coordinate position of the pixel in the volume pixel distribution is By correcting, not all pixels of the element image are necessarily assigned.

Therefore, the interpolation unit 61b interpolates the pixel value of the pixel image to which the pixel value is not assigned in the element image generated by the pixel integration unit 61a from the pixel value of the adjacent pixel by interpolation processing. For this interpolation processing, a general method may be used. For example, the interpolation unit 61b performs interpolation of pixels that have not been assigned pixel values by interpolation or extrapolation from pixels that have already been assigned pixel values. Pixel value is calculated.
The interpolating unit 61 b outputs the element image obtained by calculating all the pixel values to the connecting unit 62.

The connecting means 63 connects the individual element images generated by the element image reverse converting means 61 and constitutes an element image group.
The connecting means 63 generates a corrected element image group (corrected element image group) by arranging the individual element images at positions corresponding to the arrangement positions of the element lenses in the optical element information.

As described above, since the stereoscopic image correction apparatus 1 corrects the depth range of the element image group, a stereoscopic image is not displayed at an excessively close position of the eyes, and eye strain can be suppressed. .
The stereoscopic image correction apparatus 1 can be operated by a program (stereoscopic image correction program) that causes a general computer to function as each of the above-described means.

The element optical system group has been described as a lens array in which element lenses are arranged, but may be an aperture array (spatial filter) in which minute pinholes are arranged.
In addition, since the stereoscopic image correction apparatus 1 performs the operation of executing steps S2 and S4 to S10 described in the second embodiment (FIG. 14), description thereof is omitted.

(Second Embodiment)
[Configuration of stereoscopic image correction apparatus]
With reference to FIG. 10, the difference from the first embodiment will be described for the stereoscopic image correction apparatus 1B according to the second embodiment of the present invention (see FIG. 2 as appropriate).
The stereoscopic image correction apparatus 1B is different from the first embodiment in that in addition to the depth range, the stereoscopic image correction apparatus 1B also corrects the depth time change. Therefore, the stereoscopic image correction apparatus 1B includes a display state calculation unit 10B, a stereoscopic image processing unit 20B, and a delay unit 70.

  The stereoscopic image correction apparatus 1B receives standard depth time change information and reference depth time change information as information necessary for correcting the delay element image group from the stereoscopic image capturing apparatus 100.

The standard depth time change information is information indicating a preset depth time change (standard depth time change) when a stereoscopic image is displayed on a standard display device.
The reference depth time change information is information indicating preset depth time change (reference depth time change) appropriate for the observer. In this embodiment, the reference depth time change information is assumed to be a depth time change common to all observers (common depth time change).

Here, the delay means 70 will be described first.
The delay means 70 receives an element image group from the stereoscopic image capturing apparatus 100, delays the input element image group by a delay time, and outputs a delay element image group. The delay means 70 is composed of, for example, a frame memory (not shown) that stores element image groups and a control means (not shown) that controls the frame memory.

In the following description, the time at which the element images is input from the three-dimensional image photographing apparatus 100 and T _1, the time after the delay processing in the delay means 70 and T _2. That is, the delay time Δ = T ₂ −T ₁ and the element image group delayed by the delay time Δ is referred to as a delay element image group.
The delay time Δ is set in advance as an arbitrary value, for example, indicating a minute time.

  The display state calculation unit 10B calculates a state in which the element image group and the delay element image group are displayed in the stereoscopic image display apparatus 200, and includes a depth range calculation unit 11B, a depth range determination unit 13, and a depth time. A change calculating means 15 and a depth time change determining means 17 are provided.

  The depth range calculation unit 11B calculates a delay display depth range when the delay element image group is displayed on the stereoscopic image display device 200 in the same manner as the depth range calculation unit 11 of FIG. 1 in addition to the display depth range. is there.

The depth time change calculation unit 15 calculates a display depth time change which is a change amount between the display depth range and the delay display depth range calculated by the display state calculation unit 10B. That is, the depth time change calculation means 15 calculates the difference between the display depth range and the delay display depth range as the display depth time change.
This display depth time change is information indicating a change in depth in the delay time Δ (minute time).

The depth time change determination means 17 determines whether or not the display depth time change calculated by the depth time change calculation means 15 exceeds the reference depth time change.
When the display depth range exceeds the reference depth range, the depth time change determination unit 17 generates a command signal for correcting the delay element image group.
On the other hand, when the display depth range does not exceed the reference depth range, the depth time change determination unit 17 generates a command signal indicating that the delay element image group is not corrected.
The command signal generated by the depth time change determination unit 17 is output to the stereoscopic image processing unit 20B together with the standard depth time change information and the reference depth time change information.

As illustrated in FIG. 11, the display depth time change η indicates the amount of change in the depth range between the stereoscopic image space in the element image group at time T _{1 and} the stereoscopic image space in the element image group at time T ₂ . In this example, the depth time change determination unit 17 generates a command signal for correcting the delay element image group because the display depth time change η exceeds the reference depth time change γ.
In FIG. 11, illustrating the three-dimensional image space in element image group of time T ₁ by a two-dot chain line. Further, the description of the symbol ε in FIG. 11 will be described later.

Returning to FIG. 10, the description of the configuration of the stereoscopic image correction apparatus 1B will be continued.
The stereoscopic image processing means 20B has the same configuration as that of the stereoscopic image processing means 20 in FIG. 2, and corrects the delay element image group in addition to the element image group based on the command signal.
When a command signal for correcting the delay element image group is input, the stereoscopic image processing unit 20B corrects the delay element image group.
On the other hand, when a command signal indicating that the delay element image group is not corrected is input, the stereoscopic image processing unit 20B outputs the signal to the stereoscopic image display apparatus 200 as it is without correcting the delay element image group.

The distance plane image generation means 30 (FIG. 2) generates a distance plane image for each distance plane from the delay element image group by the same method as the element image group, and associates the distance plane image with the distance in the volume pixel distribution storage means 40. Write.
Thus, the volume pixel distribution storage means 40 (FIG. 2) stores the volume pixel distribution in the delay element image group in addition to the volume pixel distribution in the element image group.

  The depth range correction means 50 (FIG. 2) corrects the coordinates of the volume pixel distribution in the delay element image group in the depth direction using the display depth time change information and the reference depth time change information. That is, the depth range correction unit 50 corrects the volume pixel distribution in the delay element image group by the difference between the display depth time change and the reference depth time change, similarly to the volume pixel distribution in the element image group.

  The correction element image group generation means 60 (FIG. 2) generates a correction delay element image group from the correction volume pixel distribution in the delay element image group by the same method as the correction element image group.

Here, the correction element image group generation unit 60 presets the distance from the corrected volume light intensity in the delay element image group to the virtual element lens group based on the display device information. This position of the virtual element lens group is referred to as “reference optical element array position”. Then, as shown in FIG. 12, the correction element image group generation means 60 corrects the delay element image group in a state where the virtual element lens group V is arranged at a position shifted from the reference optical element array position S by the distance correction value ε. G ″ is generated.
This distance correction value ε indicates the difference between the display depth time change η and the reference depth time change γ, as shown in FIG.

Accordingly, the correction element image group generation unit 60 generates the correction delay element image group so that the display depth time change η in FIG. 13A does not exceed the reference depth time change γ in FIG. 13B. be able to.
In FIG. 13, the stereoscopic image space in the element image group at time T ₁ is illustrated by a two-dot chain line, and the stereoscopic image space in the delay element image group at time T ₂ is illustrated by a solid line.

[Operation of stereoscopic image correction device]
The operation of the stereoscopic image correction apparatus 1B will be described with reference to FIGS.
The stereoscopic image correction apparatus 1B outputs a delayed element image group obtained by delaying the element image group by the delay time by the delay unit 70 (step S1).

In the stereoscopic image correction apparatus 1B, the depth range calculation unit 11B calculates the display depth range from the element image group, and calculates the delay display depth range from the delay element image group (step S2).
In the stereoscopic image correction apparatus 1B, the depth time change calculation unit 15 calculates a display depth time change that is a change amount between the display depth range and the delayed display depth range (step S3).

In the stereoscopic image correction apparatus 1B, the depth range determination unit 13 determines whether or not the display depth range exceeds the reference depth range (step S4).
When the display depth range exceeds the reference depth range (Yes in step S4), the stereoscopic image correction apparatus 1B proceeds to the process of step S5.
In the stereoscopic image correction apparatus 1B, the distance plane image generation unit 30 sets distance planes having different distances from the virtual element lens group for the element image group (step S5).

  The stereoscopic image correction apparatus 1B uses the distance plane image generation means 30 to assign the pixel value of each pixel of the element image group to the distance plane corresponding to the distance set in step S5 by optical path tracking, and to obtain a plane coordinate for each distance ( A distance plane image).

That is, in the stereoscopic image correction apparatus 1B, the dividing unit 31 of the distance plane image generating unit 30 divides the element image group for each element image specified by the display device information.
In the stereoscopic image correction apparatus 1B, the element image conversion unit 32 of the distance plane image generation unit 30 sets the pixel value of each pixel of the element image as a straight line passing through the pixel and the center of the corresponding virtual element lens. The pixel value is assigned to a pixel on a planar image that intersects the distance plane.
The stereoscopic image correction apparatus 1B uses the combination unit 35 of the distance plane image generation unit 30 to convert the distance plane image for each element image in the distance plane generated by the element image conversion unit 32 (pixel allocation unit 32a) in the distance plane. Only the element images of the image group are combined (step S6).

  The stereoscopic image correction apparatus 1B writes the planar image for each distance generated in step S6 by the distance plane image generation unit 30 (combination unit 35) in the volume pixel distribution storage unit 40 (step S7). By generating this plane image for each different distance plane, a volume pixel distribution is obtained.

  In the stereoscopic image correction apparatus 1B, the depth range correction unit 50 corrects the coordinates of the volume pixel distribution stored in the volume pixel distribution storage unit 40 in the depth direction using the display depth range information and the reference depth range information. . That is, the depth range correction unit 50 compresses the volume pixel distribution in the element image group in the depth direction at the ratio of the display depth range and the reference depth range (step S8).

  The stereoscopic image correction apparatus 1B calculates the pixel value of the image plane of the element image group from the volume pixel distribution corrected in step S8 by optical path tracking by the element image inverse conversion means 61 of the element image inverse conversion means 61.

That is, the stereoscopic image correction apparatus 1B exists on a straight line passing from the pixel to the center of the virtual element lens for each pixel of the element image for which the pixel value is obtained by the pixel integration unit 61a of the element image reverse conversion unit 61. A pixel value of a pixel having a corrected volume pixel distribution is assigned. At this time, when there are a plurality of pixels to which pixel values are assigned on a straight line, the pixel integration unit 61a sets the average value of the pixels as the pixel value of the element image.
In the stereoscopic image correction apparatus 1B, the interpolation unit 61b of the element image reverse conversion unit 61 interpolates the pixel value of the pixel to which no pixel value is assigned in the element image generated by the pixel integration unit 61a by interpolation or the like ( Step S9).

  The stereoscopic image correction apparatus 1B connects the individual element images generated by the element image reverse conversion unit 61 by the connecting unit 63, and generates a corrected element image group (step S10).

When the display depth range does not exceed the reference depth range (No in step S4), or after the process of step S10, the stereoscopic image correction apparatus 1B proceeds to the process of step S11.
In the stereoscopic image correction device 1B, the depth time change determination unit 17 determines whether the display depth time change exceeds the reference depth time change (step S11).

When the display depth time change exceeds the reference depth time change (Yes in step S11), the stereoscopic image correction apparatus 1B proceeds to the process of step S12.
In the stereoscopic image correction apparatus 1B, the distance plane image generation unit 30 sets distance planes having different distances from the virtual element lens group for the delay element image group (step S12).

  The stereoscopic image correction apparatus 1B uses the distance plane image generation means 30 to assign the pixel value of each pixel of the delay element image group to the distance plane corresponding to the distance set in step S12 by optical path tracking, and to obtain plane coordinates for each distance. (Distance plane image) is generated.

That is, in the stereoscopic image correction apparatus 1B, the division element 31 of the distance plane image generation unit 30 divides the delay element image group for each element image specified by the display device information.
In the stereoscopic image correction apparatus 1B, the element image conversion unit 32 of the distance plane image generation unit 30 sets the pixel value of each pixel of the element image as a straight line passing through the pixel and the center of the corresponding virtual element lens. The pixel value is assigned to a pixel on a planar image that intersects the distance plane.
The stereoscopic image correction apparatus 1B delays the distance plane image for each element image in the distance plane generated by the element image conversion unit 32 (pixel allocation unit 32a) in the distance plane by the combining unit 35 of the distance plane image generation unit 30. Only the element images of the element image group are combined (step S13).

  The stereoscopic image correction apparatus 1B writes the plane image for each distance generated in step S13 by the distance plane image generation unit 30 (combination unit 35) in the volume pixel distribution storage unit 40 (step S14).

  The stereoscopic image correction apparatus 1B uses the depth range correction unit 50 to display the coordinates of the volume pixel distribution stored in the volume pixel distribution storage unit 40 in the depth direction using the display depth time change information and the reference depth time change information. to correct. In other words, the depth range correction unit 50 compresses the volume pixel distribution in the delay element image group in the depth direction based on the difference between the display depth time change and the reference depth time change (step S15).

  The stereoscopic image correction apparatus 1B calculates the pixel value of the image plane of the delay element image group from the volume pixel distribution corrected in step S15 by optical path tracking by the element image inverse conversion means 61 of the element image inverse conversion means 61. .

That is, the stereoscopic image correction apparatus 1B exists on a straight line passing from the pixel to the center of the virtual element lens for each pixel of the element image for which the pixel value is obtained by the pixel integration unit 61a of the element image reverse conversion unit 61. A pixel value of a pixel having a corrected volume pixel distribution is assigned.
In the stereoscopic image correction apparatus 1B, the interpolation unit 61b of the element image reverse conversion unit 61 interpolates the pixel value of the pixel to which no pixel value is assigned in the element image generated by the pixel integration unit 61a by interpolation or the like ( Step S16).

The stereoscopic image correction apparatus 1B connects the individual element images generated by the element image reverse conversion unit 61 by the connecting unit 63, and generates a corrected delayed element image group (step S17).
When the display depth time change does not exceed the reference depth time change (No in step S11), or after step S17, the stereoscopic image correction apparatus 1B ends the process.

  As described above, since the stereoscopic image correction apparatus 1B corrects the depth range of the element image group and the depth time change of the delay element image group, the stereoscopic image is displayed at an excessively close position of the eye, Since the position of the stereoscopic image does not change frequently in the depth direction, it is possible to suppress eye strain.

  In the second embodiment, the stereoscopic image correction apparatus 1B has been described as correcting both the depth range and the depth time change. However, only the depth time change may be corrected. In this case, the stereoscopic image correction apparatus 1B does not need to include the depth range determination unit 13.

(Third embodiment)
[Configuration of stereoscopic image correction apparatus]
With reference to FIG. 16, a difference from the second embodiment will be described regarding a stereoscopic image correction apparatus 1 </ b> C according to the third embodiment of the present invention.
The stereoscopic image correction apparatus 1C is different from the second embodiment in that a warning is issued under a predetermined condition. Therefore, the stereoscopic image correction apparatus 1C includes a display state calculation unit 10C, a stereoscopic image processing unit 20B, a delay unit 70, a parameter setting unit 80, and a warning unit 90.

Here, the parameter setting means 80 will be described first.
The parameter setting means 80 is for the observer to set various parameters (individual depth range information, individual depth time change information, operation mode information).
The parameters set in the parameter setting unit 80 are referred to by the display state calculation unit 10C, the stereoscopic image processing unit 20B, and the warning unit 90.

The individual depth range information is information indicating a preset depth range (individual depth range) for each observer.
The individual depth time change information is information indicating a preset depth time change (individual depth time change) for each observer, which is set in advance.
The operation mode information is information indicating whether the stereoscopic image correction apparatus 1C operates in the reference setting mode or the viewer setting mode.

  The reference setting mode is that the stereoscopic image correction apparatus 1C operates using the reference depth range information and the reference depth time change information. That is, in the case of the reference setting mode, the stereoscopic image correction apparatus 1C uses a common depth range and a common depth time change common to all viewers.

  The viewer setting mode is the operation of the stereoscopic image correction apparatus 1C using the individual depth range information and the individual depth time change information. That is, in the case of the viewer setting mode, the stereoscopic image correction apparatus 1C uses the individual depth range and the individual depth time change unique to each viewer. Furthermore, the stereoscopic image correction apparatus 1C issues a warning when the correction element image group and the correction delay element image group may cause eye strain.

The display state calculation unit 10C includes a depth range calculation unit 11C, a depth range determination unit 13C, a depth time change calculation unit 15C, and a depth time change determination unit 17C.
Further, the display state calculation unit 10C receives the correction element image group and the correction delay element image group from the stereoscopic image processing unit 20B.

In the viewer setting mode, the depth range calculation unit 11C determines the depth range (correction depth range) of the correction element image group and the depth range of the correction delay element image group in the same manner as the depth range calculation unit 11B of FIG. Is to be calculated.
The corrected depth range calculated by the depth range calculation unit 11C is output to the warning unit 90.

  The depth range determination unit 13C selects one of the individual depth range information and the reference depth range information based on the operation mode information, and makes the determination in the same manner as the depth range determination unit 13 in FIG.

In the viewer setting mode, the depth time change calculation means 15C changes the depth time change (correction depth time change) between the correction element image group and the correction delay element image group in the same manner as the depth time change calculation means 15 in FIG. Is to be calculated.
The corrected depth time change calculated by the depth time change calculation means 15C is output to the warning means 90.

  Depth time change determination means 17C selects either individual depth time change information or reference depth time change information based on the operation mode information, and makes the same determination as depth time change determination means 17 in FIG. It is.

  In the viewer setting mode, the warning unit 90 issues a warning according to a predetermined condition, and includes a depth range warning unit 91 and a depth time change warning unit 93.

The depth range warning unit 91 determines whether or not the corrected depth range input from the depth range calculation unit 11C exceeds the common depth range, and warns when the corrected depth range exceeds the common depth range.
Here, the depth range warning means 91 may add a warning message for the depth range to the video signal without any particular limitation on the warning method. Further, the depth range warning means 91 may transmit this warning message to a preset mail address or may sound a predetermined warning sound.

The depth time change warning means 93 determines whether or not the corrected depth time change input from the depth time change calculation means 15C exceeds the common depth time change, and warns when the corrected depth time change exceeds the common depth time change. To do.
Here, the depth time change warning means 93 can give a warning in the same manner as the depth range warning means 91.

  As described above, the stereoscopic image correction apparatus 1 </ b> C can reproduce a stereoscopic image so as to be within a depth range and depth time change that are optimal for each observer, and may cause eye strain during reproduction of the stereoscopic image. If there is, it can alert the observer.

  Note that the stereoscopic image correcting apparatus 1C may be configured to use only one of the individual depth range information and the individual depth time change information, or the reference depth range information and the reference depth time change information without including the warning unit 90. Good.

1, 1B, 1C Stereoscopic image correction apparatus 10, 10B, 10C Display state calculation means 11, 11B, 11C Depth range calculation means 13, 13C Depth range determination means 15, 15C Depth time change calculation means 17, 17C Depth time change determination means 20, 20B Stereoscopic image processing means 30 Distance plane image generation means 31 Division means 33 Element image conversion means 33a Pixel allocation means 35 Combining means 40 Volume pixel distribution storage means 50 Depth range correction means 60 Correction element image group generation means 61 Element image reverse Conversion means 61a Pixel integration means 61b Interpolation means 63 Connection means 70 Delay means 80 Parameter setting means 90 Warning means 91 Depth range warning means 93 Depth time change warning means

Claims (6)

Corrects a depth range of a stereoscopic image when an elemental image group captured by a stereoscopic image capturing device using the integral photography method is displayed on a stereoscopic image display device, and a change in depth time that is a change amount of the depth range. A stereoscopic image correction device,
A delay means for inputting the element image group, delaying the input element image group by a preset delay time, and outputting a delay element image group;
A depth range calculating means for calculating a display depth range and a delayed display depth range when the element image group and the delay element image group are respectively displayed on the stereoscopic image display device by a predetermined depth range calculation method;
Depth range determining means for determining whether or not the display depth range exceeds a preset reference depth range;
A depth time change calculating means for calculating a display depth time change which is a change amount between the display depth range and the delay display depth range;
Depth time change determination means for determining whether or not the display depth time change exceeds a preset reference depth time change;
When the display depth range exceeds the reference depth range, virtual element optics in which virtual element optical systems having the same pitch and focal length of the element optical system of the stereoscopic image display device are arranged in a two-dimensional manner for the element image group By performing ray tracing through the system group, a distance plane image showing a pixel distribution for each preset distance plane having a different distance from the virtual element optical system group is generated from the element image group, and the display When the depth time change exceeds the reference depth time change, the distance plane image is generated from the delay element image group by performing ray tracing on the delay element image group via the virtual element optical system group. A distance plane image generating means for performing,
Volume pixel distribution storage means for storing a distance plane image for each distance plane generated by the distance plane image generation means as a volume pixel distribution on a stereoscopic image space for displaying the stereoscopic image in association with a distance;
When the display depth range exceeds the reference depth range, the coordinates of the volume pixel distribution in the element image group are corrected in the depth direction at a ratio between the display depth range and the reference depth range, and the display depth time change is When the reference depth time change is exceeded, a depth range correction unit that corrects the coordinates of the volume pixel distribution in the delay element image group in the depth direction by a difference between the display depth time change and the reference depth time change;
By performing ray tracing through the virtual element optical system group for each distance plane of the volume pixel distribution corrected by the depth range correction unit, at least one of the corrected element image group and the delayed element image group is obtained. Correction element image group generation means for generating;
A three-dimensional image correction apparatus comprising: A stereoscopic image correction apparatus that corrects a depth range of a stereoscopic image when an elemental image group captured by a stereoscopic image capturing apparatus using an integral photography method is displayed on a stereoscopic image display apparatus,
Depth range calculation means for calculating a display depth range when the element image group is displayed on the stereoscopic image display device by a predetermined depth range calculation method;
Depth range determining means for determining whether or not the display depth range exceeds a preset reference depth range;
When the display depth range exceeds the reference depth range, virtual element optics in which virtual element optical systems having the same pitch and focal length of the element optical system of the stereoscopic image display device are arranged in a two-dimensional manner for the element image group A distance plane image generating means for generating a distance plane image indicating a pixel distribution for each preset distance plane having a different distance from the virtual element optical system group by performing ray tracing through the system group;
Volume pixel distribution storage means for storing a distance plane image for each distance plane generated by the distance plane image generation means as a volume pixel distribution on a stereoscopic image space for displaying the stereoscopic image in association with a distance;
When the display depth range exceeds the reference depth range, a depth range correction unit that corrects the coordinates of the volume pixel distribution in the element image group in the depth direction at a ratio between the display depth range and the reference depth range;
Correction element image group generation means for generating a corrected element image group by performing ray tracing through the virtual element optical system group for each distance plane of the volume pixel distribution corrected by the depth range correction means. When,
A three-dimensional image correction apparatus comprising: A stereoscopic image correction device that corrects a depth time change, which is a change amount of a depth range of a stereoscopic image when an element image group captured by a stereoscopic image capturing device by an integral photography method is displayed on a stereoscopic image display device. And
A delay means for inputting the element image group, delaying the input element image group by a preset delay time, and outputting a delay element image group;
A depth range calculating means for calculating a display depth range and a delayed display depth range when the element image group and the delay element image group are respectively displayed on the stereoscopic image display device by a predetermined depth range calculation method;
A depth time change calculating means for calculating a display depth time change which is a change amount between the display depth range and the delay display depth range;
Depth time change determination means for determining whether or not the display depth time change exceeds a preset reference depth time change;
When the display depth time change exceeds the reference depth time change, virtual element optical systems having the same pitch and focal length of the element optical system of the stereoscopic image display device are two-dimensionally arranged for the delay element image group. Distance plane image generation means for generating a distance plane image indicating a pixel distribution for each preset distance plane having a different distance from the virtual element optical system group by performing ray tracing through the virtual element optical system group When,
Volume pixel distribution storage means for storing a distance plane image for each distance plane generated by the distance plane image generation means as a volume pixel distribution on a stereoscopic image space for displaying the stereoscopic image in association with a distance;
When the display depth time change exceeds the reference depth time change, a depth range for correcting the coordinates of the volume pixel distribution in the delay element image group in the depth direction by a difference between the display depth time change and the reference depth time change Correction means;
Correction element image group generation means for generating a corrected element image group by performing ray tracing through the virtual element optical system group for each distance plane of the volume pixel distribution corrected by the depth range correction means. When,
A three-dimensional image correction apparatus comprising: The depth range determination means, as the reference depth range, is preset specific individual depth range for each observation's said display depth range is determined whether more than the individual depth range,
The depth range calculation means further calculates a depth range when the corrected elemental image group is displayed on the stereoscopic image display device by the depth range calculation method,
A common depth range common to the observer is set in advance, and it is determined whether or not a depth range calculated from the corrected element image group exceeds the common depth range, and the depth range corresponds to the common depth range. Depth range warning means to warn when exceeding,
The stereoscopic image correction apparatus according to claim 1, further comprising: The depth time change determination means, as the change the reference depth time is set separately depth time variation inherent to each observation's advance, the display depth time variation is determined whether more than the individual depth time change ,
The depth range calculation means further calculates a depth range when the corrected delay element image group is displayed on the stereoscopic image display device by the depth range calculation method,
The depth time change calculating means further calculates a depth time change which is a change amount between the display depth range and the corrected depth element image group,
A common depth time change common to the observer is preset, it is determined whether the depth time change calculated from the corrected delay element image group exceeds the common depth time change, and the depth time change Depth time change warning means for warning when the common depth time change is exceeded,
The stereoscopic image correction apparatus according to claim 1, further comprising:   A stereoscopic image correction program for causing a computer to function as the stereoscopic image correction apparatus according to claim 1.

Images