JP6543096B2 - Element image generating device for viewpoint position, program therefor, and integral three-dimensional simulation system - Google Patents

Description

  The present invention relates to an element image generating device for viewpoint position for simulating an integral solid, a program thereof, and an integral solid simulation system.

Conventionally, an integral stereo method is known as a method for displaying a stereo image.
In this integral stereo method, as shown in FIG. 10, the imaging device captures light emitted from a subject O through a lens array La including a plurality of element lenses Lp arranged two-dimensionally. At this time, the element image G is captured at the lens interval of the element lens Lp on the imaging surface E separated by the focal length of the element lens Lp.
Then, in the integral stereo method, as shown in FIG. 11, the element image G captured in FIG. 10 is displayed on the display surface D of the display device (display) via the lens array La of the same specification as imaging. At this time, the same light beam as the imaged subject space is reproduced, and the observer M can visually recognize the subject O (see FIG. 10) as a stereoscopic image T.
In this integral three-dimensional method, a method of generating an element image from a three-dimensional model in a virtual space has also been proposed in addition to imaging an object having a substance and generating an element image (Patent Document 1, Non-Patent Document 1) Reference 1).
In this method, all element images corresponding to element lenses are generated from the three-dimensional model in virtual space, corresponding to the specifications of a high definition display device and a lens array.

  Further, as another method for displaying a three-dimensional image, a twin-lens stereoscopic method such as a lenticular method is known. For example, in the lenticular method, images for left and right two eyes are alternately arranged and displayed in stripes, and the observer moves the viewpoint in the left and right direction by using a lenticular lens to view a three-dimensional image viewed from different viewpoints. It is a method which can visually recognize (refer to patent documents 2). Patent Document 2 discloses an invention which detects the viewpoint position of the observer and generates an image in the observation direction for each of the left and right two-eye images.

JP, 2013-196532, A JP-A-9-238369

Iwatsuki, Katayama, "Method of generating an integral three-dimensional image by oblique projection," Technical Report of The Institute of Image Information and Television Engineers, vol. 34, no. 43, pp. 17-20, October 2010

In the integral stereo method, parameters such as the pixel spacing of the display device, the lens spacing of the lens array, and the focal length become important factors that determine the image quality of the stereoscopic image. Therefore, by changing these parameters in various ways, it is possible to investigate the parameters necessary to optimally display an integral-type three-dimensional image (hereinafter, integral three-dimensional image).
However, since the pixel spacing and the like of the display device is determined by the manufacturer's design, it is difficult to align the display devices meeting the parameter specifications to be investigated, and it is difficult to investigate the image quality of the integral stereo image.

  On the other hand, without using an actual display device for displaying an integral three-dimensional image, for example, a plurality of elemental images for displaying a three-dimensional image generated by Patent Document 1 etc. are converted into a two-eye three-dimensional image In order to perform simulation, it is necessary to generate all elemental images in advance and to generate an image for two-lens stereo.

In this case, all element images are generated and stored in the main storage memory, and only information for two-eye stereoscopy is transferred to the graphic memory. Therefore, in the case of a high resolution display device or element image, There is a problem that a large-scale main memory is required.
Also, in this case, since the display image is generated from the large-capacity element image group, the higher the resolution of the screen of the display device or the element image, the more time it takes to generate a moving image, making it difficult to display moving images. become.

  Although the invention described in Patent Document 2 generates left and right two-eye images according to the viewpoint position and reduces the load of image generation for display, to apply to the integral stereo method, It is necessary to generate a large-capacity element image in advance for displaying a stereoscopic image, and the problem that a large-capacity main memory is required can not be solved.

  The present invention has been made in view of such a problem, and it is possible to generate a partial element image corresponding to the viewpoint position of the observer, reduce the amount of memory used, and simulate an integral 3D image. An object of the present invention is to provide an element image generating apparatus for possible viewpoint position and its program, and an integral three-dimensional simulation system.

  In order to solve the above problem, the viewpoint position element image generating device according to the present invention is an observer in order to simulate a three-dimensional image displayed by a target display device to be a simulation target of the integral three-dimensional method with another display device. A viewpoint position element image generation unit that generates a viewpoint position element image that is a partial element image that displays a stereoscopic image corresponding to the viewpoint position, and includes a viewpoint corresponding position identification unit, a virtual space imaging unit, and , Filtering means, and element image reconstruction means.

  In such a configuration, the viewpoint position element image generation device virtually arranges the target display device on the same coordinate system (three-dimensional coordinate system) as the other display devices by the viewpoint corresponding position specifying means, and the viewpoint position detection device From the viewpoint position of the observer detected in the above, the viewpoint corresponding position of each element lens which passes through the lens center of each element lens of the object display device and reaches the display surface of the object display device is calculated. The number of vertical and horizontal pixels and the pixel interval of the target display device, the number of vertical and horizontal lenses of the lens array, the lens interval, and the focal distance necessary to virtually arrange the target display device are externally input as parameters.

  As a result, in the viewpoint position element image generation device, the pixel position to be visually recognized from the viewpoint position of the observer is specified corresponding to the element image of the element lens for the target display device to display a stereoscopic image. Become. In addition, since only a part of the element image is viewed from the viewpoint position of the observer, it is not necessary to generate an element image other than the viewed pixel position.

  The visual point position element image generation device passes the center of the lens of the element lens corresponding to the visual point corresponding position from the visual point corresponding position by the virtual space photographing means to the virtual camera at the visual distance position to the observer. It arranges, and three-dimensional space data such as CG is virtually photographed on a predetermined projection plane by perspective projection, and is rendered in a predetermined range for each element lens.

Then, the viewpoint position element image generation apparatus filters the image rendered for each element lens by the filtering unit, and calculates a pixel value corresponding to the element lens.
At this time, the image rendered by the virtual space imaging means includes the peripheral image of the viewpoint corresponding position. Therefore, the filtering means calculates the pixel value taking into consideration the image around the ideal viewpoint position, and it becomes possible to reproduce the blur in the three-dimensional image.

Then, the elemental image generation device for viewpoint position sets the pixel value of each of the component lenses calculated by the filtering unit by the component image reconstruction unit to the viewpoint corresponding position of each of the component lenses of the display that the observer visually recognizes in practice. By assigning to corresponding pixels, an element image for viewpoint position of a viewer is generated which reproduces only a part of the element image.
As a result, the viewpoint position element image generation device can generate only a part of the element image that corresponds to the viewer's viewpoint position and that is necessary for the viewer to visually recognize the stereoscopic image.

  Note that the viewpoint position element image generation device operates the computer with a viewpoint position element image generation program for causing the computer to function as a viewpoint corresponding position specifying unit, virtual space photographing unit, filtering unit, and element image reconstruction unit. it can.

  Further, in order to solve the above problems, an integral three-dimensional simulation system according to the present invention is an integral three-dimensional simulation system that simulates a three-dimensional image displayed by a target display device that is an integral three-dimensional simulation target. A position detection device, a viewpoint position element image generation device, and an element image display device are provided.

In such a configuration, the integral stereo simulation system detects the viewpoint position of the observer by the viewpoint position detection device.
Then, the integral stereo simulation system generates a viewpoint position element image which is a partial element image for displaying a stereo image corresponding to the viewpoint position of the observer by the viewpoint position element image generation device.
Then, the integral three-dimensional simulation system displays the viewpoint position element image generated by the viewpoint position element image generation device through the lens array by the element image display device.

As a result, the elemental image display apparatus can generate a partial image of the elemental image and display it so that the observer can visually recognize the stereoscopic image at the viewpoint position.
Thus, the integral 3D simulation system reduces the amount of memory used by generating only a part of the element image viewed from the viewpoint of the observer without generating all the element images. The three-dimensional image displayed by the integral three-dimensional display device (target display device) can be simulated.

The present invention exhibits the following excellent effects.
According to the viewpoint position element image generation apparatus and the viewpoint position element image generation program of the present invention, in order to generate only partial element images corresponding to the observer's viewpoint position, when all element images are generated In comparison, it is possible to generate an element image necessary to display a stereoscopic image at high speed and with a reduced memory usage.

  Further, since the integral 3D simulation system of the present invention generates and displays only partial element images corresponding to the viewpoint position of the observer, it is faster than generating and displaying all of the element images. And, it is possible to display a stereoscopic image while suppressing the memory usage.

  As described above, since the present invention can reduce the memory usage for generating the element image, it is possible to cope with the further high resolution of the element image. The present invention also enables simulation of a three-dimensional image of a moving image because the element image is partially generated.

FIG. 1 is a configuration diagram showing an entire configuration of an integral three-dimensional simulation system according to an embodiment of the present invention. The coordinate system at the time of detecting an observer's viewpoint position is shown, (a) shows xy coordinates, (b) shows yz coordinates. It is a block diagram which shows the block configuration of the element image production | generation apparatus for viewpoint positions which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the calculation method of the viewpoint corresponding position in a viewpoint corresponding position identification means. It is an explanatory view for explaining correspondence with a viewpoint corresponding position and a sub pixel center of a pixel. It is an explanatory view for explaining field composition of a projection plane at the time of virtual space photography means performing perspective projection. It is an explanatory view for explaining arrangement of a virtual camera at the time of virtual space photographing means performing perspective projection. It is an explanatory view for explaining a perspective projection matrix. It is a flowchart which shows operation | movement of the element image production | generation apparatus for viewpoint positions which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the imaging system of the conventional integral system. It is explanatory drawing for demonstrating the display system of the conventional integral system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Configuration of Integral 3D Simulation System]
First, the configuration of an integral three-dimensional simulation system S according to an embodiment of the present invention will be described with reference to FIG.

The integral three-dimensional simulation system S simulates and displays an integral three-dimensional image (integral three-dimensional image) displayed by a virtual display device to be simulated.
As shown in FIG. 1, the integral three-dimensional simulation system S includes a viewpoint position element image generation device 1, a viewpoint position detection device 2, and an element image display device 3.
In the following, the virtual display device to be simulated is referred to as a target display device, and is distinguished from the element image display device 3 which is a display device (other display device) that the observer visually recognizes in real.

The viewpoint position element image generation device 1 displays the viewpoint position of the observer M displayed on the element image display device 3 from three-dimensional space data such as CG including the subject and parameters of the target display device (not shown). The corresponding partial element image (element image for viewpoint position) is generated.
The viewpoint position element image generation device 1 inputs the viewpoint position of the observer M from the viewpoint position detection device 2, and outputs the generated viewpoint position element image to the element image display device 3.
The viewpoint position element image generating apparatus 1 will be described in detail later.

  The viewpoint position detection device 2 detects the viewpoint position of the observer M. The viewpoint position detection device 2 obtains viewpoint positions of left and right two eyes of the observer M, for example, in a three-dimensional coordinate system (x, y, z) as shown in FIG. The viewpoint position detection device 2 outputs the detected viewpoint position to the viewpoint position element image generation device 1. The element image display device 3 and the virtual target display device use the same coordinate system.

  The measurement of the viewpoint position in the viewpoint position detection device 2 can use a known method. For example, the viewpoint position detection device 2 is configured by the observer M who is photographed by two cameras installed at predetermined positions in a three-dimensional space by the sclera reflection method or the cornea / pupil reflection method used for eye movement measurement. The right and left corneas of the observer M are detected from the camera image. And the viewpoint position detection apparatus 2 can obtain | require the three-dimensional viewpoint position of right-and-left two eyes by the principle of triangulation of two camera images.

  The viewpoint position detection method of the viewpoint position detection device 2 is not limited to this method. For example, using Kinect (registered trademark) of Microsoft Corporation, the midpoint (mx, my, mz) (unit: mm) of the “right and left eye position” of the face model is output as the output of “FaceTracking SDK” of Kinect. If given, the viewpoint position detection device 2 generates the left eye viewpoint coordinates (mx-6.5 / 2, my, mz), the right eye point coordinates (mx + 6.5 / 2, mx, my) in the coordinate system of FIG. It is possible to determine the viewpoint position of the left and right two eyes. In addition, 6.5 mm is a common adult's binocular distance here.

  The element image display device 3 displays the viewpoint position element image generated by the viewpoint position element image generation device 1. The element image display device 3 is a general integral three-dimensional display device provided with a lens array composed of a plurality of element lenses on the display surface. The number of pixels of the element image display device 3 and the number of element lenses are the same as those of the target display device to be simulated. Further, the arrangement of the element lenses is similar to the arrangement of the element lenses of the target display device. For example, if the arrangement of the element lenses of the target display device is a square arrangement, the arrangement of the element lenses of the element image display device 3 is a square arrangement.

  Since this element image display device 3 displays the partial element image generated by the viewpoint position element image generation device 1 according to the viewpoint position of the observer M, a stereoscopic image can be displayed on the observer M at that viewpoint position. T can be made visible.

  As described above, the integral three-dimensional simulation system S partially generates and displays an element image according to the viewpoint position of the observer M without generating all the element images for displaying the integral three-dimensional image. The simulation can be performed with a reduced amount of memory used. In addition, the integral three-dimensional simulation system S can display a three-dimensional image at high speed, and can also be applied to the simulation of a three-dimensional image of a moving image.

[Configuration of Element Image Generating Device for Viewpoint Position]
Next, with reference to FIG. 3, the configuration of the viewpoint position element image generation device 1 according to the embodiment of the present invention will be described.
As shown in FIG. 3, the viewpoint position element image generating device 1 includes a viewpoint corresponding position specifying unit 10, a virtual space photographing unit 11, a filtering unit 12, and an element image reconstruction unit 13.

The viewpoint corresponding position specifying unit 10 detects, based on the parameters of the target display device input from the outside, the viewpoint positions of the left and right two eyes detected by the viewpoint position detection device 2 for each element lens of the target display device. The position (viewpoint corresponding position) on the display surface of the line of sight reaching the display surface of the element image passing through the lens center of the element lens is specified. Specifically, the parameters of the target display device are the number of vertical and horizontal pixels and the pixel interval which are specifications of the target display device, the number of vertical and horizontal lenses of the lens array, the lens interval, and the focal length.
The viewpoint corresponding position specifying unit 10 calculates viewpoint corresponding positions (three-dimensional coordinates) corresponding to all the element lenses at the viewpoint positions of the left and right eyes, and the virtual space photographing unit 11 and the element image reconstructing unit 13 Output to

Here, with reference to FIG. 4 (refer to FIG. 3 as needed), a method of calculating three-dimensional coordinates of the viewpoint corresponding position calculated by the viewpoint corresponding position specifying means 10 will be described.
As shown in FIG. 4, the viewpoint corresponding position specifying unit 10 sets parameters of the display surface D and the lens array La (element lens Lp group) with the pixel spacing Pb, the lens spacing Lb and the focal length F specified by the parameters. Are arranged on the virtual space by the number of pixels and the number of lenses specified in step d, and from the viewpoint position Vp, the display surface D of the line of sight b reaching the display surface D passing through the lens center Lc of each element lens Lp Let the intersection of the points be the viewpoint corresponding position R. The viewpoint corresponding position R is, for example, a subject point O ₁ on the line of sight b when the subject is closer to the display surface D than the viewpoint position Vp, or a position where the subject is farther than the display surface D the subject point O ₂ of the line of sight b when present an ideal position on the corresponding display surface D.

  Here, assuming that the three-dimensional coordinates of the viewpoint position Vp are (hx, hy, hz), the distance (visual distance) between the viewpoint position Vp and the lens array La is L, and the focal distance of the element lens Lp is F The corresponding position specifying means 10 calculates the three-dimensional coordinates of the viewpoint corresponding position R by the following equation (1).

Returning to FIG. 3, the configuration of the viewpoint position element image generation device 1 will be described.
The virtual space imaging means 11 virtually captures an image in a predetermined range including pixels corresponding to the viewpoint corresponding position specified by the viewpoint corresponding position specifying means 10. The target to be virtually photographed is three-dimensional space data such as CG stored in advance externally.

  Here, the virtual space photographing means 11 divides the pixels into predetermined sub-pixels, and the sub-pixel center and the element lens for each of the predetermined sub-regions of sub-pixels centering on the sub-pixel corresponding to the viewpoint corresponding position. The virtual camera (three-dimensional space data) is virtually photographed (perspective projection) and rendered by a virtual camera arranged at a position separated from the element lens by a visual distance on a line connecting to the lens center of.

Here, as shown in FIG. 5, the sub-pixel refers to an area obtained by dividing the pixel P into a predetermined number. Although FIG. 5 shows an example in which the pixel P is divided into 3 × 3 sub-pixels p, it may be 5 × 5, 7 × 7 or the like. Hereinafter, nine examples will be described.
Further, as shown in FIG. 5, a sub-pixel to be a target for imaging (perspective projection) virtually in a virtual space is a sub-pixel in a pixel P including the viewpoint corresponding position R.

  That is, as shown in FIG. 7, the virtual space photographing means 11 displays the display surface D and the lens array La (element lens Lp group) with the pixel spacing Pb, the lens spacing Lb and the focal length F specified by the parameters. Arrange in the virtual space for the number of pixels and the number of lenses specified by the parameters. Then, the virtual space photographing means 11 is virtually separated from the lens array La by the visual distance L on a straight line connecting the sub-pixel center pc on the display surface D and the lens center Lc of the element lens Lp. The virtual space (three-dimensional space data) is projected on a predetermined projection plane by perspective projection with the arranged virtual camera Vc as a viewpoint (sub-pixel central viewpoint position). Then, the virtual space imaging unit 11 renders an image corresponding to the element lens from an image obtained by imaging (perspective projection) of nine sub-pixel centers pc.

  In this rendering, for example, images taken at nine sub-pixel central viewpoint positions may be weighted and added according to the distance between each sub-pixel center pc and the viewpoint corresponding position R (see FIG. 5). It may be a simple average of images taken at nine sub-pixel central viewpoint positions. The rendering range is, as shown in FIG. 6, an area (here, a diamond area) for each element lens in which the lens centers Lc of the element lenses are separated at a ratio of the lens interval specified by the parameter. In addition, the size (number of pixels) of the rendering area (rhombic area) on the projection plane J perspectively projected by the virtual space photographing means 11 needs to include at least a pixel corresponding to the lens center Lc. Ten pixels are predetermined. FIG. 6 shows a diamond region of 4 horizontal pixels and 8 vertical pixels.

A virtual photographed image by perspective projection can be calculated using a general perspective projection matrix.
Here, the perspective projection matrix will be described with reference to FIG. As shown in FIG. 8, the range of the projection target is the −near plane from the z-axis to the −far plane, the projection range of the −near plane is the maximum of x coordinates, the minimum is left, and the minimum is y maximum Is the top, and the minimum is the bottom. At this time, a coordinate point X (P _x P _y P _z 1) ^T (T is transposed) in the three-dimensional space is projected onto the −near plane of the frustum, and (s _x s _y s _z ) ^T (where Assuming that −1 ≦ s _x , s _y , s _z ≦ 1), the perspective projection matrix is expressed by the following equation (2) using homogeneous coordinates (s _x s _y s _z w) ^T expressed.

Here, since the projection position on the −near plane is a position normalized in the range of “−1” and “1”, the pixels of the actual projection plane are of the normalized position. The pixel position may be set according to the ratio.
Returning to FIG. 3, the configuration of the viewpoint position element image generation device 1 will be described.

As described above, the virtual space photographing unit 11 generates rendering images as many as the number of element lenses, and generates a photographed image composed of the rendering images.
By this, the virtual space photographing means 11 generates two photographed images observed from the viewpoint position for each of the left and right eyes. The virtual space photographing unit 11 outputs the generated photographed image to the filtering unit 12.
As described above, the virtual space photographing unit 11 can reproduce blur in a stereoscopic image to be displayed by performing perspective projection by expanding the range up to a position near the viewpoint corresponding position.

The filtering unit 12 filters an area corresponding to an element lens in the photographed image for each of the left and right eyes generated by the virtual space photographing unit 11. That is, as shown in FIG. 6, the filtering unit 12 performs filtering in a filter area (here, a diamond area) divided by horizontal and vertical pixel intervals of the lens center Lc in the projection plane J (photographed image) .
By this, the filtering means 12 calculates the value of a part (one pixel) of the element image corresponding to the element lens for each of the left and right eyes.

Specifically, the number of pixels in the filter area is I, the filter coefficient is a _i (i is an identification number of the pixel in the filter area), the pixel value of the photographed image in the filter area is w _i , and the filter area is S In this case, the filtering unit 12 calculates a pixel value R _j (j is an identification number of an element lens) corresponding to an element lens according to the following equation (3).

For example, the filtering means 12 averages the pixel values in the filter area. In this case, the filter coefficients _ai are all "1".
In addition, the filtering unit 12 may perform weighted averaging by increasing the weight (filter coefficient a _i ) as the pixel value in the filter area is closer to the lens center, for example.

As described above, the filtering unit 12 can suppress aliasing (folding noise) caused by arranging the element lenses at discrete positions by averaging pixel values in the filter area.
The filtering unit 12 filters the photographed image for each of the left and right eyes, and outputs the filtered image to the elemental image reconstruction unit 13.

  The element image reconstruction means 13 reconstructs an element image for the viewpoint position of the observer displayed on the element image display device 3 from the photographed image (filtered image) for each of the two left and right eyes filtered by the filtering means 12 It is.

That is, the elemental image reconstruction means 13 calculates the pixel value (pixel value R _j of the equation (3) above) of each element lens of the filtered image by the viewpoint corresponding position of the element lens calculated by the viewpoint corresponding position specifying means 10 And the pixel value of the pixel including At this time, the elemental image reconstruction means 13 specifies the pixel values of the pixels including the viewpoint corresponding positions of the left and right eyes calculated by the viewpoint corresponding position specifying means 10 for the left and right filtered images.

  Then, the elemental image reconstruction unit 13 assigns an image signal (for the viewpoint position) in which the pixel value of the corresponding filtered image is assigned to the two-dimensional coordinate position (xy coordinate position) of the viewpoint corresponding position of left and right two eyes for each element lens. Elemental images are generated and output to the element image display device 3.

  As described above, by forming the viewpoint position element image generation device 1, the viewpoint position element image generation device 1 generates only a part of the element image corresponding to the viewpoint position of the observer M. Compared to the conventional method of generating an element image corresponding to a viewpoint, it is possible to generate an element image for viewpoint position at high speed while suppressing the memory capacity to be used. Further, as described above, since the viewpoint position element image generating device 1 can realize high speed, it is possible to simulate a stereoscopic image of a moving image.

  Note that the viewpoint position element image generation device 1 can be operated by a program (viewpoint position element image generation program) that causes a computer (not shown) to function as the above-described units.

[Operation of Element Image Generating Device for Viewpoint Position]
Next, with reference to FIG. 9 (refer to FIG. 3 for the configuration as appropriate), the operation of the viewpoint position element image generation device 1 according to the embodiment of the present invention will be described.

First, the viewpoint position element image generation apparatus 1 inputs the parameters (number of pixels in vertical and horizontal directions, number of pixels in vertical and horizontal directions, number of lenses in vertical and horizontal directions of lens array, focal length) of the target display device To do (step S1).
Thereafter, the viewpoint position element image generation device 1 inputs the viewpoint positions of the left and right two eyes outputted from the viewpoint position detection device 2 by the viewpoint corresponding position specifying means 10 (step S2).

Then, the viewpoint position element image generation device 1 causes the viewpoint corresponding position specifying unit 10 to display the position of the display surface of the element image corresponding to the viewpoint position input in step S2 based on the parameters input in step S1. The viewpoint corresponding position is calculated (step S3).
That is, as shown in FIG. 4, the viewpoint corresponding position specifying unit 10 displays the display surface D of the element image with the pixel interval Pb of the target display designated by the parameters, the lens interval Lb of the element lens Lp and the focal length F. A lens array La is disposed on a virtual space. Then, the viewpoint corresponding position specifying means 10 calculates, as the viewpoint corresponding position R, an intersection point of the line of sight b passing through the lens center Lc of the element lens Lp with the display surface D from the viewpoint position Vp. Here, the viewpoint corresponding position specifying means 10 calculates viewpoint corresponding positions R corresponding to all the element lenses for the viewpoint positions Vp of the left and right eyes.

  Then, the viewpoint position element image generation device 1 causes the virtual space photographing means 11 to select the subpixel center and lens center of each of the subpixels corresponding to the viewpoint corresponding position identified in step S3 and the peripheral subpixels determined in advance. The virtual space is photographed (perspective projection) with a virtual camera on a line passing through (step S4). This virtual space is virtual three-dimensional space data stored in advance externally.

  Further, the viewpoint position element image generation device 1 generates a rendering image (photographed image) for each element image from the image for each sub-pixel photographed at step S4 by the virtual space photographing means 11 (step S5). By this, the virtual space photographing means 11 generates respective photographed images of the left and right eyes.

  Then, the viewpoint position element image generation device 1 filters the rendering area corresponding to the element lens in the rendering image (captured image) for each of the two left and right eyes generated in step S5 by the filtering unit 12 (step S6) . That is, the filtering means 12 averages the image of the rendering area for each element lens.

  Then, the viewpoint position element image generation device 1 reconstructs the element image for the viewpoint position of the observer from the filtered image averaged in step S6 by the element image reconstruction means 13 (step S7). That is, the element image reconstruction unit 13 assigns an image signal (viewpoint position) to which the pixel value of the corresponding filtered image is assigned to the viewpoint corresponding position (xy coordinate position) of the left and right two eyes identified in step S2 for each element lens. Element image) is generated.

  Then, the element image generating device for viewpoint position 1 outputs the image signal (element image for viewpoint position) generated in step S6 to the element image display device 3 by the element image reconstructing means 13 (step S8).

By the above-described operation, the viewpoint position element image generating device 1 can generate only a partial element image corresponding to the viewpoint position of the observer and can present it to the observer.
As a result, the viewpoint position element image generation device 1 can generate the viewpoint position element image at high speed while suppressing the memory capacity to be used compared to the conventional method.

As mentioned above, although composition and operation of an element picture generating device 1 for viewpoint position concerning an embodiment of the present invention were explained, the present invention is not limited to this embodiment.
For example, in the virtual space photographing means 11 and the filtering means 12 here, the object to be rendered is a diamond region as shown in FIG. 6, but this may be a region according to the arrangement of element lenses. For example, if the horizontal interval and the vertical interval of the element lenses are the same, the area to be rendered may be a square area.

S Integral 3D Simulation System 1 Element Image Generating Device for Viewpoint Position 10 Point-to-Point Corresponding Position Identification Means 11 Virtual Space Photographing Means 12 Filtering Means 13 Element Image Reconstruction Means 2 Viewpoint Position Detection Device 3 Element Image Display Device

Claims (5)

It is a partial element image that displays the three-dimensional image corresponding to the viewpoint position of the observer in order to simulate the three-dimensional image displayed by the target display device to be simulated by the integral three-dimensional method on another display device. A viewpoint position element image generation apparatus that generates a viewpoint position element image, comprising:
The target display device is virtually arranged on the same coordinate system as the other display devices, and the lens centers of the element lenses in the target display device are determined from the viewpoint position of the observer detected by the viewpoint position detection device. Viewpoint corresponding position specifying means for calculating the position corresponding to the viewpoint of each of the element lenses passing through and reaching the display surface of the target display device;
Three-dimensional space data was previously determined by perspective projection with a virtual camera arranged from the viewpoint corresponding position to the visual distance position to the observer passing through the lens center of the element lens corresponding to the viewpoint corresponding position Virtual space photographing means for virtually photographing on a projection plane and rendering in a predetermined range for each of the element lenses;
Filtering means for filtering an image rendered for each of the element lenses by the virtual space imaging means, and calculating pixel values corresponding to the element lenses;
An element image for generating the viewpoint position element image by assigning the pixel value of each element lens calculated by the filtering unit to a pixel corresponding to the viewpoint corresponding position of each element lens of the other display device Reconstruction means,
An apparatus for generating an element image for viewpoint position, comprising:   The virtual space photographing means is configured to, from a plurality of positions in a predetermined near eye area including the eye point corresponding position, pass through the lens center of an element lens corresponding to the eye point corresponding position to view to the observer The virtual camera arranged at the distance position captures a plurality of three-dimensional space data, and performs rendering by weighting addition according to the distance between the viewpoint corresponding position and the plurality of positions. An element image generating device for viewpoint position described.   2. The image processing apparatus according to claim 1, wherein the filtering unit calculates a pixel value corresponding to the element lens by averaging within a rendering range an image rendered for each element lens of the target display device. An element image generating apparatus for a viewpoint position according to Item 2.   The viewpoint position element image generation program for causing a computer to function as the viewpoint position element image generation device according to any one of claims 1 to 3. An integral three-dimensional simulation system for simulating, with another display device, a three-dimensional image displayed by a target display device to be simulated by the integral three-dimensional method,
A viewpoint position detection device that detects a viewer's viewpoint position;
The viewpoint position element image generation apparatus according to any one of claims 1 to 3, which generates a viewpoint position element image for displaying a stereoscopic image corresponding to the observer's viewpoint position.
An element image display apparatus which is the other display apparatus for displaying the element image for viewpoint position generated by the element image generating apparatus for viewpoint position;
An integral three-dimensional simulation system comprising:

Images