JP2012084105A - Stereoscopic image generation device and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic image generation device generating a display image with a small calculation cost even when not meeting a condition that each interval between element lenses has to be an integral multiple of a pixel size of an element image.SOLUTION: A stereoscopic image generation device 1 comprises: stereoscopic display device modeling means 11 to which various parameters are input; regularized image setting means 12 for setting an image size of a regularized image by the parameters; filter parameter setting means 13 for setting a filter application area; virtual camera photographing means 14 for generating a virtual photograph image by photographing a display object by a virtual camera; a first filter 15 for reducing folding noises caused by sampling in a lens array L; regularized image generation means 16 for generating a regularized image; and a second filter 17a for reducing folding noises caused by sampling in an image display panel 21.

Description

  The present invention relates to a technique for generating a three-dimensional image from a three-dimensional shape model to be displayed in a three-dimensional display method such as an integral method or a lenticular method.

  In recent years, research and development of 3D display devices have progressed, and autostereoscopic 3D display devices such as an integral method and a lenticular method that allow a viewer to view a 3D image without using special glasses have been developed. . For example, an integral-type stereoscopic display device includes a high-definition image display panel and a two-dimensional lens array disposed on the front surface thereof, and a stereoscopic image is reproduced when an element image is displayed on the high-definition image display panel. Light rays from each pixel of the element image are emitted in a direction determined by the pixel position and the position of the element lens of the lens array. That is, as many rays as the number of pixels of the element image are emitted from one element lens. Here, when the condition that the distance between the element lenses is an integral multiple of the pixel size of the element image, the direction of the light rays emitted from each element lens is finite. Therefore, the orthogonal projection is performed by collecting the light rays in the same direction. It can be represented as an image. That is, the light ray group emitted from the stereoscopic display device can be represented by a finite number of orthographic images.

  Here, techniques for converting a three-dimensional shape model obtained by computer graphics (CG) or a range sensor into an integral stereoscopic image have been proposed (for example, Non-Patent Documents 1 and 2). These techniques described in Non-Patent Documents 1 and 2 are basic techniques for converting a three-dimensional shape model into an integral stereoscopic image. For example, in the technique described in Non-Patent Document 1, a three-dimensional shape model of computer graphics is used as a subject, a lens array and a lens for depth control are installed in a three-dimensional virtual space. A three-dimensional image is generated.

  In other words, in the techniques described in Non-Patent Documents 1 and 2, if orthographic projection can be used, a display image can be generated from a three-dimensional shape model by rendering light rays of the number of element lenses of the lens array once. Can do. In this case, in the techniques described in Non-Patent Documents 1 and 2, it is possible to generate a display image from a three-dimensional shape model with a smaller amount of computation than in the case where center projection is used.

Non-Patent Document 3 proposes a technique for generating an element image from CG data using a ray tracing method.
Non-Patent Document 4 relates to a technique of generating an element image using a graphics processing unit (GPU), and a rendering method using Z-buffer is proposed.

Non-Patent Document 5 proposes a technique for generating element images at high speed in order to stereoscopically display a medical 3D object in an integral manner. In the technique described in Non-Patent Document 5, the speed is increased by a distributed computer, and each node uses the same three-dimensional object.
Non-Patent Document 6 proposes a technique for generating an element image from a three-dimensional object in consideration of reduction in image quality degradation caused by a lens array.

  Patent Document 1 proposes an invention for generating element images at high speed in a three-dimensional space composed of a moving part and a stationary part. In the invention described in Patent Document 1, an element image is generated only by a moving part and is synthesized with an element image of a calculated still part.

Patent Document 2 proposes an invention for generating an integral 3D image with a high degree of freedom using general-purpose computer graphics.
Patent Document 3 proposes an invention for generating an element image from a three-dimensional object in consideration of reduction in image quality degradation caused by a lens array.

JP 2003-109042 A JP 2006-146597 A JP 2009-175866 A

Athineos, Spyros S ,: “Physical modeling of a microlens array setup for use in computer generated IP”, Proceedings of the SPIE, Volume 5664, pp.472-479 (2005). Huy Hoang Tran, et al: “Interactive 3D Navigation System for Image-guided Surgery”, Journal of Virtual Reality, 8 (1), pp. 9-16 (2009) Nakajima et al .: “High-speed image generation method of 3D display using the principle of Integral Photography”, The Institute of Image Information and Television Engineers, Vol.54, No.3, pp.420-425 (2000). Koike, “Rendering integral photography images using programmable graphics hardware”, IPSJ Research Report, 2003-CG-113, pp.70-74 (2003). N. Sakai, “High Performance Computing for Parallel Rendering in Surgical Autostereoscopic Display and navigation”, CARS2003; ICS 1256, pp.403-407 (2003). Miwa katayama, Yuichi Iwadate: “A method for converting three-dimensional models into auto-stereoscopic images based on integral photography”, Proceedings of SPIE, vol.6805, p.68050Z.1-68050Z.8 (2008)

  However, in the techniques described in Non-Patent Documents 1 and 2, if the distance between the element lenses does not satisfy the condition of an integral multiple of the pixel size of the element image, the direction of the light rays differs from element lens to element lens. It cannot be represented by one orthogonal projection image.

Furthermore, in the techniques described in Non-Patent Documents 1 and 2, when orthographic projection is used, geometric distortion occurs in the orthographic projection image. In this case, the techniques described in Non-Patent Documents 1 and 2 have a problem that an extra calculation cost for correcting the geometric distortion is required.
In addition, the inventions described in Patent Documents 1 to 3 and the techniques described in Non-Patent Documents 3 to 6 do not solve the problems of the techniques described in Non-Patent Documents 1 and 2.

  Therefore, the present invention solves the above-described problem, and a stereoscopic image generating apparatus and its program capable of generating a display image with low calculation cost even when the interval between the element lenses does not satisfy the condition of an integral multiple of the pixel size of the element image The purpose is to provide.

  In order to solve the above-described problem, the stereoscopic image generating device according to the first invention of the present application is a three-dimensional shape as a display target to be displayed on a stereoscopic display device including a lens array in which element lenses are two-dimensionally arranged and image display means. A stereoscopic image generating apparatus that captures a model with a virtual camera that is a virtual imaging camera and generates a display image for the stereoscopic display apparatus, the parameter input means, a normalized image pixel size calculating means, a normal image It comprises a normalized image pixel position calculating means, a virtual camera position calculating means, a virtual captured image generating means, a normalized image generating means, and a display image generating means.

  According to such a configuration, the stereoscopic image generating apparatus receives the focal length of the element lens, the interval of the element lens, and the pixel interval of the image display unit by the parameter input unit. Further, the stereoscopic image generating apparatus is configured such that the normalized image pixel size calculating unit divides the element lens interval by the pixel interval of the image display unit and multiplies the element lens interval by a preset constant. The pixel size of the normalized image is a value that is a fraction of an integer with respect to the distance between the element lenses by calculating the number of pixels and dividing the distance between the element lenses by the number of pixels between the element lenses. calculate. Then, the stereoscopic image generating device uses the normalized image pixel position calculation unit to multiply the pixel coordinates of the display image by the pixel interval of the image display unit and divide by the pixel size of the normalized image. The pixel coordinates of the digitized image are calculated. In this way, the stereoscopic image generating apparatus prepares a normalized image whose pixel size is 1 / integer with respect to the interval between the element lenses.

  In addition, the stereoscopic image generation apparatus multiplies the distance set in advance so as to indicate the position from the lens array to the position of the virtual camera by the virtual camera position calculation unit, and the pixel size of the normalized image to multiply the element lens. The position of the virtual camera is calculated based on the value divided by the focal length. Then, the stereoscopic image generating apparatus projects the three-dimensional shape model obliquely onto the projection plane using the transformation matrix at the position of the virtual camera calculated by the virtual camera position calculating unit by the virtual captured image generating unit. A virtual photographed image obtained by photographing the three-dimensional shape model with the virtual camera is generated. That is, the stereoscopic image generating apparatus can generate a virtual captured image in which pixels are aligned vertically and horizontally without causing geometric distortion in the virtual captured image because the captured image generating unit performs oblique projection.

The stereoscopic image generating apparatus generates the normalized image by assigning pixel values of the pixels of the virtual photographed image to the pixels of the normalized image using a predetermined coordinate conversion formula by the normalized image generating unit. In other words, the pixels of the normalized image and the pixels of the virtual photographed image are arranged on a straight line by the oblique projection described above, and the stereoscopic image generating apparatus uses the prepared normal values of the pixel values of the virtual photographed image according to the straight line. Assigned to each pixel of the digitized image. Therefore, the stereoscopic image generating apparatus forms an element image on the normalized image.
Furthermore, the stereoscopic image generating apparatus generates the display image from the normalized image generated by the normalized image generating unit by the display image generating unit.

Further, in the stereoscopic image generating apparatus according to the second invention of the present application, the virtual camera position calculating means further calculates a direction from the position of the virtual camera to the center position of the lens array as a shooting direction of the virtual camera, It said virtual captured image generating means are defined by equation (9) for the model view matrix M _O of the first viewpoint the virtual camera located on the perpendicular of said lens array toward the center position of the lens array, wherein model view matrix M _C in a second viewpoint, wherein the virtual camera at the position of the virtual camera virtual camera position calculating means is calculated by facing the shooting direction is defined by the equation (10), said second viewpoint from the first viewpoint transformation matrix T _C to are defined by the formula (11), orthogonal projection conversion matrix ortho is defined by equation (12), parallel movement distance of the lens array in the projection coordinate system L _{d ist} is defined by equation (15), translation matrix T _Z of the lens array in the projection coordinate system is defined by equation (16), the first to the third axial displacement of the projected coordinate system, Biaxial change amounts S _X and S _Y are defined by Expression (19), an oblique projection transformation matrix Oblique is defined by Expression (20), and the translation inverse matrix of the lens array in the projection coordinate system When T _Z ⁻¹ is defined by Equation (21) and the projection matrix P is defined by Equation (22), the transformation matrix F at the position of the virtual camera defined by Equation (23) is It is characterized by obliquely projecting a three-dimensional shape model.

  Here, in 3D graphic languages such as OpenGL (registered trademark), the transformation matrix F at the position of the virtual camera may not be defined. Even in this case, the stereoscopic image generating apparatus can derive the transformation matrix F at the position of the virtual camera from the transformation matrix ortho for orthogonal projection defined in the 3D graphic language.

  Further, the stereoscopic image generating apparatus according to the third invention of the present application surrounds the virtual captured image generated by the virtual captured image generating means with a midpoint of a line segment connecting pixels corresponding to the center position of the element lens. And a first aliasing noise reduction unit that performs low-pass filter processing for each filter application region.

Here, the observer of the stereoscopic display device sees the stereoscopic image through the lens array. This is equivalent to sampling a three-dimensional image according to the distance between the element lenses of the lens array. For this reason, in the lens array, aliasing noise (folding distortion) due to sampling occurs.
Therefore, the stereoscopic image generating apparatus regards the pixel corresponding to the center of the element lens of the virtual captured image as the sample point by the first aliasing noise reduction unit, and passes the low-pass for each filter application region centered on the sample point. Apply filtering. Thereby, the stereoscopic image generating apparatus can reduce aliasing noise caused by sampling in the lens array.

  Further, in the stereoscopic image generating apparatus according to the fourth invention of the present application, the normalized image pixel size calculating means is preset with an even number exceeding 2 as the constant, and the display image generating means is adapted for the normalized image. It is further characterized by further comprising second aliasing noise reduction means for performing low-pass filter processing.

Here, the observer of the stereoscopic display device sees the stereoscopic image reproduced by the image display means. This is equivalent to sampling a stereoscopic image according to the interval between element images. For this reason, aliasing noise (folding distortion) due to sampling occurs in the image display means.
Therefore, the stereoscopic image generating apparatus prepares a normalized image whose pixel size is equal to or smaller than ½ of the pixel size of the image display unit and is an integer of 1 with respect to the distance between the element lenses. Then, the stereoscopic image generating device performs low-pass filter processing on the normalized image according to the pixel size of the image display unit. As a result, the stereoscopic image generating device can reduce aliasing noise caused by sampling in the image display means.

  Note that the stereoscopic image generating apparatus according to the first invention of the present application uses a general computer, parameter input means, normalized image pixel size calculating means, normalized image pixel position calculating means, virtual camera position calculating means, virtual photographed image generating It can also be realized by a stereoscopic image generation program that functions as a means, a normalized image generation unit, and a display image generation unit.

The present invention has the following excellent effects.
The first invention of the present application prepares a normalized image whose pixel size is 1 / integer with respect to the interval between the element lenses. In the first invention of this application, a virtual captured image in which no geometric distortion has occurred is generated by oblique projection, and pixel values of pixels of the virtual captured image are assigned to each pixel of the normalized image. Therefore, the first invention of the present application does not need to correct geometric distortion as in orthographic projection when the distance between the element lenses does not satisfy the condition that the element size is an integer multiple of the pixel size of the element image. Can be generated.

Since the 3D graphic language in which the transformation matrix F at the position of the virtual camera is not defined can be used in the second invention of the present application, the versatility of the stereoscopic image generating apparatus can be enhanced.
In the third invention of the present application, aliasing noise caused by sampling in the lens array can be reduced, so that the display image can be of high quality.
In the fourth invention of the present application, aliasing noise caused by sampling in the image display means can be reduced, so that the display image can be of high quality.

It is the schematic which shows the outline of the three-dimensional display apparatus in this invention. It is a block diagram which shows the structure of the stereo image production | generation apparatus which concerns on embodiment of this invention. It is a conceptual diagram which shows modeling of the three-dimensional display apparatus in this invention, and is a figure which shows the positional relationship with an element image, an image display panel, and the pinhole of a lens array. In this invention, it is a figure which shows the positional relationship of the normalized image and the pinhole of a lens array. It is a block diagram which shows the structure of the virtual camera imaging | photography means of FIG. In this invention, it is explanatory drawing explaining imaging | photography with a virtual camera, and is a figure which shows the positional relationship of a lens array, a normalized image, and a three-dimensional shape model. In this invention, it is explanatory drawing explaining imaging | photography with a virtual camera, and is a figure which shows the positional relationship of the sample point on a lens array, and a normalized image. In this invention, it is explanatory drawing explaining imaging | photography with a virtual camera, and is a figure which shows the positional relationship of a normalized image, a lens array, and a projection surface. In this invention, it is a figure explaining orthographic projection and oblique projection, (a) is a figure of orthographic projection, (b) is a figure of oblique projection. In this invention, it is a figure explaining the viewpoint movement of a virtual camera. In this invention, it is a figure explaining the confrontation of a projection surface and a lens array. In this invention, it is a figure explaining the parallel movement of the lens array in a projection coordinate system, (a) is a figure before a parallel movement, (b) is a figure after a parallel movement. In this invention, it is a figure explaining the oblique projection of a three-dimensional shape model. It is explanatory drawing explaining the low-pass filter process by the 1st filter of FIG. It is explanatory drawing explaining the production | generation of the display image by the 2nd filter of FIG. 3 is a flowchart illustrating an overall operation of the stereoscopic display device of FIG. 2. It is a flowchart which shows operation | movement of the virtual camera imaging | photography means of FIG. In this invention, it is a figure which shows the structure of the three-dimensional video generation system which produces | generates the integral three-dimensional video of a dynamic three-dimensional shape model.

[Outline of stereoscopic display device]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

  First, as a premise of the stereoscopic image generating apparatus 1 (see FIG. 2) according to the embodiment of the present invention, an outline of the stereoscopic display apparatus 2 in the present invention will be described with reference to FIG.

The stereoscopic display device 2 displays a display image (an image in which a plurality of element images are arranged vertically and horizontally) generated by the stereoscopic image generation device 1 (see FIG. 2) by an integral method, and displays an image such as a liquid crystal display. comprising a panel (image display means) 21, and a lens array L _a placing the element lenses L _P two-dimensionally.

The image display panel 21 has an image display surface 21a on the side of the lens array L _A, displays an elemental image light displayed (three-dimensional shape model) is emitted is recorded.
Lens array L _A is a micro convex lens element lenses L _P is a two-dimensional array arranged in rows and columns.

First, the stereoscopic display device 2 displays a display image (hereinafter referred to as display image I) generated by the stereoscopic image generation device 1 on the image display surface 21 a of the image display panel 21. As shown in FIG. 1, light from each pixel element image orientation is the element lenses L _P corresponding to each pixel. Therefore, the stereoscopic display device 2 reproduces the light beam emitted from the display object, and forms a stereoscopic image as if the display object exists. That is, the observer (not shown), through element lenses L _P of the lens array L _A, each of the element images by viewing, it is possible to visually recognize a stereoscopic image.
At this time, a display image for displaying on the stereoscopic display device 2 is required. Therefore, the display image is generated by using the stereoscopic image generating apparatus 1 according to the embodiment of the present invention.

[Configuration of stereoscopic image generation apparatus]
Hereinafter, with reference to FIG. 2, the configuration of the stereoscopic image generating apparatus 1 according to the embodiment of the present invention will be described.
As shown in FIG. 2, the stereoscopic image generating device 1 shoots a three-dimensional shape model as a display target to be displayed on the stereoscopic display device 2 with a virtual camera, and generates a display image I of the stereoscopic display device 2. Yes, a stereoscopic display device modeling means (parameter input means) 11, a normalized image setting means (normalized image pixel size calculating means) 12, a filter parameter setting means (filter application area setting means) 13, and a virtual camera The imaging unit 14 includes a first filter (first aliasing noise reduction unit) 15, a normalized image generation unit 16, and a display image generation unit 17.

The stereoscopic display device modeling means 11 receives parameters (F, R _H , R _V , P) for modeling the stereoscopic display device 2 and outputs these parameters to the normalized image setting means 12. These parameters are defined as shown in FIG.

F: focal length of the element lenses _{L P} (distance between the image display panel 21 and the lens array _{L A)}
R _H: the lens array _L horizontal spacing element lenses _{L P} in _{A R V:} lens array _L vertical spacing element lenses _{L P} in _A P: pixel spacing of the image display panel 21

As in this embodiment, the element lens L _P are offset for each row, a √3 / 2 times the horizontal spacing R _H of vertical spacing R _V elements lens L _P element lenses L _P explain.
In addition, it is _{assumed that} the horizontal interval _RH of the element lens L _{P and} the vertical interval R _{V of the} element lens L _P are not an integral multiple of the pixel interval of the element image G.
In the stereoscopic display device 2, it is regarded the center position of the element lenses L _P pinhole H. That is, the center position of the element lenses L _P is the position of the pinhole H.

Since the interval R _V vertical horizontal gap R _H and element lenses L _P element lenses L _P is not an integer multiple of the pixel spacing element image G, taken using a virtual camera (isometric camera) Even in this case, it is not possible to directly acquire the light beam at the pixel position of the element image G. Therefore, as shown in FIG. 4, in the process of shooting a virtual camera (calculation process), to define the pixel spacing element image G, the vertical direction in the horizontal direction between R _H and element lenses L _P element lenses L _P distance R _V of preparing the image surface as an integral multiple of the pixel spacing element image G. Hereinafter, the image on this image plane is referred to as a normalized image (normalized element image) I ′.

It also defines world coordinates that indicate the physical location. The world coordinates, element lenses L _P lens center which is located in the center of the lens array L _A (pin holes) as the origin, X-axis lateral of the lens array L _A, the longitudinal direction of the Y-axis and a lens array surface The vertical direction is the Z axis. Further, (i, j) is defined as pixel coordinates of the display image I, that is, image coordinates of the entire element image G. It is assumed that the origin of the image coordinates is the pixel closest to the origin of the world coordinates.

Normalized image setting unit 12, the normalized image I ', as represented by the following formula (1), the vertical gap horizontal gap R _H and element lenses L _P element lenses L _P R by dividing the _V pixel number M between elements lens L _P, to calculate the vertical pixel size P _V of the 'horizontal pixel size P _H and the normalized image I' of normalized image I.

Then, the normalized image setting unit 12 outputs these parameters (F, R _H , R _V , P, P _H , P _V ) to the filter parameter setting unit 13. In this case, the normalized image setting means 12, as represented by the following formula (2), the constant by dividing the horizontal gap R _H of the element lenses L _P in the pixel spacing P of the image display panel 21 K based on the value obtained by multiplying the calculated number of pixels M between the element lenses L _P.

Here, in formula (2), “<a>” means rounding off a to an integer. _{That, "<R H / P>} " becomes the number of pixels between the element lenses L _P in FIG. 3, for low-pass filtering in a second filter 17a to be described later, to K times the value. Here, the constant K can be set to an arbitrary value. Further, when it is set in the even that the constant K exceeds 2, the pixel size of the normalized image I 'is less than half the pixel size of the image display panel 21, and the horizontal spacing of the element lenses L _P the integral submultiple against R _H and element lenses _{L P} of vertical spacing _{R V.} Thereby, the low-pass filter processing by the second filter 17a described later can be easily applied.

The filter parameter setting unit 13 receives parameters (F, R _H , R _V , P, P _H , P _V ) from the normalized image setting unit 12 and a preset constant N. The constant N is a value that indicates that sampling by a constant multiplying the number of pins holes in the lens array L _A. The filter parameter setting means 13, as the origin (0,0) of the later-described local coordinates (u, v), surrounded at the midpoint of the line segment connecting the pixel corresponding to the center position of the element lenses L _P A filter application area A _R (see FIG. 14) is set on the virtual photographed image.

Thereafter, the filter parameter setting means 13 outputs the parameters (F, R _H , R _V , P, P _H , P _V ) to the virtual camera photographing means 14 and sets the set filter application area _AR to the first. Output to the filter 15.
The filter application area A _R is for indicating a range in which the first filter 15 to be described later to perform low pass filtering, the details will be described later.

  The virtual camera photographing means 14 generates a virtual photographed image by virtually photographing a display target with the virtual camera Vc (see FIG. 6). For this reason, as shown in FIG. 5, the virtual camera photographing means 14 includes a normalized image pixel position calculating means 141, a pinhole position calculating means 142, a virtual camera position calculating means 143, and a virtual photographed image generating means 144. Is provided.

The normalized image pixel position calculation unit 141 receives parameters (F, R _H , R _V , P, P _H , P _V ) from the filter parameter setting unit 13. Here, using the above-described equation (1), the world coordinates of the normalized image I ′ can be expressed by the following equation (3).

Therefore, the normalized image pixel position calculation unit 141 multiplies the pixel coordinate (i, j) of the display image I by the pixel interval P of the image display panel 21 as represented by the following equation (4), The pixel coordinates of the normalized image I ′ are calculated by dividing the pixel sizes P _H and P _V of the normalized image I ′. That is, the pixel coordinates of the display image I can be converted into the pixel position of the normalized image I ′ using the following equation (4).
In equation (4), (i ′, j ′) represents the pixel coordinates of the normalized image I ′.

Then, the normalized image pixel position calculating unit 141 uses the parameters (F, R _H , R _V , P, P _H , P _V ) and the calculated pixel coordinates of the normalized image I ′ as the pinhole position calculating unit 142. Output to.

The pinhole position calculating unit 142 receives parameters (F, R _H , R _V , P, P _H , P _V ) and the calculated pixel coordinates of the normalized image I ′ from the normalized image pixel position calculating unit 141. Is done. Then, the pinhole position calculating means 142 calculates the position of the nth pinhole H in the horizontal direction and the mth pinhole H in the vertical direction using the following equation (5).
In Expression (5), “%” is a symbol indicating a remainder operation, and H (n, m) is the position of the pinhole H.

Then, the pinhole position calculation means 142 uses the pixel coordinates of the normalized image I ′, the position of the pinhole H, and the parameters (F, R _H , R _V , P, P _H , P _V ) as the virtual camera position. It outputs to the calculation means 143.

The virtual camera position calculation unit 143 receives the pixel coordinates of the normalized image I ′, the position of the pinhole H, and the parameters (F, R _H , R _V , P, P _H , P _V ) from the pinhole position calculation unit 142. Are entered. Then, the virtual camera position calculation unit 143 generates a unit vector to be described later, and then calculates the position of the virtual camera Vc.
Here, the pixel of the element image G corresponding to the pinhole position H (n, m) is represented as I ′ _n , _m (u, v). In addition, (u, v) is the local image coordinates on the normalized image I ′ with H (n, m) as the origin.

First, as shown in FIG. 6, the virtual camera position calculation unit 143 sets the pinhole position to H (0, 0) in the normalized image I ′, and the pixel of the element image G corresponding to this H (0, 0). _{I '0, 0 (u,} v) for each pixel in the, using equation (6) below, the pixel _{I' 0, 0 (u,} v) from the pinhole H to the position of (0, 0) A unit vector (i _x , i _y , i _z ) is generated.
Here, the pixels I ′ ₀ , ₀ (u, v) = (X _P , Y _P , −F) = (uP _H , vP _V , −F). Further, the pinhole position H (0, 0) = (X _R , Y _R , 0) = (0, 0, 0).
This unit vector indicates the direction in which the pixel I ′ ₀ , ₀ (u, v) is visually recognized on the extension line.

Next, the virtual camera position calculation unit 143 calculates the position of the virtual camera Vc based on the generated unit vector and the distance Length (see FIG. 8). Specifically, the virtual camera position calculation unit 143 multiplies the distance Length by the pixel size (P _H , P _V ) of the normalized image I ′ as represented by the following formula (7). Based on the value divided by the focal length F of the lens, the position of the virtual camera Vc is calculated.

The distance Length is the distance from the origin lens array _{L A} to the virtual camera Vc (e.g., 2 meters).
The virtual camera Vc is a virtual photographing camera that photographs a display target, and represents a viewing distance when the observer A views the stereoscopic display device 2. At this time, the distance from the virtual camera Vc to the stereoscopic display device 2 is assumed to be equal to the distance Length. Further, the virtual camera Vc from the origin lens array L _A, assumed that apart than 3D geometric model is displayed.

Then, the virtual camera position calculating unit 143 calculates the reverse direction r _z (see FIG. 6) of the generated unit vector as the shooting direction of the virtual camera Vc. That is, the imaging direction of the virtual camera Vc is a direction from the camera position calculated by the formula (7) to the origin (center position) of the lens array L _A. Thereafter, the virtual camera position calculation unit 143 outputs the calculated position of the virtual camera Vc to the virtual captured image generation unit 144.

  The virtual photographed image generating means 144 receives the position of the virtual camera Vc from the virtual camera position calculating means 143, and displays a display target by a transformation matrix F at the position of the virtual camera Vc (see formula (23) described later). Is obliquely projected onto the projection plane, thereby generating a virtual photographed image obtained by photographing the display target with the virtual camera Vc.

<Shooting with a virtual camera>
Hereinafter, with reference to FIGS. 6 to 9, photographing by the virtual camera Vc will be described in detail (see FIG. 5 as appropriate).
In FIG. 6, only a part of the surface of the three-dimensional shape model (display target) Obj is illustrated for the sake of simplicity.
In FIGS. 7 and 8, “●” and “△” indicate sample points, “●” is a sample point at the pinhole H, and “Δ” is increased except for “●”. This is a sample point. In FIG. 7, “■” is a pixel to be calculated in the same relative position with respect to the pinhole H in each element image G. That is, the virtual captured image generation unit 144 can represent the pixel position of each element image G by local coordinates (u, v) with the pinhole H as the origin.
Further, in FIGS. 7 and 8, the number of pixels the virtual camera Vc is the lens array L _A "●" are and "△" the sample points and the same number of combined. Further, in FIG. 8, each of the light rays is indicated by an arrow.

Specifically, the virtual photographic image generating unit 144, as shown in FIG. 6, in the opposite direction r _z around the pinhole H (0,0) present on the unit vectors, the lens array L _A whole pin The stereoscopically arranged stereoscopic display device 2 is virtually photographed at an angle of view including the hole H. That is, the virtual captured image generation unit 144 uses the position of the pinhole H (0, 0) existing in the reverse direction r _z of the unit vector as the center of the virtual captured image and is parallel to the reverse direction r _z of the unit vector. and, the color of the three-dimensional shape model Obj surfaces that exist on a straight line passing through each pinhole H of the lens array L _a on the projection plane of the virtual camera Vc by isometric generates a virtual captured image.

At this time, virtual-photographed-image generating unit 144, as shown in FIGS. 7 and 8, in imaging by a virtual camera Vc, which collectively sample the light passing through the pinhole lens array L _A. Here, the virtual photographic image generating means 144, in order to apply the first filter to be described later, samples the number of rays that the number of pinholes H N times the lens array L _A (e.g., N = 4) . Then, the virtual captured image generation unit 144 acquires a light beam corresponding to the pixel located at the local coordinates (u, v) of each element image G. Thus, in the virtual photographic image, the spacing of the element lenses L _P is to meet the integral multiple of the pixel size of the elemental image G, the direction of the light beam emitted from each element lens L _P is finite, the same It becomes possible to collect light rays in the direction. In other words, the virtual photographed image generating unit 144 can represent the emitted light group by a finite number of virtual photographed images.

As described above, the virtual-photographed-image generating unit 144, since the oblique projection, lens array L _A and can parallel to the projection plane P _P (see FIG. 8), a virtual captured image pixels are aligned in rows and columns Can be generated (see FIG. 14). Thereby, the first filter 15 described later does not need to correct the geometric distortion of the virtual photographed image, and the calculation cost can be reduced.

  Hereinafter, a method for performing oblique projection will be described in detail by taking as an example the case where the virtual photographed image generating unit 144 uses OpenGL (registered trademark) as a 3D graphic language. As described above, in OpenGL (registered trademark), a conversion matrix at the position of the virtual camera is not prepared. Therefore, in the present embodiment, the transformation matrix at the position of the virtual camera is derived from the orthogonal projection transformation matrix Ortho etc. defined in OpenGL (registered trademark).

<< Explanation of orthographic / oblique projection >>
The orthographic projection and the oblique projection will be compared and described with reference to FIG. Here, a coordinate system composed of the X axis (first axis) -Y axis (second axis) -Z axis (third axis) is defined as a projected coordinate system. The Z-axis, and passing through the center position of the lens array L _A.
The projection plane P _P is to be positioned on the X-axis -Y-axis plane. The virtual camera Vc is directed in the Z-axis direction. Furthermore, 3-dimensional shape model Obj1,2 is to be positioned within the field of view _{V R} of the virtual camera Vc.

As shown in FIG. 9 (a), 3-dimensional shape model Obj1,2 is along the shooting direction of the virtual camera Vc (viewing direction) is projected (orthogonal projection) perpendicular to the projection plane _{P P} . In this case, 3-dimensional shape model is outside field of view V _R (not shown) to be clipped (cut out part of the image), is not orthogonal projection on the projection plane P _P.

Further, as shown in FIG. 9 (b), the shooting direction of the virtual camera Vc is tilted with respect to the projection plane P _P. Thus, three-dimensional shape model Obj1,2 is projected in an oblique direction with respect to the projection plane _{P P.} That is, the orthogonal projection of Fig. 9 (a), can be treated as oblique projections not tilted with respect to the projection plane P _P.

<< Virtual camera viewpoint movement >>
With reference to FIG. 10, the viewpoint movement of the virtual camera Vc will be described.
The lens array L _A and the normalized image I ', it is assumed that arranged parallel to the X-axis (see FIG. 9) -Y-axis plane. Each viewpoint is positioned on the X-axis-Y-axis plane. Further, the virtual camera Vc, regardless of each viewpoint, always, and it faces the gaze point Reg is the center position of the lens array L _A.

Viewpoint O (first viewpoint), the virtual camera Vc is a perspective when facing gazing point Reg from the vertical line passing through the center position of the lens array L _A. The viewpoint C (second viewpoint) is a viewpoint when the virtual camera Vc faces the shooting direction (reverse direction r _z ) in FIG. 6 from the camera position calculated by Expression (7). In FIG. 10, a character (“O” or “C”) indicating a viewpoint is illustrated in a circle representing the virtual camera Vc. Further, in FIG. 10, the illustrated virtual camera Vc and field of view V _R at the viewpoint O by a broken line, illustrating the virtual camera Vc and field of view V _R at the viewpoint C by a solid line.

  When the virtual camera Vc moves from the viewpoint O to the viewpoint C, the three-dimensional shape models Obj1 and Obj1 and 2 are orthogonally projected as shown in FIG. Here, on the basis of the viewpoint O, the transformation matrix F at the viewpoint C can be expressed by the following equation (8).

Incidentally, P is a projection matrix, converts the camera coordinate system in the projection coordinate system, is to normalize the field of view V _R. Here, the projection matrix P is, for example, a 4 × 4 matrix.

M _O is a model view matrix at the viewpoint O, and converts the world coordinate system into a camera coordinate system whose viewpoint O is the origin. Here, the model view matrix M _O is represented by a 4 × 4 matrix as in the following Expression (9), and each element o _{11 to} o ₃₄ represents rotation and translation.

M _C is a model view matrix of the perspective C, it is to convert the world coordinate system to the camera coordinate system viewpoint C is the origin. Here, the model view matrix _{M C,} as in equation (10) below, is expressed by a matrix of 4 × 4, each element _c 11 _{to c 34} represents a parallel movement and rotation.

T _C is the transformation matrix from the viewpoint O to viewpoint C, represented by the following formula (11).

  ortho is a projection matrix (transformation matrix for orthographic projection) by orthographic projection, and is represented by a 4 × 4 matrix as in the following equation (12). Here, in this embodiment, in order to represent rotation and translation with one matrix, each coordinate system is extended from three dimensions to four dimensions. For this reason, for example, as in Expressions (9) to (12), one element is added to the three-dimensional vector, and it is handled as four-dimensional.

Incidentally, l is the leftmost position of the field of view V _R, r is the rightmost position of the field of view V _R. Further, t is the upper end position of the field of view V _R, b is the lower end position of the field of view V _R. Further, f is the back end position of the field of view V _R, n is the front end position of the field of view V _R.

<< Direction of projection plane and lens array >>
Referring to FIG. 11, a description will be given positive pairs of the projection plane P _P and the lens array L _A.
When moving the virtual camera Vc from the viewpoint O to the viewpoint C (see FIG. 10), the projection plane P _P is not necessarily parallel to a lens array L _A. Therefore, as shown in FIG. 11, as represented in the following equation (13) and (14), by multiplying an inverse matrix _T ^{C -1} in orthogonal projection conversion matrix ortho, projection plane _{P P} and the lens array Return _LA to parallel.
In FIG. 11, the illustrated virtual camera Vc and field of view V _R at the viewpoint O by solid lines, illustrating the virtual camera Vc and field of view V _R at the viewpoint C by dashed lines.

Here, the model view matrices M _O and M _C define the position and orientation of the three-dimensional object Obj and the polygon normal. Therefore, the model view matrix _M O, when alter the _{M C,} when performing a model shading 3-dimensional object Obj, to determine the shading color from the positional relationship between the apex and the viewpoint of the three-dimensional object Obj "SPECLAR Color" It will adversely affect. In other words, in a 3D graphic language such as OpenGL (registered trademark), the reflection of light in the three-dimensional object Obj is calculated based on the normal. In this case, the model view matrix M _O, changing the M _C the normal even end up with different, reflection of light in the three-dimensional object Obj, it deviates from the correct direction.

Therefore, in the present invention, the model view matrix M _O, without changing the M _C, as in the above-mentioned formula (14), and in changing the projection matrix P. At this time, the virtual camera Vc remains at the viewpoint C, and the shooting direction is not changed. Thus, the virtual-photographed-image generating unit 144, without receiving an adverse effect reflected from the correct direction of the light is shifted, it directly facing the projection plane P _P and the lens array L _A.

<< Translation of lens array in projected coordinate system >>
Referring to FIG. 12, described translation of the lens array L _A.
In FIG. 12 (a), shown with far surface far side of the plane of the field of view V _R in the Z-axis direction, shown as the near surface front side of the field of view V _R in the Z-axis direction.
Further, when expressed in the distance in the Z direction with the position of the virtual camera Vc (viewpoint C) as the origin, f is a distance to the near plane and n is a distance to the far plane.

It projected coordinate system, by the model view matrix _{M C} of formula (14) projection matrix P and the formula (10), the state of FIG. 12 (a). To apply isometric to the projected coordinate system, the lens array L _A is to match the projection plane P _P, it is moved parallel to the Z-axis direction. In orthogonal projection conversion matrix Ortho, Z-axis direction of the distance of the field-of-view range _{V R} becomes a near surface to the far surface. Here, in OpenGL (registered trademark), the distance from the near plane to the far plane is normalized (normalized) in the range from the minimum value −1 to the maximum value 1 (that is, Z = [− 1, 1]). ). Therefore, parallel movement distance _{L dist} lens array _{L A} in the projection coordinate system can be determined by the following equation (15).

In OpenGL (R), in the Z-axis direction, the projection plane _{P P} is positioned at the center of the field of view _{V R (f + n) /} 2. Thus, in Formula (15), (Length- (f + n) / 2) term represents the translation distance of the lens array _{L A,} 2 / term normalization factor (f-n) (normalized coefficient) Represents. Thus, translation matrix T _Z of the lens array L _A in the projection coordinate system is expressed by the following equation (16).

It projected coordinate system, when translating the lens array L _A, the state of FIG. 12 (b). At this time, the projection matrix is expressed by the following equation (17).

<< An oblique projection transformation matrix >>
An oblique projection transformation matrix will be described with reference to FIG.
In order to derive a transformation matrix for oblique projection, the amount of change in the X and Y axis directions with respect to the displacement in the Z axis direction in the imaging direction in the projection coordinate system of FIG. As described above, the model view matrix M _O, Applying M _C, perspective camera coordinates of the virtual camera Vc becomes coordinate origin. When this coordinate origin is converted into the projection coordinate system of FIG. 12B, the fourth column vector in the projection matrix of Expression (17) becomes the viewpoint of the virtual camera Vc in the projection coordinate system (see Expression (18)). .

In this equation (18), X _C , Y _C and Z _C are the positions of the virtual camera Vc calculated in equation (7). Then, when the changes S _X and S _Y in the X and Y axis directions with respect to the displacement in the Z axis direction are defined by the following equation (19), a transformation matrix for oblique projection (transformation matrix for oblique projection) Oblique is: (20).

<< Clipping position correction >>
Here, the correction of the clipping position will be described.
Wherein the but was translated to the lens array L _A formula (16), in this state, the clipping of the field of view V _R in the Z-axis direction is not correct. Therefore, in order to return only the translation distance _Ldist , the translation inverse matrix T _Z ⁻¹ is multiplied as shown in the following equation (21).

<< Transformation matrix at virtual camera position >>
The transformation matrix F at the position of the virtual camera will be described with reference to FIG.
Using the above-described Expression (14), Expression (16), Expression (20), and Expression (21), the projection matrix P by oblique projection can be expressed by the following Expression (22). The addition of terms of a model view matrix M _C to the equation (22), the transformation matrix F at the position of the virtual camera can be expressed by the following equation (23).

As described above, the virtual photographed image generating unit 144 uses the above-described equation (23), so that the three-dimensional shape models Obj1 and Obj1 and Obj2 are projected onto the projection plane P _P (X-axis-Y-axis plane) as shown in FIG. ) Can be generated. Thereafter, the virtual captured image generation unit 144 outputs the generated virtual captured image to the first filter 15.

<Reduction of aliasing noise caused by lens array sampling>
Hereinafter, with reference to FIG. 14, according to a first filter 15, it will be described the reduction of aliasing noise due to sampling at the lens array L _A (see properly Figure 2).
Here, the filter parameter setting means 13, in the virtual photographic image will be described the example of setting the filter application region A _R as shown in FIG. 14. That is, the filter application area A _R is around a pixel corresponding to a pinhole H, the pin hole H at the midpoint C _P of the line segment connecting the pixels corresponding to the other of the pinhole H adjacent 1 This is a region surrounded by 4 (for example, a rhombus). Further, since using the oblique projection, "●" denotes a pixel of, as shown in FIG. 14, according to the arrangement of the element lenses L _P of the lens array L _A, are aligned vertically and horizontally. Further, the arrangement of the element lenses L _P of the pixel spacing and lens array L _A virtual photographic image, integer ratio (e.g., 1: 4) becomes.

The first filter 15 (see FIG. 2), the filter application area A _R from the filter parameter setting unit 13 is inputted, the virtual captured image is inputted from the virtual camera photographing means 14. The first filter 15 performs low-pass filtering in the filter applied in the area A _R in the virtual photographic image. Specifically, the first filter 15, as a low-pass filtering, to calculate an average value of all pixel values existing in the filter application area A _R, the center of the filter the average coverage area A _R The pixel value of the “●” pixel located at. Moreover, the 1st filter 15 may obtain | require a weighted addition value or a median value instead of calculating | requiring an average value as a low-pass filter process. Thereafter, the first filter 15 outputs the virtual photographed image subjected to the low-pass filter processing to the normalized image generation means 16.

Thus, the first filter 15 may be light rays from the three-dimensional object Obj to reduce the aliasing noise (aliasing) caused to be sampled by the lens array L _A. At this time, the pixels of the virtual photographed image are aligned vertically and horizontally, and no geometric distortion has occurred in the virtual photographed image, so the first filter 15 reduces the calculation cost for correcting the geometric distortion. (See FIG. 14).

The filter parameter setting means 13, also with the pixel corresponding to another pinhole H other than the pinhole H, to set the filter application area A _R Likewise. The first filter 15 is naturally subjected to low-pass filter processing for each the filter application area A _R.

  2 receives the virtual captured image from the first filter 15 and assigns the pixel value of the pixel of the virtual captured image to the pixel of the normalized image I ′ by a predetermined coordinate conversion formula. Thus, the normalized image I ′ is generated. Specifically, the normalized image generation means 16 uses the sample point (“●” in FIG. 14) of the virtual photographed image that has been subjected to the low-pass filter processing as the calculation target pixel (in FIG. 7). Assigned to “■”). In this case, the pixel coordinates of the normalized image I ′ are expressed by the following formula (24). Note that the first term on the left side of Equation (24) indicates the position of the pinhole H.

Thereafter, in the three-dimensional image generation device 1, each pixel of the element image G is designated as local coordinates (u, v), and the imaging process by the virtual camera Vc by the virtual camera imaging unit 14 is performed. The pass filter processing and the pixel value assignment processing by the normalized image generation means 16 are executed for all the pixels of the element image G. Therefore, in the stereoscopic image generating apparatus 1, it is only necessary to repeat these processes for the number of pixels of the element image G, and it is not necessary to repeat the process for all the pixels existing in the image as in the conventional case.
In this way, the normalized image generating means 16 can obtain all pixel values of the normalized image I ′. Thereafter, the normalized image generating unit 16 outputs the generated normalized image I ′ to the display image generating unit 17.

  The display image generation unit 17 receives the normalized image I ′ from the normalized image generation unit 16 and generates the display image I from the normalized image I ′. For this reason, as shown in FIG. 2, the display image generation means 17 is provided with the 2nd filter (2nd aliasing noise reduction means) 17a.

<Reduction of aliasing noise caused by sampling on the image display panel>
Hereinafter, the reduction of aliasing noise caused by sampling on the image display panel 21 by the second filter 17a will be described with reference to FIG. 15 (see FIG. 2 as appropriate).
The second filter 17a performs low-pass filter processing on the normalized image I ′ because the normalized image I ′ has M-times pixels in the vertical and horizontal directions of the display image I displayed on the image display panel 21. . Specifically, as shown in FIG. 15, the second filter 17 a includes a pixel value of one pixel of the display image I in a region corresponding to one pixel of the display image I as a low-pass filter process. The average value of all the pixels of the normalized image I ′ is processed. Furthermore, the second filter 17a may obtain a weighted addition value or a median value as a low-pass filter process instead of obtaining an average value.

  That is, since the second filter 17a can perform coordinate conversion by the above-described equation (4), the display image I is generated by performing the low-pass filter processing represented by the following equation (25). Then, the second filter 17a outputs the generated display image I to the stereoscopic display device 2 (see FIG. 1).

In Expression (25), I ′ (i ′ + s, j ′ + t) represents the pixel value of the pixel (i ′ + s, j ′ + t) of the normalized image I ′, and I (i, j) Indicates the pixel value of the pixel (i, j) of the display image I. D is the number of I (i ′ + s, j ′ + t) that satisfies the condition of Σ, and is the number of pixels that contributed to the sum. Moreover, s and t is an integer from 0 to half the number of pixels between the element lenses L _P.

[Operation of stereoscopic image generation device]
Hereinafter, the overall operation of the stereoscopic image generating apparatus 1 of FIG. 2 will be described with reference to FIG. 16 (see FIG. 2 as appropriate).
In the stereoscopic image generating device 1, the parameters (F, R _H , R _V , P) of the stereoscopic display device 2 are input by the stereoscopic display device modeling means 11 (step S1).

In addition, the stereoscopic image generating apparatus 1 sets the normalized image I ′ by using the normalized image setting unit 12 (Step S2). That is, the stereoscopic image generating apparatus 1 uses the normalized image setting unit 12 to calculate the pixel size P _H and the normalized image I in the horizontal direction of the normalized image I ′ using the above-described equations (1) and (2). The pixel size P _V in the vertical direction is calculated.

In addition, the stereoscopic image generating apparatus 1 sets the filter parameter by the filter parameter setting unit 13 (step S3). That is, the stereoscopic image generating apparatus 1, the filter parameter setting means 13, for example, sets a rhombic filter applying area A _R on the virtual photographic image.

  In addition, the stereoscopic image generating apparatus 1 uses the virtual camera photographing unit 14 to photograph with the virtual camera Vc to generate a virtual photographed image (step S4). Details of the operation of the virtual camera photographing means 14 will be described later.

In addition, the stereoscopic image generating device 1 performs low-pass filter processing on the virtual photographed image using the first filter 15 (step S5). That is, the stereoscopic image generating apparatus 1, the first filter 15 calculates the virtual photographic image, the average value of all pixel values existing in the filter application area A _R, the weighted addition value or the median value, the the pixel values of all pixels existing values in filter application region a _R.

  In addition, the stereoscopic image generating apparatus 1 normalizes the normalized image generating unit 16 by assigning the pixel value of the pixel of the virtual captured image to the pixel of the normalized image I ′ using the above-described equation (24). An image I ′ is generated (step S6).

  Thereafter, the stereoscopic image generating apparatus 1 uses the normalized image generating unit 16 to perform the imaging process with the virtual camera by the virtual camera imaging unit 14 and the low-pass filter using the first filter 15 for all the pixels of the element image G. It is determined whether the process and the pixel value assignment process by the normalized image generation means 16 have been executed (step S7).

Here, when the process is not executed on all the pixels of the element image G (No in step S7), the stereoscopic image generating apparatus 1 returns to the process of step S4.
On the other hand, when the process is executed on all the pixels of the element image G (Yes in step S7), the stereoscopic image generating apparatus 1 proceeds to the process of step S8.

  In addition, the stereoscopic image generating apparatus 1 performs low-pass filter processing on the normalized image I ′ using the second filter 17a (step S8). That is, the stereoscopic image generating apparatus 1 uses the second filter 17a as a low-pass filter process to convert the pixel value of the pixel of the display image I to the average value of all the pixels of the normalized image I ′ corresponding to the pixel. , A weighted addition value or a median value is processed.

Hereinafter, the operation of the virtual camera photographing means 14 of FIG. 5 will be described with reference to FIG. 17 (see FIG. 5 as appropriate).
The virtual camera photographing unit 14 calculates the pixel position of the normalized image I ′ using the above-described Expression (3) and Expression (4) by the normalized image pixel position calculation unit 141 (Step S41).

  Further, the virtual camera photographing means 14 calculates the position of the pinhole H by using the above-described formula (5) by the pinhole position calculating means 142 (step S42).

Further, the virtual camera photographing means 14 uses the virtual camera position calculation means 143 to generate a unit vector from the pixel I ′ ₀ , ₀ (u, v) to the position of the pinhole H (0, 0) in the normalized image I ′. (See equation (6)). Then, the virtual camera photographing unit 14 calculates the position of the virtual camera Vc by using the above-described equation (7) by the virtual camera position calculating unit 143, and calculates the reverse direction of the unit vector as the photographing direction of the virtual camera Vc. (Step S43).

Further, the virtual camera photographing unit 14, by the virtual-photographed-image generating unit 144, by the transformation matrix F at the position of the virtual camera (see equation (23)), by oblique projection onto the projection plane P _P display target, the display A virtual captured image obtained by capturing the target with the virtual camera Vc is generated (step S44).

As described above, the stereoscopic image generating apparatus 1 according to an embodiment of the present invention, the normalized image setting unit 12, the normalized image I pixel size becomes integral submultiple respect to the spacing element lenses L _P ' Prepare. Then, the stereoscopic image generating apparatus 1, by the virtual camera photographing means 14 performs the oblique projection, such as the center of the element lenses L _P corresponds to a pixel of the virtual photographic image, the pixel values of the pixels of the virtual photographic image, was prepared Assigned to each pixel of the normalized image I ′. Thus, the stereoscopic image generating apparatus 1, even if the distance between the element lenses L _P does not satisfy the condition that an integral multiple of the pixel size of the elemental image G, can generate a display image I by isometric. Further, since the stereoscopic image generating apparatus 1 does not generate geometric distortion unlike an orthogonal projection, the calculation cost for correcting the geometric distortion can be reduced. Specifically, a stereoscopic image generating apparatus 1, as compared with the conventional technique of performing the projection conversion for all the pixels present in the image, it is possible to suppress the computational cost on the number below 1 / element lens L _P.

Further, the stereoscopic image generating apparatus 1, the first filter 15, so reducing the aliasing noise due to sampling at the lens array L _A, it is possible to display image I high quality. At this time, the stereoscopic image generating apparatus 1 can set the filter application region by the filter parameter setting unit 13 without human intervention, and enables accurate low-pass filter processing. Then, the stereoscopic image generating device 1 reduces the aliasing noise caused by sampling on the image display panel 21 by the second filter 17a, so that the quality of the stereoscopic image can be improved.

  Here, even in the conventional technique for performing projective transformation (for example, Japanese Patent Application Laid-Open No. 2009-175866), the aliasing noise reduction filter is used. Since the calculation is performed, the calculation amount is increased. However, since the first filter 15 and the second filter 17a can be applied to oblique projection in the stereoscopic image generating apparatus 1, the amount of calculation required for reducing the aliasing noise is less than that of the conventional aliasing noise reduction filter. can do.

In the embodiment of the present invention, the stereoscopic image generating apparatus 1 has been described as including the first filter 15 and the second filter 17a. However, only one or both of them may be included. Good.
Here, the case of providing only the first filter 15, the stereoscopic image generating apparatus 1, since reducing the aliasing noise due to sampling at the lens array L _A, to obtain a display image I of practically sufficient quality it can. As described above, when only the first filter 15 is provided (that is, when the second filter 17 a is not provided), the stereoscopic image generating apparatus 1 displays the center pixel of the normalized image I ′ by the display image generating unit 17. The display image I may be generated by using the pixels of the image I.
Further, when only the second filter 17a is provided, the stereoscopic image generating device 1 can reduce the aliasing noise caused by the sampling on the image display panel 21, and thus can obtain a display image I having a practically sufficient quality. .

  In the embodiment of the present invention, the example applied to the integral method has been described, but the present invention is not limited to this. For example, by setting the vertical direction of the display image I to one pixel, the present invention can also be applied to the lenticular method.

  In the present invention, when the ratio between the pixel size of the element image G in the normalized image I ′ and the pixel size of the image display panel 21 is not an integer, the second filter 17a performs an interpolation process to obtain a sampling position. The pixel value of the display image I according to the above may be calculated.

  In the embodiment of the present invention, the stereoscopic image generating apparatus according to the present invention has been described as an independent apparatus. However, in the present invention, a general computer is realized by a stereoscopic image generating program that functions as each of the above-described units. You can also. This stereoscopic image generation program may be distributed via a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

[Application to dynamic 3D shape model]
Hereinafter, a method of generating a stereoscopic image (integral stereoscopic video) of a dynamic three-dimensional shape model using the stereoscopic image generating device 1 will be described with reference to FIG.

As shown in FIG. 18, the stereoscopic video generation system S includes a three-dimensional modeling device 3 and a stereoscopic image generation device 1.
The three-dimensional modeling apparatus 3 arranges a plurality of cameras so as to surround a display target, and generates a dynamic three-dimensional shape model from the plurality of camera images. As the three-dimensional modeling apparatus 3, for example, a three-dimensional modeling apparatus disclosed in Japanese Patent No. 4014140 can be used.

  Accordingly, the stereoscopic video generation system S generates a dynamic three-dimensional shape model for each video frame from a plurality of camera images in the three-dimensional modeling device 3, and the stereoscopic image generation device 1 generates the dynamic three-dimensional shape model from the dynamic three-dimensional shape model. Generate an integral 3D image. Thus, the stereoscopic video generation system S can generate a moving image (integral stereoscopic video) by generating an integral stereoscopic image for each video frame.

DESCRIPTION OF SYMBOLS 1 3D image production | generation apparatus 11 3D display apparatus modeling means (parameter input means)
12 Normalized image setting means (normalized image pixel size calculating means)
13 Filter parameter setting means (filter application area setting means)
14 Virtual camera photographing means 141 Normalized image pixel position calculating means 142 Pinhole position calculating means 143 Virtual camera position calculating means 144 Virtual photographed image generating means 15 First filter (first aliasing noise reducing means)
16 Normalized image generating means 17 Display image generating means 17a Second filter (second aliasing noise reducing means)
2 stereoscopic display device 21 image display panel (image display means)
L _A lens array _{L P} element lenses

Claims (5)

A 3D model as a display object to be displayed on a stereoscopic display device including a lens array in which element lenses are arranged two-dimensionally and an image display means is photographed by a virtual camera which is a virtual photographing camera, and the stereoscopic display device A stereoscopic image generating device for generating a display image for use,
Parameter input means for inputting a focal length of the element lens, an interval between the element lenses, and a pixel interval of the image display means;
The number of pixels between the element lenses is calculated based on a value obtained by dividing the interval between the element lenses by the pixel interval of the image display means and multiplying by a preset constant, and the interval between the element lenses is calculated as the element lens interval. A normalized image pixel size calculating unit that calculates the pixel size of the normalized image by a value that is a fraction of an integer with respect to the interval between the element lenses by dividing by the number of pixels in between,
Normalized image pixel position calculating means for calculating the pixel coordinates of the normalized image by multiplying the pixel coordinates of the display image by the pixel interval of the image display means and dividing by the pixel size of the normalized image;
The position of the virtual camera is calculated based on a value obtained by multiplying the distance set in advance so as to indicate the position from the lens array to the position of the virtual camera and the pixel size of the normalized image and dividing by the focal length of the element lens. Virtual camera position calculating means for calculating;
A virtual photographed image obtained by photographing the three-dimensional shape model with the virtual camera is obtained by obliquely projecting the three-dimensional shape model onto a projection surface by a transformation matrix at the position of the virtual camera calculated by the virtual camera position calculating means. Virtual captured image generation means for generating;
A normalized image generating means for generating the normalized image by assigning a pixel value of the pixel of the virtual photographed image to a pixel of the normalized image by a predetermined coordinate transformation formula;
Display image generating means for generating the display image from the normalized image generated by the normalized image generating means;
A stereoscopic image generating device comprising: The virtual camera position calculation means further calculates a direction from the position of the virtual camera to the center position of the lens array as a shooting direction of the virtual camera,
It said virtual captured image generating means, wherein is defined in the model view matrix M _O for the following formula in the first viewpoint virtual camera toward the center position of the lens array (9) located on the perpendicular of said lens array the model view matrix M _C in a second viewpoint virtual camera position calculating means the virtual camera at the position of the virtual camera calculated faces the shooting direction is defined by the following equation (10), from the first viewpoint the transformation matrix T _C to the second viewpoint is defined by the equation (11) below, orthogonal projection conversion matrix ortho is defined by the following equation (12), translation of the lens array in the projection coordinate system distance L _dist is defined with the following equation (15), translation matrix T _Z of the lens array in the projection coordinate system is defined by the following equation (16), a third axial direction of the projection coordinate system To the displacement of The first and second axial variation _S X which, _{S Y} is defined by the following equation (19), oblique projection transformation matrix Oblique is defined by the following equation (20), within the projection coordinate system When the translation inverse matrix T _Z ^{−1 of the} lens array is defined by the following equation (21), and the projection matrix P is defined by the following equation (22), it is defined by the following equation (23): The stereoscopic image generating apparatus according to claim 1, wherein the three-dimensional shape model is obliquely projected by a transformation matrix F at the position of the virtual camera.

Here, o _{11 to} o ₃₄ are elements representing rotation and translation, c _{11 to} c ₃₄ are elements representing rotation and translation, and l is the left end position of the visual field range of the virtual camera, r is the right end position of the visual field range, t is the upper end position of the visual field range, b is the lower end position of the visual field range, f is the far end position of the visual field range, and n is the visual field range. The length is the distance from the virtual camera to the lens array, X _C is the position of the second viewpoint in the first axis direction, and Y _C is the second axis direction. The position of the second viewpoint, and Z _C is the position of the second viewpoint in the third axis direction. A low-pass filter process is performed on the virtual captured image generated by the virtual captured image generation unit for each filter application region surrounded by a midpoint of a line segment connecting pixels corresponding to the center position of the element lens. 1 aliasing noise reduction means,
The stereoscopic image generating apparatus according to claim 1, further comprising: The normalized image pixel size calculation means is preset with an even number exceeding 2 as the constant,
The display image generation means includes second aliasing noise reduction means for applying a low-pass filter process to the normalized image;
The stereoscopic image generating apparatus according to any one of claims 1 to 3, further comprising: A 3D model as a display object to be displayed on a stereoscopic display device including a lens array in which element lenses are arranged two-dimensionally and an image display means is photographed by a virtual camera which is a virtual photographing camera, and the stereoscopic display device Computer to generate display images for
Parameter input means for inputting a focal length of the element lens, an interval between the element lenses, and a pixel interval of the image display means,
The number of pixels between the element lenses is calculated based on a value obtained by dividing the interval between the element lenses by the pixel interval of the image display means and multiplying by a preset constant, and the interval between the element lenses is calculated as the element lens interval. A normalized image pixel size calculating unit that calculates the pixel size of the normalized image by a value that is a fraction of an integer with respect to the interval between the element lenses by dividing by the number of pixels in between,
Normalized image pixel position calculating means for calculating the pixel coordinates of the normalized image by multiplying the pixel coordinates of the display image by the pixel interval of the image display means and dividing by the pixel size of the normalized image;
The position of the virtual camera is calculated based on a value obtained by multiplying the distance set in advance so as to indicate the position from the lens array to the position of the virtual camera and the pixel size of the normalized image and dividing by the focal length of the element lens. Virtual camera position calculating means for calculating,
A virtual photographed image obtained by photographing the three-dimensional shape model with the virtual camera is obtained by obliquely projecting the three-dimensional shape model onto a projection surface by a transformation matrix at the position of the virtual camera calculated by the virtual camera position calculating means. Virtual captured image generation means for generating,
A normalized image generating means for generating the normalized image by assigning a pixel value of the pixel of the virtual photographed image to a pixel of the normalized image by a predetermined coordinate transformation formula;
Display image generating means for generating the display image from the normalized image generated by the normalized image generating means;
A three-dimensional image generation program characterized by functioning as

Images