JP6788472B2 - Element image generator and its program - Google Patents

Description

The present invention relates to an element image generator and a program thereof for generating an integral stereoscopic element image from a three-dimensional model.

Conventionally, the integral stereoscopic method is known as a method for displaying a stereoscopic image.
In this integral stereoscopic method, as shown in FIG. 12, the light emitted from the subject O is imaged by an imaging device via a lens array La composed of a plurality of element lenses Lp arranged two-dimensionally. At this time, the element image G is imaged at the lens spacing of the element lens Lp on the imaging surface E separated by the focal length of the element lens Lp.
Then, as shown in FIG. 13, the integral stereoscopic method displays the element image G captured in FIG. 12 on the display surface D of the display device (display) via the lens array La having the same specifications as those at the time of imaging. At this time, light rays similar to those in the captured subject space are reproduced, and the observer M can visually recognize the subject O (see FIG. 12) as a stereoscopic image T.

In this integral stereoscopic method, a method of generating an element image from a three-dimensional model in a virtual space has been proposed in addition to capturing a real subject and generating an element image.
For example, a method of generating an element image from a three-dimensional model by ray tracing (hereinafter, ray tracing method) has been proposed (see Patent Document 1). As shown in FIG. 14, this ray tracing method is a method of obtaining the pixel value of the element image G of the display surface D one pixel at a time by ray tracing.

As shown in FIG. 14, in the ray tracing method, first, the pixels (In _{, m} (u)) of the local coordinates (here, (u, v)) included in the element image G (here, In _{, m} ) are used. , V)), the unit vector i facing the lens center H of the corresponding element lens Lp separated by the focal length F from the positions of the pixels In _{, m} (u, v) is obtained.
Then, in the ray tracing method, the virtual camera V is arranged at a position separated by the viewing distance L from the lens array La surface on the extension line of the unit vector i, and the virtual camera V is placed on the pixels In _{, m} (u, v). _Let the value of the pixel (black circle in FIG. 14) of the surface A of the three-dimensional model taken at the center taken in the direction be the pixel value of the pixel In _{, m} (u, v).
Then, the ray tracing method similarly obtains the unit vector i'for the other element image G (here, In _{', m'} ), and arranges the virtual camera V on the extension line of the unit vector i'. _Let the value of the pixel (black circle in FIG. 14) of the surface B of the three-dimensional model photographed in ′ be the pixel value of the pixels In _{′, m ′} (u, v).

As described above, since the ray tracing method calculates the pixel value for each pixel of all the element image G one pixel at a time, the element image can be generated with high accuracy, but the amount of calculation is large and the element image can be generated. It takes time.
Therefore, as another method, a method of generating an element image from a three-dimensional model by oblique projection (hereinafter, oblique projection method) has been proposed (see Patent Document 2 and Non-Patent Document 1). As shown in FIG. 15, this oblique projection method is a method of obtaining the pixel value of the element image G of the display surface D by the oblique projection transformation of the three-dimensional model.

As shown in FIG. 15, in the oblique projection method, first, the local coordinates (here, (u, v)) included in the element image G (here, I ₀ , ₀ ) corresponding to the center of the lens array La are used. From the pixel (I _0,0 (u, v)), the unit vector i facing the lens center H of the corresponding element lens Lp separated by the focal length F is obtained.
The oblique projection method, as an extension of the unit vectors i, the lens array is viewed from La plane distance L apart virtual camera position (isometric virtual camera) arranged V _C, inclined with respect to the lens array La obliquely projecting a three-dimensional model along the shooting direction of the virtual camera V _C (including orthogonal projection).

Here, the oblique projection method is a virtual method in which the shooting direction arranged at a position separated by the viewing distance L on the perpendicular line of the lens array La surface from the center of the lens array La is arranged facing the lens array La. and camera V _O, the relationship between the virtual camera V _C the photographing direction inclined to obtain the transformation matrix obliquely projected on the display surface D to 3-D model, determined collectively the pixel values of pixels that can be regarded as parallel light ray group .. That is, oblique projection method, the local coordinates (u, v) in each element image G is for the same pixel, a photographing direction from the virtual camera V _C are the same (parallel), the pixels by the same transformation matrix Find the value.
As a result, the oblique projection method can generate an element image at a higher speed than the ray tracing method in which the pixel values of all the element images G are calculated one by one.

JP-A-2009-175866 Japanese Unexamined Patent Publication No. 2012-84105

Iwatate, Katayama, "Method of generating an integral stereoscopic image by oblique projection", Technical Report of the Institute of Image Information and Television Engineers, vol. 34, No. 43, pp. 17-20, October 2010

As described above, the oblique projection method can generate an element image at a higher speed than the ray tracing method.
This oblique projection method generates an element image by assuming that it is projected by a group of parallel rays at the same local pixel position for each element image. Therefore, when the observer visually recognizes the element image generated by this method through the lens array, the observer strictly observes the distorted light rays.

For example, when an element image is generated for a stereoscopic display device having a small display surface of about 23 to 24 inches, as shown in FIG. 16A, it is projected to the observer's viewpoint position Vp by oblique projection. The pixel position of the display surface D that projects light onto the surface U (◯ in FIG. 16) and the pixel position when the observer actually observes the display surface D through the lens array La at the viewpoint position Vp (FIG. 16). There is little error between middle and ●).

However, for example, when an element image is generated for a stereoscopic display device having a large display surface of more than 50 inches, as shown in FIG. 16B, a display in which light is projected onto the projection surface U by oblique projection. Between the pixel position of the surface D (◯ in FIG. 16) and the pixel position (● in FIG. 16) when the observer actually observes the display surface D through the lens array La at the viewpoint position Vp. The error of is large.
In this way, the method using oblique projection has a problem that the larger the display surface, the larger the error in the peripheral portion, and the observer can visually recognize a stereoscopic image with distortion with respect to the actual 3D model. There is.

The present invention has been made in view of such a problem, and it is possible to generate an element image with a small error from three-dimensional model data by using oblique projection while suppressing a decrease in the generation speed of the element image. An object of the present invention is to provide a possible element image generator and a program thereof.

In order to solve the above problems, the element image generation device according to the present invention generates an element image to be displayed on an integral three-dimensional stereoscopic display device provided with a lens array in which element lenses are arranged two-dimensionally from three-dimensional model data. The element image generation device is configured to include a division area setting means, an element image generation means for each division area, and an element image integration means.

In such a configuration, the element image generation device divides the range in which the element image is generated by performing oblique projection by setting the division area with respect to the display surface of the stereoscopic display device by the division area setting means. The divided area may be equally divided or unequally divided.

Then, the element image generation device virtually arranges a virtual camera at a predetermined viewing distance by the element image generation means for each division area, and from the virtual camera to the element image in the center of the division area for each division area. With the direction of the corresponding element lens toward the lens center as the shooting direction, the three-dimensional model data is obliquely projected onto the projection plane of the predetermined projection coordinate system, and the pixel value of the element image in the divided region is calculated. As a result, the error in the peripheral portion can be reduced.

Then, the element image generation device displays the element image group displayed on the display surface of the stereoscopic display device by connecting the element image groups for each division area generated by the element image generation means for each division area by the element image integration means. To generate.

In this way, the element image generator generates an element image from the three-dimensional model data by italic projection for each divided region, as compared with the case where the element image of the entire display surface is collectively generated by italic projection. The error around the generated image area can be reduced.
The element image generation device can be operated by an element image generation program for operating the computer as each of the above-mentioned means.

The present invention has the following excellent effects.
According to the present invention, an element image can be generated by oblique projection transformation for each region of the display surface of the stereoscopic display device. As a result, the present invention can generate an element image in a state where the lens center is shifted for each divided region, as compared with the conventional case where an element image is generated by oblique projection at one lens center. It is possible to suppress an error in the pixel position of the element image in the periphery. Further, according to the present invention, by using the oblique projection method, an element image can be generated at a higher speed than the conventional ray tracing method.

It is a block diagram which shows the structure of the stereoscopic image display system which includes the element image generation apparatus which concerns on embodiment of this invention. It is a figure which shows the projection coordinate system, (a) is an XY plan view, (b) is a YZ plan view. It is a block block diagram which shows the structure of the element image generation apparatus which concerns on embodiment of this invention. It is a figure which shows the example of the division of a display surface, (a) is a figure which shows the state before division, (b) is a figure which shows the state after division. It is explanatory drawing for demonstrating the arrangement position of a reference virtual camera and the shift amount of a lens center. It is explanatory drawing for demonstrating the unit vector which passes from the pixel in the element image through the lens center. It is explanatory drawing for demonstrating an oblique projection method, (a) is a figure which shows normal projection, (b) is a figure which shows oblique projection. It is explanatory drawing for demonstrating low-pass filter processing after oblique projection. It is explanatory drawing for demonstrating operation of the element image generation apparatus which concerns on embodiment of this invention. It is a figure which shows another example of division of a display surface. It is a figure which shows the example which divides the display surface including the overlap area. It is explanatory drawing for demonstrating the conventional integral type imaging system. It is explanatory drawing for demonstrating the display system of the conventional integral system. It is explanatory drawing for demonstrating the element image generation method by the conventional ray tracing method. It is explanatory drawing for demonstrating the element image generation method by the conventional oblique projection method. It is explanatory drawing for demonstrating the problem which occurs by the size of the display surface of the conventional oblique projection method, and (b) shows the state which the display surface is larger than (a).

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Structure of stereoscopic image display system]
First, the configuration of the stereoscopic image display system S including the element image generation device 1 according to the embodiment of the present invention will be described with reference to FIG.

The element image generation device 1 is a stereoscopic image display device 2 that generates a plurality of element images (element image groups) that allow the observer M to visually recognize the stereoscopic image T by an integral stereoscopic method. This element image generation device 1 produces a stereoscopic image T from three-dimensional model data such as CG in a projected coordinate system (X, Y, Z) with the screen center of the stereoscopic image display device 2 as the origin as shown in FIG. Generates a group of element images to be displayed.

The element image generation device 1 generates an element image group from the three-dimensional model data by oblique projection transformation. At this time, the element image generation device 1 virtually divides the display surface into a plurality of regions, and generates an element image group by oblique projection transformation for each region.
As a result, the element image generation device 1 can generate an element image in which the deviation between the pixels observed by the observer M and the pixels that emit parallel light at the position of the observer M by the oblique projection conversion is suppressed. The element image generation device 1 will be described in detail later.

The element image generation device 1 outputs the generated element image (element image group) to the stereoscopic image display device 2. Of course, the element image generated by the element image generation device 1 may be recorded on a recording medium such as a hard disk or a DVD, and then the stereoscopic image display device 2 may read the element image.

The stereoscopic image display device 2 is a general display device that displays a stereoscopic image T in an integral stereoscopic system. That is, as shown in FIG. 13, the stereoscopic image display device 2 includes a lens array La in which element lenses Lp are two-dimensionally arranged, and observes the element image G displayed on the display surface D via the lens array La. The person M is made to visually recognize as a stereoscopic image T.

[Configuration of element image generator]
Next, the configuration of the element image generation device 1 according to the embodiment of the present invention will be described with reference to FIG. 3 (see FIG. 1 as appropriate).
As shown in FIG. 3, the element image generation device 1 includes a division area setting means 10, an element image generation means 20 for each division area, and an element image integration means 30.

The divided area setting means 10 sets a divided area on the display surface D of the stereoscopic image display device 2 based on the parameters of the stereoscopic image display device 2 to be displayed and the screen division information.

The parameters are the number of vertical and horizontal pixels and the pixel spacing of the display surface of the stereoscopic image display device 2, the number of vertical and horizontal lenses of the lens array, the lens spacing, the focal length, and the like. Further, the parameter includes a distance (viewing distance) from the stereoscopic image display device 2 for the observer to visually recognize the stereoscopic image.

The screen division information is information indicating how the display surface D is divided. For example, the number of vertical divisions and the number of horizontal divisions, the position and size of individual division areas, and the like. The display surface D may be divided equally or unequally. However, since the observer observes with reference to the center of the display surface D, it is preferable that there is no division boundary at the center of the screen. For example, if the display surface D is divided into equal parts, the number of vertical divisions and the number of horizontal divisions are preferably odd numbers.

As for the number of divisions of the display surface D, the larger the number, the more the error of the element image can be suppressed, but the larger the amount of calculation for performing oblique projection. Further, if the number of divisions is reduced, the amount of calculation can be suppressed, but the error becomes large. Therefore, the number of divisions of the display surface D may be determined in advance by the creator of the element image within a range in which an error can be tolerated.

Here, the description is simplified, and the number of vertical divisions and the number of horizontal divisions are set to “3”, respectively. That is, when the display surface D has the number of vertical pixels High and the number of horizontal pixels Wide as shown in FIG. 4 (a), the division area setting means 10 has the number of vertical pixels High as shown in FIG. 4 (b). The division area R is set with 3/3 and the number of horizontal pixels Wid / 3.
In FIG. 4B, when each division region R is set to n = 1, ..., N, m = 1, ..., M from the lower left to the upper right, the display surface D is the division region R (n, m). It shows the state divided into.
The division area setting means 10 outputs the division area information (for example, the upper left coordinates of the division area and the number of vertical and horizontal pixels) to the division area-specific element image generation means 20 together with the input parameters.

The element image generation means 20 for each division area generates an element image group for each division area set by the division area setting means 10.
The divided regional element image generating means 20, as shown in FIG. 5, includes for each divided region R obtained by dividing the display screen D, based on the virtual camera (a reference virtual camera) V _O, an oblique projection (orthogonal projection ) To generate an element image group. This virtual camera _VO is arranged so as to be separated from the lens center of the lens array La by a predetermined distance (viewing distance) L in the vertical direction of the lens array La surface. Here, the lens center H _0,0 of the lens array La may be described as a match with the center of the divided regions lens center of R (2, 2) shown in FIG. 4 (b) [the pinhole center position]).
As shown in FIG. 3, the element image generation means 20 for each division region includes a lens center shift amount calculation means 21 and an oblique projection calculation means 22.

The lens center shift amount calculation means 21 calculates the shift amount from the lens center of the lens array La to the lens center in the divided region R for each divided region R in which the display surface D is divided.

Here, the shift amount for each division region R calculated by the lens center shift amount calculation means 21 will be described with reference to FIG. 5 (see FIG. 4B as appropriate).
If the coordinates on the XY plane of the lens center of the lens array La _(X R, _{Y R)} to the lens center of each divided region R, in response to the divided sections of the divided regions _{(X R + H shift, Y} R + V shift ) Is the shifted position.
_{Here, H Shift} is the shift amount in the horizontal direction (X direction) from the center of the lens _{H 0,0} of the lens array La. _{Also, V Shift} is the shift amount of the vertical method (Y-direction) from the center of the lens _{H 0,0} of the lens array La.

For example, the shift amount of the divided region R (1,1) is H _shift = −Width / 3, V _shift = −Height / 3, and the shift amount of the divided region R (1,1) is H _shift = −Wids / 3. , V _shift = 0, etc. Here, the shift amount of the divided region R (2, 2) is H _shift = 0 and V _shift = 0.

That is, the lens center shift amount calculation means 21 extends from the lens center of the lens array La to the lens center in the division region R based on the number of vertical and horizontal pixels of the display surface D and the screen division information input as parameters. Calculate the shift amount.
The lens center shift amount calculation means 21 outputs the calculated shift amount for each division region R to the oblique projection calculation means 22.

The oblique projection calculation means 22 calculates the pixel value of the coordinates in the element image by a transformation matrix that obliquely projects the three-dimensional model data onto the projection plane (display plane) for each division region R. Here, the oblique projection calculation means 22 obliquely projects the three-dimensional model data onto the projection plane via the individual lens centers of the divided regions R shifted by the shift amount calculated by the lens center shift amount calculation means 21.

Hereinafter, the processing of the oblique projection calculation means 22 will be specifically described with reference to FIG. In FIG. 6, it is assumed that the division area is divided by the division boundary Div.
First, for simplification of the description, the division region R (2, 2) having the lens center H ₀ , ₀ of the lens array La as the lens center of the division region will be described.

As shown in FIG. 6, the pixel position in the three-dimensional space in the element image G ₀ , ₀ corresponding to the lens center (lens center of the divided region R (2, 2)) H ₀ , ₀ of the lens array La is I. _{Let it be 0,0} (u, v). Here, (u, v) are local coordinates with the center of the element image G as the origin (0,0). That is, next to the pixel spacing _{P H} of the display surface D which is input in the parameter, the vertical pixel spacing _{P V} of the display surface, the focal length was set to _{F, I 0,0 (u, v} ) = (X _{P, Y P, -F)} = a _{(uP H, vP V, -F} ). Note that ( _XP , Y _P ) is the XY coordinates of the local coordinates (u, v) of the element image G ₀ , ₀ .

Here, if the element lens Lp has a delta arrangement as shown in FIG. 13, the horizontal pixels of the element image obtained by dividing the number of pixels on the side of the display surface D input by the parameter by the number of lenses on the side of the lens array La. when the number G _H, the number of vertical pixels of the display surface D of the vertical by dividing the number of pixels in the vertical number of lenses of the lens array La 2 / √3 times the element image was G _V, -G H / ₂ < uP _H ≤ _GH / 2, -G _v / 2 <uP _v ≤ G _v / 2. Incidentally, the element lens Lp is if square arrangement, G _V is a vertical value obtained by dividing the number of pixels in the vertical number of lenses of the lens array La of the display surface D.

At this time, the three-dimensional coordinates _{H (0,0)} of the lens center _{H 0,0} = become (X R, Y R, 0 ) = (0,0,0). Note that (X _R , Y _R ) is the XY coordinates of the lens center H ₀ , ₀ .
The position of the virtual camera _{V C (X C, Y C} , Z C) , when the viewing distance is L, can be expressed by the following equation (1).

From the pixel position of the pixel in the center of the element image _{G 0,0} of the divided regions _{R I 0,0 (u, v)} , the position of the virtual camera _{V C} present on an extension line passing through the lens center _{H 0,0} (X _{C, Y} C, the unit vector _i to _{Z C)} (i _x, i y, _{i z)} can be expressed by the following equation (2). In the equation (2), | a | indicates the length of the vector a.

The isometric calculating means 22, a virtual camera (a reference virtual camera) V O _(see FIG. 5), the relationship between the virtual camera V _C tilted shooting direction, oblique projection on the display surface D to 3-D model data The transformation matrix to be used is obtained, and the pixel values of the pixels that can be regarded as parallel light rays are collectively obtained.
As a result, the element image group of the divided region R (2, 2) is generated.

Next, the division area R other than the division area R (2, 2) will be described. Hereinafter, the lens center of the divided region R other than the divided region R (2, 2) _{H '0,0,} lens center _H' element image corresponding to _{0,0 G 'represents 0,0,} elemental image G' pixel positions in a three-dimensional space in _{0,0 I '0,0 (u, v} ) to.
Here, isometric calculating means 22, by the shift amount calculated by the lens center shift amount calculation means 21, by shifting the lens center, the pixel position _{I '0,0 (u, v)} , the virtual camera V _C Find the unit vector to the position (X _C , Y _C , Z _C ) of.

At this time, the lens center _{H '3-dimensional} coordinates of _{0,0 H' (0,0) = (} X R + H shift, Y R + V shift, 0) = a _{(H shift, V shift, 0} ). Note that (H _shift , V _shift ) is the shift amount calculated by the lens center shift amount calculation means 21. Further, (X _R , Y _R ) is the XY coordinates of the lens center H ₀ , ₀ of the lens array La.
Also, next to the pixel spacing _{P H} of the display surface D which is input in the parameter, the vertical pixel interval _{P V} of the display surface D, the focal length was set to _{F, I '0,0 (u,} v) = _{(X P + H shift, Y} P + V shift, -F) = a _{(uP H + H shift, vP} V + V shift, -F). Note that ( _XP , Y _P ) is the XY coordinates of the local coordinates (u, v) of the element image G ₀ , ₀ .

Thus, the _'pixel position of the pixel in _{0,0 I'} element image _G 0,0 (u, v), the position of the virtual camera _{V C} present on an extension line passing through the lens center _{H '0,0 (X C} , Y _C , Z _C ) The unit vector i'(i _x , i _y , _iz )'can be expressed by the following equation (3).

That is, isometric calculating means 22, the coordinates (u, v) of the elemental images each, arranged virtual camera _{V C} at the position indicated by the equation (1), the divided region R (2, 2), the The direction represented by the inverse vector of the unit vector of the equation (2) and the direction represented by the inverse vector of the unit vector of the equation (3) except for the division regions R (2, 2) are the virtual camera V, respectively. _As the shooting direction of _C , oblique projection conversion is performed to obtain an element image group.

In this way, the oblique projection calculation means 22 performs the oblique projection conversion by changing the shooting direction in consideration of the shift amount for each division region R. As the method using oblique projection, the conventional method described with reference to FIG. 15 can be used.
The oblique projection calculation means 22 outputs the element image group for each divided region to the element image integrating means 30 together with the divided region information. The divided area information is, for example, the upper left coordinate position, the number of horizontal pixels, and the number of vertical pixels for each divided area.

Here, with reference to FIG. 7, a transformation matrix based on the conventional method used by the itlique projection calculation means 22 will be briefly described.
Method using oblique projection, as shown in FIG. 7 (a), the reference projection coordinate system (X, Y, Z) in the positive against shooting by the viewpoint position of the virtual camera V _O to the lens array La a (orthogonal projection) as, as shown in FIG. 7 (b), when taken from the virtual camera V _C, it is a technique for projecting a field of view V _R of the three-dimensional model data to the projection surface. Incidentally, in FIG. 7 (b), the position of the virtual camera V _C is one in which the position of the virtual camera V _{C (see} FIG. 6) is moved parallel to the XY plane when shifting the lens center.

Isometric calculating means 22, a transformation matrix obliquely projected onto the projection surface a three-dimensional model data, using the transformation matrix M _F as shown in formula (4).

Here, M _C is a model view matrix for transforming the object coordinate system of the three-dimensional model data to the viewpoint position of the virtual camera V _C to the camera coordinate system with the origin. The model view matrix M _C, as shown in the following equation (5), a 4 × 4 matrix comprising a rotational component and the translation component from the elements c ₁₁ represented by C _34.

Incidentally, the model view matrix _{M C} is projected coordinate system (X, Y, Z) position of the virtual camera _{V C} at the origin of the projected coordinate system (gazing point of the virtual camera _{V C),} the upward direction (Y axis) parameters As a result, it can be obtained by using the gluLookAt function of OpenGL (registered trademark).
Also, P of the formula (4), the field of view V _R normalizing a projection matrix for transforming the points in space defined by the camera coordinate system in the projection coordinate system. This projection matrix P can be expressed by the following equation (6).

Here, T _C is the coordinate transformation matrix from the viewpoint position of the virtual camera V _O to the viewpoint position of the virtual camera V _C. As for formula (5), the model view matrix M _O for converting the viewpoint position of the virtual camera V _O the object coordinate system to the camera coordinate system with the origin, as shown in the following equation (7), each element o when a 4 × 4 matrix comprising a rotational component and the translation component represented by o ₃₄ to _11, the coordinate transformation matrix T _C can be expressed by the following equation (8).

Ortho in equation (6) is a normal projection matrix shown in equation (9) below.

Here, f is the back end position of the field of view _{V R} (far), n is the front end position of the field of view _{V R} (near), t is the upper end position of the field of view _{V R} (top), b is the field of view range _{V R} lower end position of the (bottom), l is the leftmost position of the field of view _{V R} (left), r is the rightmost position of the field of view _{V R} (right). However, l and r are not shown in FIG. 7 (a).
Further, _{T Z} is the formula (6), as shown in the following equation (10), a projection plane, a translation matrix to only translate the distance _{L dist} from XY plane to the lens array La surface.

In the above equation (6), the projection plane is finally returned to the XY plane by multiplying by _TZ ^-1 .
Further, Oblique of formula (6), as shown in equation (11) below, when changing the viewpoint position from the virtual camera _{V O} to the virtual camera _{V C,} which is oblique projection matrix.

_{Here, (X C, Y C,} Z C) shows the coordinates (viewpoint position) of the virtual camera _{V C.}
Thus, isometric calculating means 22, for each divided area, by a single transformation matrix M _{F (Formula} (4)), by projecting the three-dimensional model data to the projection plane, the virtual camera in the element image it can be calculated pixel values of pixels corresponding to the lens center to be photographed from the direction of V _C.

The image projected on the projection surface by the oblique projection calculation means 22 is, for example, an image including pixels corresponding to the lens center H (projection surface image J) as shown in FIG. 8, and corresponds to the lens center H. pixels is a predetermined range around the pixel as the filter area a _R, is preferably subjected to low-pass filtering. For example, isometric calculating means 22, the pixel values of pixels corresponding to the lens center H, an average value in the filter region A _R, weighted addition value, and median value or the like.
Then, the oblique projection calculation means 22 calculates all the pixel values in the element image and generates an element image group for each divided region.
Returning to FIG. 3, the configuration of the element image generation device 1 will be described.

The element image integrating means 30 integrates the element image group for each divided region generated by the element image generating means 20 for each divided region. That is, the element image integrating means 30 generates one element image group by allocating the element image group generated for each divided area by the divided area-based element image generating means 20 to the area before division and connecting them. ..
Here, the element image integrating means 30 assigns the element image group of each divided area generated by the italic projection calculation means 22 to the position specified by the divided area information input from the oblique projection calculation means 22. Generate one element image group.

By configuring the element image generation device 1 as described above, the element image generation device 1 generates an element image by using oblique projection for each divided region, as compared with a method using ray tracing. Element images can be generated at high speed. Further, even if the display surface of the stereoscopic image display device becomes large, the element image generation device 1 can suppress the error of the element image low by performing oblique projection for each divided region.
The element image generation device 1 can be operated by a program (element image generation program) for operating the computer as each of the above-mentioned means.

[Operation of element image generator]
Next, the operation of the element image generation device 1 according to the embodiment of the present invention will be described with reference to FIG. 9 (see FIG. 3 as appropriate for the configuration).
First, the element image generation device 1 inputs the parameters of the stereoscopic image display device 2 and the screen division information (step S1).
Then, the element image generation device 1 sets the division area on the display surface D of the stereoscopic image display device 2 by the division area setting means 10 based on the parameters and the screen division information, and specifies each division area (step S2). ..

Then, the element image generation device 1 generates an element image group for each division area specified in step S2 by the element image generation means 20 for each division area.
That is, the element image generation device 1 calculates the shift amount from the lens center of the lens array La to the lens center of the division region by the lens center shift amount calculation means 21 of the element image generation means 20 for each division region (step S3). ).

Then, the element image generation device 1 passes through the lens center from each pixel position in the element image corresponding to the lens center shifted by the shift amount calculated in step S3 by the oblique projection calculation means 22, and only the viewing distance. The pixel value in the element image is calculated by obliquely projecting the three-dimensional model data captured by the separated virtual cameras onto the projection surface (display surface) (step S4).
As a result, the pixel values for the same pixel positions of all the element images in the divided region are calculated.
Then, the element image generation device 1 determines whether or not the calculation of the pixel values for all the pixels in the element image of the divided region is completed (step S5).

Here, when the calculation of the pixel values for all the pixels in the element image is not completed (No in step S5), the element image generation device 1 returns to step S4 and targets other pixels in the element image. , And sequentially calculate the pixel values in the element image.

On the other hand, when the calculation of the pixel values for all the pixels in the element image is completed (Yes in step S5), whether or not the element image generation device 1 has generated the element image group for all the divided regions. (Step S6).

Here, when the generation of the element image group is not completed for all the divided areas (No in step S6), the element image generation device 1 returns to step S2 and identifies the next divided area. An element image group corresponding to the divided area is generated.
On the other hand, when the generation of the element image group is completed for all the divided areas (Yes in step S6), the element image generating device 1 integrates the element image group for each divided area by the element image integrating means 30 (Yes). (Concatenation) to generate one element image group (step S7).

By the above operation, the element image generation device 1 can generate an element image group by oblique projection for each divided region in which the display surface of the stereoscopic image display device is divided.
Although the configuration and operation of the element image generation device 1 according to the embodiment of the present invention have been described above, the present invention is not limited to this embodiment.

[Modification 1]
For example, here, in the division area setting means 10, as shown in FIG. 4B, an example in which the display surface D is equally divided by the screen division information is shown. However, in the element image group using oblique projection, as described with reference to FIG. 16, the larger the screen, the larger the error in the peripheral portion.
Therefore, as shown in FIG. 10, the present invention may be divided so that the size is changed between the central portion and the peripheral portion of the display surface D.
As a result, the error becomes smaller with respect to the divided region in the center of the display surface that the observer gazes at more, so that the error in the region where the observer can easily gaze can be suppressed.

[Modification 2]
Further, here, as shown in FIG. 4B, the division area setting means 10 shows an example in which each division area of the display surface D is divided by a boundary line according to the screen division information.
However, as shown in FIG. 11, the overlapping region Q may be provided for each division region R and divided. At this time, the screen division information itself may specify the division area provided with the overlapping area Q, or the division area setting means 10 may use the overlapping area Q for the screen division information divided by the boundary line. May be added to set the division area.
In this case, the divided area-specific element image generation means 20 generates element images at the same position in the overlapping area Q. Then, the element image integrating means 30 averages the pixel values of each pixel of the element image for the same element image that overlaps in the divided region.
As a result, it is possible to suppress an error in the element image in the peripheral portion of the divided region R.

[Modification 3]
Further, as shown in FIG. 5, the lens center of the divided region R (2, 2) in the center of FIG. 4B has been described as being coincident with the lens center of the lens array La, but they are not necessarily the same. do not have to.
That is, the divided region having the lens center that does not match the lens center of the lens array La undergoes oblique projection conversion according to the shift amount of the lens center, as in the case other than the divided regions R (2, 2) in FIG. 4 (b). Just do it.

1 element image generation device 10 division area setting means 20 element image generation means for each division area 21 lens center shift amount calculation means 22 oblique projection calculation means 30 element image integration means

Claims (6)

An element image generator that generates an element image to be displayed on an integral stereoscopic display device equipped with a lens array in which element lenses are arranged two-dimensionally from three-dimensional model data.
A division area setting means for setting a division area with respect to the display surface of the stereoscopic display device, and
A virtual camera is virtually arranged at a predetermined viewing distance, and the direction from the virtual camera to the lens center of the element lens corresponding to the element image in the center of the divided area is set as the shooting direction for each divided area. A division area-specific element image generation means that obliquely projects the three-dimensional model data onto a projection plane of a predetermined projection coordinate system and calculates the pixel value of the element image in the division area.
An element image integrating means for generating an element image group to be displayed on the display surface of the stereoscopic display device by connecting the element image groups for each of the divided areas generated by the element image generation means for each divided area.
An element image generator, characterized in that it comprises. The element image generation means for each divided area is
A lens center shift amount calculating means for calculating the shift amount from the lens center of the lens array to the lens center of the element lens corresponding to the element image at the center of the divided area set by the divided area setting means.
For each pixel position of the element image corresponding to the element lens in the center of the lens array, the direction from the virtual camera arranged on the extension line passing through the lens center of the element lens to the lens center shifted by the shift amount. As the shooting direction, the oblique projection means for obliquely projecting the three-dimensional model data and calculating the pixel value of the pixel position of the element image in the divided region.
The element image generation device according to claim 1, further comprising. The division area setting means is characterized in that the number of vertical divisions and the number of horizontal divisions of the display surface are odd numbers, and the display surface is equally divided in the vertical direction and the horizontal direction, respectively. Alternatively, the element image generator according to claim 2. The element image generation device according to claim 1 or 2, wherein the divided area setting means sets the size of the divided area smaller in the central portion than in the peripheral portion of the display surface. The division area setting means sets the division area on the display surface with an overlapping area.
The element image integrating means is characterized in that, for the same element image overlapping in the divided region, the pixel value of each pixel of the element image is averaged to obtain the pixel value of the element image. The element image generator according to any one of claims 1 to 4. An element image generation program for causing a computer to function as each means of the element image generation device according to any one of claims 1 to 5.

Images