JP2008176522A - Method and device for generating defocus image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means efficiently generating and displaying a defocus image in real time in three-dimensional CG. <P>SOLUTION: The method for generating a two-dimensional image by applying perspective transformation to an object disposed within a three-dimensional space comprises steps of: (a) setting an adjustment limit of focal point by an observer's eyeballs; (b) disposing, for an object outside the adjustment limit, a copy of the object around the original position of the object; and (c) generating an image seen from the visual point of the observer by the perspective transformation. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a method and apparatus for generating a defocused image in the field of three-dimensional computer graphics (3DCG). That is, the present invention relates to a method and apparatus capable of generating a defocused image when generating a two-dimensional image by performing perspective transformation on an object placed in a three-dimensional space.

  Reflection and refraction display is one of the important themes in computer graphics research. Conventionally, these processes have been realized by ray tracing, but the processing time is enormous. Therefore, in recent years, a method of approximately displaying reflection / refraction in real time using environment mapping, which is a function of graphics hardware, is generally used.

  Ray tracing is originally a method for optical system design and is still used to simulate accurate refraction. Approximation methods are not currently used for designing optical systems that require accuracy.

  However, for the purpose of simulating how a human can see through a spectacle lens, it can be expected that there is no practical problem with the approximation method. Real-time display not only increases the efficiency of design verification, but also enables applications such as presentation to customers at dealers.

  The related research on the technology to display in consideration of real-time reflection and refraction is described below. In addition, related research on human vision correction simulation will be introduced.

(Real time reflection / refraction display)
The environment mapping proposed by Blinn to approximate reflection (Non-Patent Document 1) can be implemented by hardware (Non-Patent Documents 2 and 3) and is therefore widely used for real-time display applications. Since environment mapping involves the premise that the object to be reflected is at infinity, there is a drawback that the error is particularly large for a nearby (local) object.

  In recent years, methods for improving this have been devised. Hakura et al. Solved this problem by using a separate layer for local objects and preparing individual environment maps by preprocessing by ray tracing (Non-Patent Document 4). However, in this method, when the reflecting / refracting object moves and the local object changes frequently, the cost of re-processing is large. Szirmay-Kalos et al. Found a cube map that also has a depth value z for each pixel, and approximated the intersection of the ray and the object in the reflection / refraction direction by iterative calculation, thereby making more accurate reflection / refraction of local objects Realized in real time (Non-Patent Document 5). Since the purpose of this technique is a game, refraction was aimed at creating an atmosphere of a transparent object placed in the scene.

  In order to verify the state of the entire scene viewed through the lens, a more accurate method is required. In view of this, the present inventors have proposed a method of performing the intersection calculation of the ray and the scene through each vertex of the lens and realizing the same accuracy as ray tracing (see Japanese Patent Application No. 2006-124656).

(Simulation of vision correction)
As a drawing technique that takes into account the characteristics of the human eye, simulation of depth of field has been performed for some time (Non-Patent Documents 6 and 7).

A simulation model that considers the eyeball as a lens system and considers correction of vision has been proposed by Mostafawy et al. (Non-patent Document 8). Loos et al. Used wavefront tracing, which is a wave-optical approach, to simulate visual acuity correction using a progressive focus lens (a bifocal lens with no boundary) (Non-Patent Document 9).

Since all these conventional methods perform processing using distributed ray tracing, the display speed is slow and real-time processing is impossible.
1) JF Blinn and ME Newell.Texture and reflection in computer generated images.Communications of the ACM, Vol. 19, No. 10, pp. 542-547, 1976. 2) P. Haeberli and M. Segal. Texture mapping as A fundamental drawing primitive. In Fourth Eurographics Workshop on Rendering, pp. 259-266, 1993. 3) D. Voorhies and J. Foran. Reflection vector shading hardware. In Proc. SIGGRAPH '94, pp. 163-166, 1994. 4) ZS Hakura, JM Snyder, and JE Lengyel.Parameterized environment maps.In SI3D '01: Proceedings of the 2001 symposium on Interactive 3D graphics, pp. 203-208, 2001. 5) L. Szirmay-Kalos, B. Aszodi, I. Lazanyi, and M. Premecz. Approximate Ray-Tracing on the GPU with Distance Impostors. Computer Graphics Forum (Proc. Eurographics 2005), Vol.24, No.3, pp. 685-704, 2005. 6) RL Cook, T. Porter, and L. Carpenter. Distributed Ray Tracing. In Proc. SIGGRAPH'84, pp. 137-146, 1984. 7) P. Haeberli and K. Akeley. The accumulation buffer: hardware support for high-quality rendering. In Proc. SIGGRAPH '90, pp. 309-318, 1990. 8) S. Mostafawy, O. Kermani, and H. Lubatschowski. Virtual Eye: Retinal Image Visualization of the Human Eye. IEEE CG & A, Vol. 17, No. 1, pp. 8-12, January-February 1997. 9) J.Loos, Ph. Slusallek, and H.-P. Seidel.Using wavefront tracing for the visualization and optimization of progressive lenses.Computer Graphics Forum (Proc.Eurographics 1998), Vol.17, No.3, pp. 255-263, 1998. 10) J. Amanatides and A. Woo. A fast voxel traversal algorithm for ray tracing. In Proc. Eurographics 1987, pp. 3-10, 1987.

  The present invention has been made in view of the above situation. An object of the present invention is to provide means for efficiently generating a defocused image and enabling display in real time.

  In order to solve the above-described problems, the present invention has a configuration described in any of the following items.

(Item 1)
A method for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising the following steps:
(A) setting a focus adjustment limit by the eye of the observer;
(B) For an object present at a position beyond the adjustment limit, placing a copy of the object around the original position in the object;
(C) generating an image viewed from the viewpoint of the observer by the perspective transformation;

  According to the invention according to item 1, a copy of the object is placed around the object that exists at a position that exceeds the focus adjustment limit. Therefore, in an image obtained by perspective transformation, The object will appear blurred. Thereby, a defocused image can be generated.

(Item 2)
Item 2. The image generation method according to Item 1, wherein there are a plurality of copies of the object.

(Item 3)
3. The image generation method according to item 2, wherein the copy of the object is arranged on a circumference centered on an original position of the object.

(Item 4)
3. The image generation method according to item 2, wherein the copy of the object is arranged on a plurality of concentric circles centered on the original position of the object.

(Item 5)
The object is modeled by a polygon with at least three vertices;
5. The image generation method according to any one of items 1 to 4, wherein the copy of the object in step (b) is arranged in units of the polygon.

  In the invention according to item 5, a copy of the object is generated and arranged for each polygon. By finely setting the polygons constituting the model of the object, the amount of blur can be changed according to the part of the object, and a defocus image closer to the actual can be generated.

(Item 6)
In the step (b), the distance between the copy of the object and the original position in the object is set according to the distance from the reference point to the object. An image generation method according to any one of the above.

  According to the invention according to item 6, the distance between the object and its copy can be increased as the position of the object is further away from the adjustment limit. Thereby, it is possible to generate a defocused image that is close to reality.

(Item 7)
Item 7. The image generating method according to Item 6, wherein the reference point is an observer's viewpoint.

  According to the invention according to item 7, it is possible to simulate the “viewing from the observer” in the case of the naked eye.

(Item 8)
The image generation method according to item 6, wherein the reference point is an intersection of the lens surface and the line of sight.

  According to the invention according to item 8, it is possible to simulate the appearance accompanied by refraction at the lens, for example, when glasses are used.

(Item 9)
The object is modeled by a polygon having at least three vertices, and the distance from the reference point to the object is a distance from the reference point to the vertex of the polygon. The image generation method according to any one of the above.

(Item 10)
The image generation method according to any one of items 1 to 9, wherein the focus adjustment limit is a near point or a far point of focus adjustment.

  Of course, both the near point and the far point of the focus can be set as adjustment limits. Here, the near point is a point where the focus is not achieved near this point. When the near point is the adjustment limit, “exceeding the adjustment limit” means closer to the near point. Further, the far point is a point at which the focus is not achieved at a far distance. When the far point is the adjustment limit, “exceeding the adjustment limit” means being far from the far point. Focus adjustment is possible between the near point and the far point.

Here, the near point and the far point can be set as follows, for example.
・ Set for each use based on the eyeball of the subject (observer);
-Preset based on average eyeball condition;
・ Set assuming glasses are used.

(Item 11)
The image generation method according to item 10, wherein the far point or the near point is after correction by a lens.

  According to the invention according to item 11, the state after correction by the lens can be simulated easily.

(Item 12)
Any of items 1 to 11, wherein in step (b), the distance between the copy of the object and the original position in the object is set by a process including the following steps: 2. The image generation method according to item 1.
(B1) arranging a plurality of voxels in a three-dimensional space to be subjected to the perspective transformation;
(B2) A step of presetting the distance for each voxel.

  According to the invention according to item 12, the real-time property of display can be improved by performing a prior calculation for each voxel.

(Item 13)
13. The image generation method according to item 12, wherein the area occupied by the voxel is set to be finer as it is closer to the viewpoint of the observer.

  According to the invention according to item 13, the closer to the observer's viewpoint, the finer the blur amount (defocus amount) can be changed, so that the actual appearance can be simulated more accurately. In addition, such voxel setting is a process having high affinity with the perspective transformation essential for image generation using the three-dimensional modeling space, and thus it is possible to shorten the processing time required for image generation.

(Item 14)
An apparatus for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space,
The apparatus includes a storage unit and an image generation unit,
The storage unit stores data and computer software necessary for processing,
The image generation unit performs the following processing:
(A) a process of setting a focus adjustment limit by the eyeball of the observer;
(B) For an object existing at a position exceeding the adjustment limit, a process of arranging a copy of the object around the original position of the object;
(C) Processing for generating an image viewed from the observer's viewpoint by the perspective transformation.

(Item 15)
A method of generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising causing a computer to execute an image generation method including the following steps: Computer program:
(A) setting a focus adjustment limit by the eye of the observer;
(B) For an object present at a position beyond the adjustment limit, placing a copy of the object around the original position in the object;
(C) generating an image viewed from the viewpoint of the observer by the perspective transformation;

(Item 16)
Image data obtained by perspective-transforming an object placed in a three-dimensional space,
For objects existing beyond the limit of focus adjustment by the observer's eyeball, the perspective is converted with multiple copies of the object placed around the original position of the object. Feature image data.

  According to the invention of item 16, for an object at a position exceeding the focus adjustment limit, an object obtained by copying the object is disposed around the original position of the object. For this reason, it is possible to provide a defocused image having a blur that is actually close.

(Item 17)
A method for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising the following steps:
(A) For an object existing at a position beyond the focus adjustment limit by the observer's eyeball, placing a copy of the object around the original position of the object;
(B) generating an image viewed from the viewpoint of the observer by the perspective transformation;

  The focus adjustment limit may be set at each use, or some fixed value or an automatically generated value (for example, a value calculated from the user's age or the like) may be used. Also in this case, the same effect as item 1 can be exhibited.

  According to the present invention, it is possible to provide means for efficiently generating a defocused image and enabling real-time display.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

(Image generation device)
First, an image generation apparatus used for implementing this embodiment will be described with reference to FIG. The image generation apparatus includes an input unit 10, an input interface 11, a display unit 20, an output interface 21, a storage unit 30, an image generation unit 40, and a bus 50. These components can be configured by hardware itself such as a workstation or a computer, or by incorporating a computer program in the hardware.

  The input unit 10 is connected to the bus 50 via the input interface 11. The input unit 10 is an appropriate input device such as a keyboard, a mouse, or a touch pad, for example, and can be used to input a command from an operator.

  The display unit 20 is connected to the bus 50 via the output interface 21. The output unit 20 is an appropriate output device such as a display or a printer. The output unit 20 is configured to receive the image generated by the image generation unit 40 (two-dimensional image obtained by perspective transformation) and present it to the user.

  The storage unit 30 is a device that can store and read data (including computer programs). As the storage unit 30, an appropriate device such as a magnetic storage device (for example, a hard disk), a semiconductor storage device (for example, a memory board), or an optical storage device (for example, a storage device using an optical disk such as a DVD) can be used. . The storage unit 30 may be built in a workstation or computer, or may be connected to the outside via an interface device. Furthermore, the storage unit 30 may be a storage device that can be used via a network. The storage unit 30 may be physically configured by a plurality of storage devices.

  The storage unit 30 according to the present embodiment stores in advance a computer program for configuring the image generation unit 40 using a workstation or a computer, and data used for performing the method of the present embodiment. In addition, the storage unit 30 can store data generated in accordance with the execution of the method of the present embodiment.

The image generation unit 40 is configured to perform the following processing.
(A) a process of setting a focus adjustment limit by the eyeball of the observer;
(B) For an object that exists at a position beyond the adjustment limit, a process of placing a copy of the object around the original position of the object;
(C) Processing for generating an image viewed from the observer's viewpoint by the perspective transformation.

  Details of these processes will be described in the description (described later) of the image generation method according to the present embodiment. The image generation unit 40 is configured as a functional element by executing a computer program on a workstation or a computer.

  The bus 50 is a transmission path that enables data exchange between the above-described units. The bus 50 is not limited to a configuration in which one bus is shared, and a configuration in which a dedicated bus is separately used. Further, the transmission path may be wireless instead of wired. In short, the transmission path only needs to be able to exchange data, and the physical configuration and the protocol to be used are not limited.

(Image generation method according to this embodiment)
Next, an image generation method according to the present embodiment will be described. In the following, vertex-based ray tracing used for refraction processing for accurately displaying distortion caused by a lens will be described first. In the description of the present embodiment, all target objects are polygon models.

(algorithm)
The processing procedure performed for each frame is as follows. A description will be given in correspondence with FIG. This algorithm is the same as that described in Non-Patent Document 11 by the inventors.
(1) Obtain all reflection / refraction vectors at the vertex 1a of the reflection / refraction object surface 1;
(2) Obtain virtual viewpoint 2 and field of view and draw environment map;
(3) For each vertex of a reflective / refractive object:
(a) Find the point (intersection) 4 where the extension of the reflection / refraction vector intersects the object 3 in the scene;
(b) Find a straight line 5 connecting the intersection 4 and the virtual viewpoint 2;
(c) Find the intersection of the straight line 5 and the environment map projection plane 6;
(d) The coordinates of the intersection 4 on the environment map image are set as the texture coordinates of the vertex 1a;
(4) Draw the reflection / refraction object and the entire scene from the original viewpoint.

  The virtual viewpoint 2 in (2) above is obtained as a point closest to the extension line by selecting several reflection / refraction vectors. This algorithm basically guarantees that the exact position of each vertex of the reflective / refractive object is reflected.

  As an exception, there may be a discrepancy between occluded objects due to a shift between the vertex and the virtual viewpoint. This is almost limited to cases where the reflective / refractive object is large and the ray spread is extremely large, and this is not a problem in lens simulation.

  Although an example of reflection is shown in FIG. 2, in the case of a refracting object such as a lens, a similar procedure is performed by obtaining a ray after two refractions of incident and outgoing.

(Reflection and refraction display results)
Display examples of vertex-based ray tracing reflection are shown in FIGS. As a comparison, FIGS. 3C and 3D show examples of reflection display by conventional cube mapping. It can be seen that the reflection of local objects, which are displaced by cube mapping, is accurately processed by vertex-based ray tracing. In these figures, an image reflected on the door mirror of the car is displayed. In the images of FIGS. (B) and (d), mirror wireframe models are superimposed. Many spheres were placed at the intersection of the reflection vector and the scene. In vertex-based ray tracing, the intersection is accurately reflected in the vertex. Further, FIG. 4 shows a display example of refraction through a lens. This figure shows the display result of refraction through a concave lens. FIG. 2B is an enlarged view of the vicinity of the right end of FIG. 2A, in which the lens apex is displayed in a wire frame, and a small sphere is arranged at the intersection of the refraction vector and the scene. Also in this figure, it can be seen that the intersection of the refraction vectors is accurately reflected at the apex.

(Defocus correction)
Hereinafter, modeling of defocus and display processing in consideration of defocus after correction by the eyeglass lens will be described. In the following method, a simple model using geometric optics is used, and the elements of wave optics are not considered.

(Example 1: In the case of the naked eye)
First, consider defocusing with the naked eye. In ophthalmic optics, when the closest focus is achieved, the distance from the eyeball to that point is referred to as the near point, and conversely, the distance when the focus is adjusted farthest is referred to as the far point. A person with a large near point has a strong presbyopia (presbyopia), and a person with a small far point has a strong myopia.

  FIG. 5 is a simple model used in the method of this embodiment. Refraction of the eyeball originally occurs in two places, the cornea and the lens, but here they are regarded as one thin lens (see FIG. 5 (a)).

Defocusing occurs when an object (object) closer to the near point is seen in the state of being matched with the near point. Assume that the diameter of the pupil opening is A, the distance from the pupil to the near point is n, and there is a point-like subject inside z _n further from the near point (FIG. 5B). Even if it tries to look at this subject, the ray from the opening does not converge to one point, but spreads in a disk shape with a diameter D _n perpendicular to the line of sight centering on the subject. here

D _n = Az _n / n (1)

It is. In the proposed method, the disk with the diameter D _n is regarded as a point spread function, and a large number of objects are sampled in the disk and then drawn with a normal CG camera model. That is, for an object existing at a position beyond the near point as the adjustment limit, a copy of the object is arranged around the original position of the object.

  The drawing result of each sample (that is, each copy) can be synthesized by weighting (averaging) using an accumulation buffer (see Non-Patent Document 7). Thereby, it is possible to create an image in which a plurality of copies exist as a semi-transparent object in the vicinity of the object. This is a defocused image. Specifically, for example, the following processing is possible. First, a perspective transformation is performed on the 0th sample to generate an image, which is put in an accumulation buffer. Next, an image is similarly generated for the first sample and placed in the accumulation buffer. Thereafter, similarly, a two-dimensional image is generated for a predetermined number of samples. Then, these two-dimensional images can be combined to generate a defocused image. At this time, if the brightness of the original image is 1, weighting is performed so that the brightness of the composite image is 1 (for example, if the number of samples is 10, the brightness of each sample image is 1/10).

  As shown in FIG. 5 (c), defocusing occurs even when a subject farther than the far point is viewed. Again, the subject

D _f = Az _f / f (2)

Defocusing is simulated by sampling in the disk. That is, for an object that exists at a position beyond the far point as the adjustment limit, a copy of the object is placed around the original position of the object.

(Vertex displacement processing)
A method of generating a defocus image for a polygon model that is an actual subject will be described with reference to the flowchart of FIG.

(Step SA-1)
First, as described above, the near point and the far point are set according to the characteristics of the assumed eyeball of the subject. This setting can be performed by the operator using the input unit 10. However, for the positions of the near point and the far point, an appropriate fixed value set in advance may be used. In this case, setting work at the time of use can be omitted.

(Step SA-2)
Next, the sampling number S is determined, and clones (that is, copies) are made for all given models (polygons in this embodiment). Each clone is assigned a clone number i (i = 0, 1, .., S-1). For convenience, the original model is also assigned a clone number 0 and treated as a kind of clone. During the display process, all S clones are displayed, but all vertices are displaced in the direction perpendicular to the line of sight within the range of Equation (1) or Equation (2) according to the distance from the viewpoint ( (However, this means that instead of deleting the original vertex, a copy of the vertex is generated). If the vertex is between the near and far points, the displacement is zero. By connecting the displaced vertices, a copy polygon can be generated (described later). Here, the displacement amount of each vertex is determined by the distance from the viewpoint to the vertex. Therefore, the displacement amounts of the three vertices constituting one polygon are respectively determined by the distance from the viewpoint. By connecting the displaced vertices, a copy polygon is generated. Therefore, as a result, the displacement amount of the copy polygon itself is also determined by the distance from the viewpoint.

The direction and magnitude of the displacement is determined by the clone number of the model to which the vertex belongs in the displacement vector set consisting of S displacement vectors d _i (i = 0, 1, ..., S-1). Therefore, select. FIG. 7 shows an example of the displacement vector. In the figure, one displacement vector set (S = 17) is illustrated for each of the inside of the near point and the outside of the far point.

At the time of implementation, only one set of normalized displacement vectors is retained, and the size is determined on the spot according to the value of D _n or D _f at the vertex position. Since the displacement is zero between the near point and the far point, the displacement vector set is such that all the displacement vectors are zero. This means that it is fully focused and has no defocus at all.

(Step SA-3)
Then, as described above, a copy of the model is placed around the object. That is, a copy polygon can be generated by connecting the vertices of the copy displaced in the above procedure. As described above, the distance between the copy and the object is determined by the displacement vector set. As is clear from the equations (1) and (2), the distance between the copy of the object and the original position in the object is set according to the distance from the reference point (the viewpoint in the above example) to the object. Is done. That is, for an object at a position closer to the near point, the shift amount increases as it is closer, and for an object at a position farther from the far point, the shift amount increases as it is further away.

(Step SA-4)
Next, the three-dimensional scene generated as described above is perspective-transformed in the view volume. Thereby, drawing can be performed. Since the perspective transformation itself can use a well-known method, detailed description thereof will be omitted. The obtained image data is “a state in which a plurality of copies of the object are arranged around the original position of the object for the object existing beyond the focus adjustment limit by the observer's eyeball”. It has been “transparent-converted”. An example of image data will be described later.

(Example 2: When using glasses)
(Spatial distribution of defocus amount)
In the method of the first embodiment, when the vertex is determined, the distance from the viewpoint is determined, and the displacement vector is obtained by calculating the displacement amount by the equation (1) or (2). This may be used as a naked eye visual model, but when a lens is used, it is difficult to calculate a displacement vector for each vertex. Therefore, a method for generating a defocused image when glasses are used will be described with reference to the flowchart shown in FIG. In this method, as shown in FIG. 9, space division is performed by cubic voxels, and a displacement vector set is pre-calculated for each voxel.

  Here, the precondition that the positional relationship between the eye and the eyeglass lens is fixed is used. The voxel space is set with a size sufficient to encompass the entire scene. Therefore, the voxel is arranged in the view volume.

Before the detailed description, the outline of the pre-calculation will be described below. First, after initializing the displacement vectors of all the voxels, the displacement vector in the state matched with the near point is recorded in each voxel, and the displacement vector in the state matched with the far point is recorded in each voxel.

  The process of recording the displacement vector once is as follows. A ray bundle (light beam) for near points or far points is emitted from a viewpoint to a point on the lens surface, and the spread of rays in the voxel through which the bundle passes is accumulated as a displacement vector set. This process is repeated by generating and processing a light beam by scanning points on the lens surface at regular intervals. When viewed from the side of each voxel, the light beam passes a plurality of times, but an average displacement vector set is finally calculated from the displacement vector set accumulated a plurality of times.

(Step SB-1)
Hereinafter, a method for processing the near-point light beam will be described in detail (see FIG. 10). First, a light ray is generated from a viewpoint 7 to a given point on the lens. In order to generate a light beam centered on this light beam, it will be referred to as a reference light beam 8. The starting point 9 of another ray in the light beam is provided in the vicinity of the viewpoint. This starting point corresponds to the surface of the pupil and is distributed almost uniformly in a disk having a diameter A (pupil opening diameter described in the first embodiment) centered on the viewpoint and perpendicular to the reference ray.

  Next, a near point 61 (in the case of the naked eye) on the reference ray is obtained, and all rays other than the reference ray pass therethrough. In this example, it is assumed that the position of the near point 61 is measured and input by an appropriate method. As a result, the light beam has an elongated conical shape that converges at a near point.

  Further, each light beam including the reference light beam is refracted twice by the lens 63 to obtain a point at which the refracted light beam converges most, and this is set as a correction near point 62. The point at which the luminous flux converges most can be obtained by calculation using the least square method. The correction far point can be obtained in the same manner. Thereby, the focus adjustment limit (correction near point and correction far point) in the case of using glasses can be set.

(Step SB-2)
Traces the voxel through which the refracted reference ray passes for a place closer to the viewpoint than the correction near point (see Non-Patent Document 10 for the method of tracing), and accumulates the spread of the light beam after refraction at the voxel as a displacement vector set. . Specifically, the midpoint of the reference ray (after refraction) cut out by the voxel is obtained, and a plane that passes through the midpoint and is perpendicular to the reference ray is obtained. A vector group drawn from the midpoint is defined as a displacement vector set at a point where each light beam of the refracted light beam intersects the vertical plane (see FIG. 10 (c)). Here, the displacement vector set itself may be the same as in the case of the naked eye (see FIG. 7).

  The above process is repeated for each point on the lens (usually a vertex). That is, in the second embodiment, if the intersection of the lens surface and the line of sight (reference ray) is taken as a reference point, the length of the displacement vector set (separation distance to the copy) depends on the distance from this reference point. It will be decided. Since the same processing is performed for the far point, the detailed description of the far point is omitted.

  Finally, an average is obtained from the displacement vectors stored for each voxel, and the preprocessing is terminated.

(Step SB-3)
When executing the display process, the following procedure is performed for the vertices of all clone models. First, the voxel to which the vertex belongs is obtained. This is obtained by a single coordinate transformation. Next, a displacement vector is selected from the displacement vector set recorded in the voxel according to the clone number of the vertex, and the vertex is actually displaced by the same method as in the first embodiment. Similarly to the first embodiment, each clone model is synthesized in an actual three-dimensional scene. This makes it possible to generate a large number of polygon copies composed of three vertices.

(Step SB-4)
Next, as in step SA-4 of the first embodiment, perspective transformation can be performed to generate a defocused image.

(Experimental result)
FIG. 11 shows a result obtained by performing defocus processing by the method of Example 2 and drawing in consideration of correction (refraction) by the lens.
(a): An image with the naked eye (near point 0.2 m, far point 10 m).
(b): Myopic and presbyopic image with the naked eye (near point 0.5m, far point 1.3m).
(c): The result of correcting (b) above with a concave lens for myopia.
(d): The result of correcting (b) above with a convex lens for presbyopia.

  In all cases, the pupil diameter A = 0: 012m, the number of voxels 110 × 77 × 42, the number of defocused samples (copy number) S = 10, and the refractive index 1:66 (glass).

  In this experiment, Pentium4 (trademark) 3.8GHz was used as the CPU, and GeForce 7800GTX (trademark) was used as the GPU. First, as shown in FIG. 4A, when only refraction processing by a lens is performed and defocusing is not performed, display can be performed at 30 to 40 fps (frame per second). This is when the relative position of the eye and the lens is fixed (regeneration of the refraction vector is not required). If this assumption is removed, the display rate drops to 6 fps. The number of polygons of the lens used is about 1,500 on the front and back sides, and the number of polygons on the scene is about 10,000.

  Next, for defocus processing, it took about 3 seconds to refract about 1,500 luminous fluxes (10 rays) and pre-calculate a displacement vector set with 10 samples for 110 × 77 × 42 voxels. The display performance of the defocus correction (FIGS. 11 (c) and 11 (d)) was 12 to 15 fps. The display performance of defocus with the naked eye without using a lens (FIGS. 11A and 11B) is 30 fps.

(Preprocessing of model data)
One of the limitations in the method of the second embodiment is that “the object needs to be polygonized with such a fineness that the vertices are arranged at the voxel interval”. Even a single quadrilateral must be finely meshed if it is large. Therefore, it is certain that it takes time to create data. Regarding the influence on the display performance, it can be minimized by so-called LOD (Level of detail) processing. In other words, models with different levels of detail for meshing are prepared in advance for each target, and models with a high level of detail are displayed at parts that are relatively close to the viewpoint where fine meshing is required. Control to display a low model.

(Concept for gaze)
Considering a certain moment, human vision concentrates high visual acuity only in a limited range of gaze direction in a wide visual field, and surrounding visual acuity is extremely low. Even when drawing is performed in consideration of the characteristics of the human eyeball as in the present embodiment, such a process considering the line of sight is difficult for two reasons. One is that it is difficult to model a visual field distribution over a wide visual field. The other is that when presenting results, it is difficult to detect the actual user's line of sight and display it promptly.

  In the present embodiment, the results obtained when the best adjustments are individually made for each part (each voxel) of the image to be drawn are accumulated into an output image, and the visual acuity distribution in a wide area is ignored. Thus, it seems strange that the input conditions are different for each part of the output image. However, since the user who finally sees the resulting image can also exhibit high visual acuity only in a small part of the image, it is not very meaningful to simulate the entire image under the same conditions. At least in the application of verification of correction results with spectacle lenses, it is useful to draw with the best adjustment in each part.

(Advantages of this embodiment)
According to the method of the present embodiment, real-time processing of reflection and refraction with the same accuracy as ray tracing is realized by real-time processing, and defocus simulation by refraction correction, which has taken a lot of time in conventional distributed ray tracing, is also interactively displayed. It can be realized at a rate.

  Since this method can increase the display accuracy, it is considered that this method can be sufficiently utilized for verification of an actual spectacle lens design.

  As described above, the method of the present embodiment proposes a method of correcting a map image by performing ray tracing for each vertex of a reflective / refractive object based on environment mapping. Using this technique, real-time display can be performed with sufficient accuracy to simulate distortion of the image through the lens. In order to model defocusing, the visual acuity characteristics with the naked human eye were expressed as a luminous flux obtained by sampling rays. A distribution of the defocus amount in the space can be obtained using these light beams, and defocus processing can be performed at each vertex of the object. As a result, it is possible to simulate the appearance with the naked eye and to simulate the correction of refraction at the eyeball by the lens.

  It should be noted that the scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present invention.

  For example, each component described above may exist as a functional block, and may not exist as independent hardware. As a mounting method, hardware or computer software may be used. Furthermore, one functional element in the present invention may be realized by a set of a plurality of functional elements. A plurality of functional elements in the present invention may be realized by one functional element.

  Moreover, the functional element may be arrange | positioned in the position physically separated. In this case, the functional elements may be connected by a network.

It is a schematic functional block diagram of the image generation apparatus which concerns on this embodiment. It is explanatory drawing for demonstrating the image generation method by vertex-based ray tracing. FIGS. 3A and 2B are diagrams showing results drawn by the method of FIG. 2, in which FIGS. 1A and 1B show results of drawing by vertex-based ray tracing, and FIGS. 2C and 2D show drawing by conventional cube mapping. Results are shown. It is a figure which shows the drawing result in the refraction through a concave lens. FIG. 2B is an enlarged view of the vicinity of the right end of FIG. It is explanatory drawing which shows the simple model of the defocus by an eyeball. FIG. (A) shows a simple eyeball model. Figure (b) shows the state of focusing close to the limit (near point). FIG. (C) shows a state in which the focus is far (far point) to the limit. 3 is a flowchart for explaining an outline of a drawing method according to the first embodiment. It is explanatory drawing which shows the example of the vertex displacement vector determined by the distance from a viewpoint. 12 is a flowchart for explaining an outline of a drawing method according to the second embodiment. It is explanatory drawing which shows the example of the spatial distribution (voxel) of a defocus amount (displacement vector). It is explanatory drawing explaining the accumulation | storage process to the voxel of a displacement vector set. Figure (a) shows the generation of the luminous flux. Figure (b) shows the light beam after refraction. FIG. 3C shows a displacement vector set accumulated in the voxel through which the reference light beam after refraction passes. It is an example of the image which shows the display of the defocus which considered visual acuity, and the result of the correction. Figure (a) shows a normal image (near point 0.2m, far point 10m). Figure (b) shows a myopic and presbyopic image (near point 0.5 m, far point 1.3 m). Fig. (C) shows the result of correcting Fig. (B) with a concave lens for myopia. Fig. (D) shows the result of correcting Fig. (B) with a convex lens for presbyopia.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Input part 11 Input interface 20 Display part 21 Output interface 30 Memory | storage part 40 Image generation part 50 Bus

Claims (17)

A method for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising the following steps:
(A) setting a focus adjustment limit by the eye of the observer;
(B) For an object present at a position beyond the adjustment limit, placing a copy of the object around the original position in the object;
(C) generating an image viewed from the viewpoint of the observer by the perspective transformation;   The image generation method according to claim 1, wherein there are a plurality of copies of the object.   The image generation method according to claim 2, wherein the copy of the object is arranged on a circumference centered on an original position of the object.   The image generation method according to claim 2, wherein the copy of the object is arranged on a plurality of concentric circles centered on an original position of the object. The object is modeled by a polygon with at least three vertices;
The image generation method according to claim 1, wherein the copy of the object in step (b) is arranged in units of the polygon. In the step (b), the distance between the copy of the object and the original position in the object is set according to the distance from a reference point to the object. 6. The image generation method according to any one of 5 above.   The image generation method according to claim 6, wherein the reference point is an observer's viewpoint.   The image generation method according to claim 6, wherein the reference point is an intersection of a lens surface and a line of sight.   The object is modeled by a polygon having at least three vertices, and the distance from the reference point to the object is a distance from the reference point to the vertex of the polygon. The image generation method according to any one of the above.   The image generation method according to claim 1, wherein the focus adjustment limit is a near point or a far point of the focus.   The image generation method according to claim 10, wherein the far point or the near point is after correction by a lens. The distance between the copy of the object and the original position in the object in the step (b) is set by a process including the following steps. The image generation method according to claim 1.
(B1) arranging a plurality of voxels in a three-dimensional space to be subjected to the perspective transformation;
(B2) A step of setting the distance for each voxel.   The image generation method according to claim 12, wherein an area occupied by the voxel is set to be finer as it is closer to an observer's viewpoint. An apparatus for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space,
The apparatus includes a storage unit and an image generation unit,
The storage unit stores data and computer software necessary for processing,
The image generation unit performs the following processing:
(A) a process of setting a focus adjustment limit by the eyeball of the observer;
(B) For an object existing at a position exceeding the adjustment limit, a process of arranging a copy of the object around the original position of the object;
(C) Processing for generating an image viewed from the observer's viewpoint by the perspective transformation. A method of generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising causing a computer to execute an image generation method including the following steps: Computer program:
(A) setting a focus adjustment limit by the eye of the observer;
(B) For an object present at a position beyond the adjustment limit, placing a copy of the object around the original position in the object;
(C) generating an image viewed from the viewpoint of the observer by the perspective transformation; Image data obtained by perspective-transforming an object placed in a three-dimensional space,
For objects existing beyond the limit of focus adjustment by the observer's eyeball, the perspective is converted with multiple copies of the object placed around the original position of the object. Feature image data. A method for generating a two-dimensional image by performing perspective transformation on an object arranged in a three-dimensional space, and further comprising the following steps:
(A) For an object existing at a position beyond the focus adjustment limit by the observer's eyeball, placing a copy of the object around the original position of the object;
(B) generating an image viewed from the viewpoint of the observer by the perspective transformation;