JP2024004042A - Simulation device, data transmission device, model generation device, preliminary data generation device, image generation device, simulation method, and simulation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve simulation accuracy of reproducing the appearance of a specific scene viewed by a user through a spectacle lens, for example.

SOLUTION: According to a first embodiment of the present disclosure, a simulation device which reproduces the appearance of a specific scene viewed by a user through a spectacle lens is provided. The simulation device comprises: a processing unit which, on the basis of at least eyeball information concerning a user's eyeball and designing information concerning the spectacle lens, generates a simulation image indicating the appearance of the scene; and an output unit which outputs the simulation image generated by the processing unit to a display device.

SELECTED DRAWING: Figure 17

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present disclosure relates to a simulation device, a data transmission device, a model generation device, a preliminary data generation device, an image generation device, a simulation method, and a simulation program.

Patent Document 1 describes a simulation device that simulates how people see through spectacle lenses. In simulation devices, it is required to improve the accuracy of simulation.

Japanese Patent Application Publication No. 8-215149

According to a first aspect of the present disclosure, a simulation device is provided that reproduces how a specific scene is viewed by a user through a spectacle lens. The simulation device includes at least a processing unit that generates a simulation image showing how a scene is viewed based on eyeball information regarding the user's eyes and design information of the spectacle lens, and a display device that displays the simulation image generated by the processing unit. and an output section for outputting to the.

According to the second aspect of the present disclosure, an acquisition unit that acquires eyeball information of a user, and a transmission that transmits the eyeball information acquired by the acquisition unit to a model generation device that generates a different eyeball model for each individual. A data transmitting device is provided, comprising: a.

According to a third aspect of the present disclosure, there is provided a model generation device that generates an eyeball model for reproducing the appearance of a specific scene viewed by a user through a spectacle lens. The model generation device generates a different eyeball model for each individual by modeling the user's eyeballs based on information including the user's eyeball information.

According to a fourth aspect of the present disclosure, there is provided an a priori data generation device that generates a priori data for generating a simulation image that reproduces how a specific scene is viewed by a user through a spectacle lens. The prior data generation device generates prior data using an optical system that includes a user's eyeball model that is modeled using the user's eyeball information and a spectacle lens model that models a spectacle lens.

According to a fifth aspect of the present disclosure, there is provided an image generation device that generates a simulation image showing how a specific scene is viewed by a user through a spectacle lens. The image generation device generates a simulation image based on a priori data generated based on eyeball information regarding the user's eyeballs and design information of the spectacle lens, and displays the generated simulation image on a display device.

According to a sixth aspect of the present disclosure, a simulation method is provided that reproduces the appearance of a specific scene viewed by a user through a spectacle lens. The simulation method includes, at least, generating a simulation image showing how a scene is viewed based on eyeball information regarding the user's eyeballs and design information of a spectacle lens, and outputting the generated simulation image to a display device. ,including.

According to a seventh aspect of the present disclosure, there is provided a simulation program that causes a computer to function as a simulation device for reproducing the way a specific scene is viewed by a user through a spectacle lens. The simulation program includes at least a processing unit that generates a simulation image showing how a scene is viewed based on eyeball information regarding the user's eyes and design information of the spectacle lens, and a simulation image generated by the processing unit. It functions as an output unit that outputs to a display device.

FIG. 1 is a diagram showing an example of a simulation system according to the present embodiment. FIG. 2 is a functional block diagram of a processor according to the present embodiment. FIG. 3 is a diagram illustrating a method of replacing the posterior corneal surface of the eyeball model with a free-form surface according to the present embodiment. FIG. 1 is a schematic diagram of an optical system according to the present embodiment. It is a schematic block diagram of the data calculation part based on this embodiment. FIG. 3 is a diagram showing an example of a viewing frustum according to the present embodiment. It is a figure explaining the calculation method of the amount of strain displacement concerning this embodiment. FIG. 3 is a diagram showing an example of distortion vector array data according to the present embodiment. FIG. 3 is a diagram illustrating a method of calculating a confusion ellipse according to an embodiment. FIG. 2 is a schematic configuration diagram of an image generation unit according to the present embodiment. FIG. 3 is a diagram illustrating a specific scene image according to the present embodiment. FIG. 2 is a schematic diagram of an image processing filter according to the present embodiment. FIG. 3 is a diagram illustrating a method for creating an image processing filter according to the present embodiment. FIG. 2 is a diagram illustrating an image processing filter according to the present embodiment. FIG. 3 is a diagram illustrating distortion processing according to the present embodiment. FIG. 3 is a diagram illustrating blur processing according to the present embodiment. It is a flowchart of the flow of creation processing of an individual eyeball model concerning this embodiment. It is a flowchart of the flow of advance data creation processing concerning this embodiment. It is a flowchart of the flow of real-time processing concerning this embodiment. 1 is a diagram showing a configuration example of a simulation system 1 according to the present embodiment.

Hereinafter, the present invention will be explained through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all combinations of features described in the embodiments are essential to the solution of the invention.

In the simulation device described in Patent Document 1, eyeball information regarding the user's eyeballs, such as pupil diameter, axial length, and corneal surface shape, is not taken into consideration when simulating how the user sees through spectacle lenses. That is, differences in eyeball information for each individual are not taken into consideration in the above-mentioned visual simulation. The simulation device 2 according to the present embodiment takes into account eyeball information regarding the user's eyeballs and simulates (reproduces) the way the user sees a specific scene through the eyeglass lens. It is possible to perform a simulation of the above-mentioned appearance according to the above, and it is possible to improve the accuracy of the simulation. Below, a simulation system 1 having a simulation device 2 according to an embodiment of the present disclosure will be specifically described.

FIG. 1 is a diagram showing an example of a simulation system according to this embodiment. As shown in FIG. 1, the simulation system 1 includes a simulation device 2 and a display device 3.

The simulation device 2 is connected to the display device 3 wirelessly or by wire. The simulation device 2 performs a simulation of how a specific scene is viewed by a user through spectacle lenses (hereinafter referred to as "view simulation"). The simulation device 2 outputs the results of the viewing simulation to the display device 3.

The visual simulation is performed, for example, when a user purchases eyeglass lenses. For example, when purchasing eyeglass lenses, a user may want to check how the user will see through the desired eyeglass lenses before purchasing. In such a case, information about the user's eyeballs and information about the eyeglass lens selected by the user are directly or indirectly input into the simulation device 2 in advance, so that the user can The results of the visual appearance simulation of this embodiment, that is, the visual appearance through the eyeglass lens (still image or video) can be confirmed before purchasing the eyeglass lens. Note that information regarding the user's eyeballs may be input indirectly to the simulation device 2, for example, by a wavefront aberration measuring device installed in a facility (e.g., an optician shop or hospital). It includes measuring data and inputting the measured data to the simulation device 2 via a communication network.

The simulation device 2 performs a data generation process that generates data (hereinafter referred to as "prior data") used in a visual simulation process (hereinafter referred to as a "visual simulation process"), and performs a visual simulation process. be carried out before. Then, the simulation device 2 performs a viewing simulation process using the prior data generated in advance.

The simulation device 2 is, for example, a computer. The simulation device 2 may be installed in a facility such as an eyeglass store or a hospital. For example, the simulation device 2 may be installed in a facility where a device for measuring eyeball information such as a wavefront aberration measuring device is installed, or in a facility where the device is not installed. The simulation device 2 may be placed on the Internet or the cloud. The simulation device 2 may include at least one physical server. The plurality of physical servers configured as the simulation device 2 are connected to each other via a communication network, so that they can communicate with each other. The physical server described above may include at least one virtual server. The virtual server may be a server on a cloud system (cloud server).

The simulation device 2 includes, for example, a storage device 4, an input device 5, an output device 6, and a processor 7. The processor 7 is an example of a processing unit. The output device 6 is an example of an output section.

The storage device 4 stores, for example, a program (software) for executing a viewing simulation, data processed by this program, and the like. Further, the storage device 4 stores information necessary for visual appearance simulation.

The storage device 4 includes, for example, nonvolatile memory such as ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive). The above-mentioned program may be provided by a computer-readable storage medium, or may be provided from an external device via a communication network. The provided program is stored in the storage device 4 and executed by the processor 7.

The input device 5 inputs information from the outside into the storage device 4 . The input device 5 is, for example, a communication interface for communicating with an external device. The communication network through which the communication interface communicates with the external device may be a transmission path for wired communication, a transmission path for wireless communication (for example, a wireless LAN), or a transmission path for wired communication and wireless communication. It may also be a combination of transmission paths.

The input device 5 may be an operating device that accepts input operations from the user. For example, upon receiving an input operation from a user, the operating device may store information corresponding to the received input operation in the storage device 4. For example, the operating device is a touch panel, a touch pad, a pointing device such as a mouse, a button, a switch, a motion-sensitive controller, a keyboard, a mouse, a gesture input device, or an audio input device (for example, a microphone). Note that the input device 5 may include either or both of the communication interface and the operating device.

The output device 6 is connected to the display device 3 wirelessly or by wire, and outputs the result of the viewing simulation to the display device 3.

The display device 3 displays various information. For example, the display device 3 is a monitor for a personal computer. However, the present invention is not limited thereto, and the display device 3 may be a display device (display) of a mobile device such as a mobile phone, a smartphone, or a tablet terminal. Further, the display device 3 may be a display of a VR headset. For example, the display device 3 may be installed in a facility such as an eyeglass store, or may be installed in a user's home. The display device 3 and the simulation device 2 may be installed in different facilities.

The processor 7 generates a simulation image showing how a specific scene looks by executing data generation processing and appearance simulation processing. The processor 7 is operable to execute a program stored in the storage device 4 and to execute operations described by the program, that is, data generation processing and appearance simulation processing. As an example, the processor 7 includes at least one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit).

The program may be provided by a computer-readable storage medium. The program is read from a computer-readable storage medium, installed in the storage device 4, which is also an example of a computer-readable storage medium, and executed by the processor 7. The program may be downloaded from an external device via a communication network.

Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable storage media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory). , Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disk Read Only Memory (CD-ROM), Digital Versatile Disk (DVD), Blu-ray (RTM) Disc, Memory Stick, Integrated circuit cards and the like may be included.

The functional units of the processor 7 of this embodiment will be explained using FIG. 2. FIG. 2 is a functional block diagram of the processor 7 according to this embodiment. The processor 7 includes a data generation section 10 and an image generation section 20, as shown in FIG. The data generation section 10 and the image generation section 20 are realized by the processor 7 executing the above programs stored in the storage device 4.

The data generation unit 10 executes data generation processing. The data generation unit 10 includes, for example, a model construction unit 11 and a data calculation unit 12.

The model construction unit 11 acquires eyeball information regarding the user's eyeballs. The eyeball information may be input into the simulation device 2 by operating an operating device, which is an example of the input device 5, or may be input into the simulation device 2 from an external device via a communication interface, which is an example of the input device 5. may be done. The eyeball information includes, for example, one or more of measurement data of a wavefront aberration measurement device, an unaided eye prescription, and a glasses prescription. Eyeball information differs for each user. In other words, the eyeball information is information that corresponds to the eyeballs of each individual.

The measurement data includes, for example, at least shape data of the anterior surface of the cornea and central corneal thickness. The shape data of the anterior surface of the cornea includes, for example, various data representing the shape of the anterior surface of the cornea (toric surface) (hereinafter referred to as "front surface toric data"), shape data of the free-form surface of the anterior surface of the cornea, and front wavefront data. The naked eye prescription is data obtained by a user's naked eye examination, and includes, for example, a spherical power S1, an astigmatic power C1, and an astigmatic direction Ax1. The eyeglass prescription is data obtained by an eye examination while the user is wearing eyeglasses, and includes, for example, a spherical power S2, an astigmatic power C2, and an astigmatic direction Ax2.

The model construction unit 11 constructs an eyeball model (hereinafter referred to as "individual eyeball model") that models the user's eyeballs based on information including the user's eyeball information. This individual eyeball model differs for each user. The individual eyeball model is, for example, a three-dimensional model. For example, in constructing an individual eyeball model, the model construction unit 11 first generates a three-dimensional standard eyeball model. The standard eyeball model is an eyeball model in which various eyeball parameters have standard values. For example, the standard eyeball model is the Gullstrand eye model. As an example, the model construction unit 11 constructs a three-dimensional structure and medium data of an eyeball model in an emmetropic state, such as a Gullstrand model eye, as construction of a standard eyeball model.

After generating the standard eyeball model, the model construction unit 11 updates the standard eyeball model based on the user's eyeball information to create an eyeball model that is as close to the user's eyeballs as possible (hereinafter referred to as "individual eyeball model"). Build. For example, if the eyeball information includes information that can be applied to the standard eyeball model, the model construction unit 11 applies that information to the standard eyeball model. Then, the model construction unit 11 constructs an individual eyeball model according to the user's eyeball information by adjusting one or more parameters in the standard eyeball model so as to match the naked-eye prescription or the spectacle prescription. The method for generating the individual eyeball model will be described below.

For example, after generating a standard eyeball model, the model construction unit 11 applies eyeball information to the standard eyeball model to generate an updated eyeball model in which various parameters of the standard eyeball model are updated with the eyeball information.

For example, if the eyeball information includes measurement data, the model construction unit 11 applies the measurement data, which is the shape data of the anterior surface of the cornea and the central corneal thickness, to the standard eyeball model. Note that the standard eyeball model to which the measurement data is applied is referred to as a "first eyeball model." The model construction unit 11 determines the axial length and posterior corneal toric parameter of the first eyeball model based on the naked eye prescription. The corneal posterior surface toric parameters are various parameters representing the shape of the posterior surface (toric surface) of the cornea. The posterior corneal toric parameters are, for example, the principal direction and the radius of curvature in the principal direction. The first eyeball model whose axial length and posterior corneal toric parameters have been determined is referred to as a "second eyeball model."

For example, when determining the ocular axial length, the model construction unit 11 sets the ocular axial length as the first variable parameter. Then, the model construction unit 11 performs known paraxial ray tracing from the fovea of the retina of the first eyeball model to the lens, pupil, and cornea in this order while changing the first variable parameter, and calculates the average refractive index of the first eyeball model. Iterative calculations are performed until P1t matches the average refractive power P1t of the prescription for the naked eye. The model construction unit 11 determines the first variable parameter when the average refractive index of the first eyeball model matches the average refractive power Plt as the axial length of the first eyeball model. Note that the average refractive power P1t is, for example, a value obtained by dividing the astigmatic power C1 of the naked eye prescription by 2 and adding it to the spherical power S1 of the naked eye prescription (=S1+(C1/2)).

For example, when determining the posterior corneal toric parameter, the model construction unit 11 sets the radius of curvature in the meridian direction of the posterior corneal surface, which is an example of the posterior corneal toric parameter, to a second variable for each cross-sectional path in the meridian direction of the cornea. Set as a parameter. The model construction unit 11 performs known paraxial ray tracing from the fovea of the retina to the lens, pupil, and cornea of the first eyeball model in this order, and the refractive power in each meridian direction matches the target refractive power P1m in each meridian direction. Repeat calculations until The model construction unit 11 determines the second variable parameter when the refractive power of the first eyeball model matches the target refractive power P1m as the posterior corneal toric parameter. Note that the target refractive power P1m is, for example, a value obtained by dividing the astigmatic power C1 of the naked eye prescription by sin ² θ and adding it to the spherical power S1 (=S1+C1·sin ² θ). θ is the angle from the X axis.

The model construction unit 11 asphericizes the front and back surfaces of the cornea in the second eyeball model. For example, the model construction unit 11 applies the shape data of the free-form surface of the anterior surface of the cornea to the second eyeball model, thereby making the anterior surface of the cornea aspherical. As an example, the shape data of the free-form surface of the anterior surface of the cornea is three-dimensional position data in the X direction, Y direction, and Z direction. The model construction unit 11 constructs a spline surface from this three-dimensional position data, and replaces the anterior surface of the cornea of the second eyeball model with a toric surface, which is an example of a free-form surface.

Further, the model construction unit 11 replaces the posterior corneal surface of the second eyeball model with a free-form surface so that the posterior corneal surface of the second eyeball model becomes wavefront data of the posterior corneal surface, which is one of the measurement data. Thereby, the model construction unit 11 aspherics the posterior surface of the cornea in the second eyeball model. The second eyeball model that has been asphericized is an example of an "updated eyeball model." Below, a method of replacing the posterior corneal surface of the second eyeball model with a free-form surface will be explained using FIG. 3. FIG. 3 is a diagram illustrating an example of a method of replacing the posterior corneal surface of the second eyeball model with a free-form surface. DOP is the cornea and is included in the ocular optical system.

First, the model construction unit 11 emits a large number of light rays in a conical shape from the fovea of the retina, for example, and calculates the difference between the optical distance OPL1 of the principal ray B1 and the optical distance OPL2 of each light ray B2 of the other light ray group. Find ΔOPL (FIG. 3(a)). The optical distance OPL1 is the optical path length that the emitted chief ray B1 travels to the measurement plane MP (intersection position Po1). The optical distance OPL2 is the optical path length that the emitted light ray B2 travels to the measurement plane MP (intersection point Po2). Note that in FIG. 3A, optical distance OPL2<optical distance OPL1.

The model construction unit 11 moves the intersection position of the ray B2 and the measurement plane MP in the ray direction (+Z direction) by a difference ΔOPL from the intersection position Po2 so that the optical distance OPL2 becomes the same as the optical distance OPL1. (FIG. 3(b)). That is, the model construction unit 11 constructs a wavefront W _cal (hereinafter referred to as a "calculated wavefront") that is a set of the same optical distances so that the optical distance OPL2 is the same as the optical distance OPL1. The calculated wavefront W _cal is a collection of points having the same optical distance as the optical distance OPL1.

After constructing the calculated wavefront W _cal , the model construction unit 11 calculates the difference ΔW between the wavefront data W _target at the corneal anterior apex position, which is one of the measurement data, and the calculated wavefront W _cal (FIG. 3(c)). . The model construction unit 11 calculates a sag movement amount Δd of the posterior surface of the cornea that increases the optical distance OPL2 by the difference ΔW (FIG. 3(d)). The sag movement amount Δd is expressed, for example, by the following equation. Note that the number of paths varies depending on the eyeball model. Here, n indicates a refractive index. d is the distance that light travels. Specifically, n ₀ is the refractive index between the retinal center and the lens cortex, and d ₀ is the distance between the retinal center and the lens cortex (point P1). n ₁ is the refractive index of the lens cortex on the retina side, and d ₁ is the distance that the light ray B2 travels through the lens cortex on the retina side (distance between points P1 and P2). n ₂ is the refractive index of the lens nucleus, and d ₂ is the distance that the ray B2 travels through the lens nucleus (distance between points P2 and P3). n ₃ is the refractive index of the lens cortex on the cornea side, and d ₃ is the distance that the ray B2 travels through the lens cortex on the cornea side (distance between points P3 and P4). n ₄ is the refractive index between the crystalline lens (LS) and the cornea (DOP), and d ₄ is the distance that the ray B2 travels between the crystalline lens (LS) and the cornea (DOP). n ₅ is the refractive index of the cornea (DOP), and d ₅ is the distance that the ray B2 travels through the cornea (DOP) (distance between points P5 and P6). n ₆ is the refractive index outside the cornea (DOP), and d ₆ is the distance that the ray B2 travels between the cornea (DOP) and the calculated wavefront W _cal (distance between the point P6 and the intersection position Po3). be.

The model construction unit 11 changes the shape of the posterior surface of the cornea based on the sag movement amount Δd to bring the calculated wavefront W _cal closer to the wavefront data W _target at the vertex position of the anterior surface of the cornea. The model construction unit 11 repeats the processing described in FIGS. 3A to 3D until the difference Δw becomes less than the allowable error.

In this way, when the eyeball information includes measurement data, the model construction unit 11 applies the measurement data to the standard eyeball model, and finally replaces the posterior corneal surface of the second eyeball model with a free-form surface. to generate an updated eyeball model. However, the eyeball information may not include measurement data. In this case, the model construction unit 11 updates the axial length and anterior corneal toric parameter of the standard eyeball model with reference to the eyeglass prescription. That is, when the eyeball information does not include measurement data, the model construction unit 11 adjusts each of the axial length and front corneal toric parameter of the standard eyeball model as variable parameters based on the glasses prescription. Generate an updated eyeball model with .

For example, when determining the ocular axial length when measurement data is not included, the model construction unit 11 sets the ocular axial length as the third variable parameter, and uses the standard eyeball model while changing the third variable parameter. A well-known paraxial ray tracing is performed in the order of the fovea of the retina, the crystalline lens, the pupil, the cornea, and the reference position in front of the cornea, until the average refractive index of the standard eye model matches the average refractive power P2t of the eyeglass prescription. Perform iterative calculations. The reference position is, for example, a position 12 mm forward from the cornea. The model construction unit 11 determines the third variable parameter when the average refractive index of the standard eyeball model matches the average refractive power P2t as the axial length of the standard eyeball model. Note that the average refractive power P2t is, for example, a value obtained by dividing the astigmatic power C2 of the eyeglass prescription by 2 and adding it to the spherical power S2 of the eyeglass prescription (=S2+(C2/2)).

For example, when determining the anterior corneal toric parameter, the model construction unit 11 sets the radius of curvature in the meridian direction of the anterior corneal surface as the fourth variable parameter for each cross-sectional path in each meridian direction of the cornea. The model construction unit 11 performs known paraxial ray tracing on the standard eyeball model in the order of the fovea of the retina, the crystalline lens, the pupil, the cornea, and the reference position, so that the refractive power in each meridian direction is determined by the target refractive power P2m in each meridian direction. Iterate calculations until it matches. The model construction unit 11 determines the fourth variable parameter when the refractive power of the standard eyeball model matches the target refractive power P2m as the posterior corneal toric parameter. Note that the target refractive power P2m is, for example, a value obtained by dividing the astigmatic power C2 of the glasses prescription by sin ² θ and adding it to the spherical power S2 (=S2+C2·sin ² θ).

The model construction unit 11 constructs the individual eyeball model by adjusting parameters of the lens of the third eyeball model (hereinafter referred to as "crystalline lens parameters") as variable parameters so that the accommodation power of the updated eyeball model reaches the target value. To construct. The crystalline lens parameters are, for example, the curvature of the crystalline lens (for example, the curvatures of the anterior and posterior surfaces of the crystalline lens) and the apex positions of the crystalline lens (for example, the apex positions of the anterior and posterior surfaces of the crystalline lens). Here, the target value of the adjustment force is not one but multiple. For example, in the range from the accommodation force during relaxation (minimum accommodation force) Pmin = 0 [D (diopters)] to the preset maximum accommodation force Pmax [D], a plurality of discrete levels (hereinafter referred to as " (referred to as "accommodation level") is assigned. That is, a plurality of adjustment levels are assigned within a range of Pmin or more and Pmax or less. The adjustment levels are, for example, 0.00D (Pmin), 1.00D, 2.00D, . . . , 10.00D. The accommodation level corresponds to the target value of the accommodation force.

The model construction unit 11 creates individual eyeball models for each accommodation level by performing a process of adjusting the crystalline lens parameters of the updated eyeball model for each accommodation level so that the accommodation power of the updated eyeball model becomes the accommodation level. This individual eyeball model for each accommodation level is stored in the storage device 4. Therefore, if 10 accommodation levels are set, 10 individual eyeball models are created for each user. As an example, individual eyeball models for each accommodation level are stored as a database in the storage device 4.

The data calculation unit 12 constructs the optical system 100 by arranging the user's individual eyeball model Me and the spectacle lens model Mg in virtual space. FIG. 4 is a schematic diagram of the optical system 100 according to this embodiment. The spectacle lens model Mg is constructed by defining two free-form surfaces at the front and rear of the spectacle lens, the center thickness of the lens, and the medium refractive index. Then, the data calculation unit 12 generates preliminary data using the constructed optical system 100. FIG. 5 is a schematic configuration diagram of the data calculation section 12 according to this embodiment. As shown in FIG. 5, the data calculation section 12 includes, for example, a distortion data calculation section 30, a wavefront calculation section 31, a confusion ellipse calculation section 32, a wavefront aberration calculation section 33, and a point spread function (PSF) calculation section. 34.

The distortion data calculation unit 30 calculates distortion caused by the spectacle lens (hereinafter referred to as "distortion displacement amount"). For example, when there is no spectacle lens, that is, in the case of the naked eye, a point (for example, a lattice point) in the line of sight direction is displaced to the position of the lattice point due to the refraction of light rays when viewed through the spectacle lens. The amount of displacement at this position corresponds to the amount of strain displacement described above.

For example, in calculating the amount of distortion displacement, the distortion data calculation unit 30 first sets an area of the viewing frustum 110 that includes the optical system 100 in virtual space. FIG. 6 is a diagram showing an example of the viewing frustum 110 according to the present embodiment. For example, the distortion data calculation unit 30 places a predetermined viewpoint on the individual eyeball model Me, and sets the range visible from that viewpoint as the viewing frustum 110. The distortion data calculation unit 30 generates lattice points for the viewing frustum 110. The viewing frustum 110 is an area formed by cutting a quadrangular pyramid into a near plane and a far plane.

As shown in FIG. 7A, the distortion data calculation unit 30 calculates the line of sight direction u when looking at a certain lattice point (hereinafter referred to as "specific lattice point") SP when there is no eyeglass lens model. Set ₀ as the initial value. The viewing direction u ₀ is expressed as follows. Pg is the position coordinate of a specific grid point. Rrc is the position coordinate of the center of rotation of the eyeball.

Next, as shown in FIG. 7A, the distortion data calculation unit 30 performs backward ray tracing in consideration of the eyeglass lens, that is, in a state where there is refraction due to the eyeglass lens, and performs backward ray tracing on the XY The coordinates (intersection coordinates) Pe of the intersection point IP1 with the plane are calculated. Backward ray tracing is a process of tracing light rays emitted from the inside of the eyeball model (for example, the fovea) toward the eyeglass lens model side (+Z direction side shown in FIG. 7). The distortion data calculation unit 30 determines the principal ray path toward the specific grid point SP through an iterative process of correcting the line-of-sight direction u ₀ based on the difference between the intersection point coordinate Pe and the position coordinate Pg of the specific grid point SP. Note that the corrected line-of-sight direction _u0 may be referred to as the line-of-sight direction u'. The viewing direction u' is specified by determining the principal ray path.

The distortion data calculation unit 30 calculates the coordinates Pv of the intersection point IP2 of the XY plane when there is no eyeglass lens and the specific grid point SP in the line of sight direction u' when a light ray heads toward the specific grid point SP with the eyeglass lens present. The difference ΔP between the position coordinate Pg and the position coordinate Pg is calculated as the amount of strain displacement. This distortion displacement amount calculation is performed for each grid point by setting each of the plurality of grid points as a specific grid point SP. In other words, the data on the amount of strain displacement is stored for each grid point. FIG. 8A shows N×M lattice points on the XY plane at a certain measurement distance (distance in the Z direction) and the coordinates of the lattice points. FIG. 8B shows an example of strain displacement amount data stored at each grid point on the XY plane at a certain measurement distance (distance in the Z direction). As shown in FIG. 8, the coordinates (x, y) of the lattice point, the coordinates (x', y') of the lattice point after displacement, and the amount of displacement (amount of distortion) Δx, Δy are stored for each lattice point. It is stored in 4. The coordinates and distortion amount subscripts distinguish each grid point. The distortion displacement amount data for each of these lattice points may be collectively referred to as "distortion vector array data."

The wavefront calculation unit 31 uses the optical system 100 to calculate the wavefront at the front surface of the spectacle lens model Mg. For example, the wavefront calculation unit 31 causes a large number of light rays to be emitted in a conical shape from the fovea of the retina of the individual eyeball model Me. The wavefront calculation unit 31 determines that the optical distance (OPL) of each ray is the OPL of the principal ray at the intersection position of the many emitted rays and the front surface of the cornea of the individual eyeball model Me or the front surface of the lens of the spectacle lens model Mg. The wavefront at the front surface of the eyeglass lens model Mg is calculated by fitting the distribution of points with the Zernike polynomial surface.

The confusion ellipse calculation unit 32 executes confusion ellipse calculation to calculate the confusion ellipse of the optical system 100. The confusion ellipse calculation unit 32 calculates an intersection area (confusion ellipse) between the wavefront calculated by the wavefront calculation by the wavefront calculation unit 31 and the plane in which the specific grid point exists. FIG. 9 is a diagram illustrating a method of calculating a confusion ellipse. By wavefront calculation, the two principal meridian directions (Tξ, Tη) and the curvatures (Kξ, Kη) of the directions at the corneal vertex or the front surface of the spectacle lens are determined (FIG. 9(a)). The confusion ellipse calculation unit 32 calculates the shape of an elliptic cone at the plane grid position in the traveling direction of the wavefront (vector N) by similarity calculation. The confusion ellipse calculation unit 32 rotates this ellipse shape so that the vector N coincides with the Z axis to find an intersection area, thereby generating shape data of the confusion ellipse at a specific grid point (hereinafter referred to as "confusion ellipse data"). ). The confusion ellipse data includes, for example, a radius aξ in one principal meridian direction, a radius aη in two principal meridian directions, an angle θξ in one principal meridian direction, and an angle θη in two principal meridian directions, as illustrated in FIG. 9B, for example. The confusion ellipse data is stored in the storage device 4 for each grid point, each accommodation level, and each pupil diameter level. For example, the confusion ellipse calculation unit 32 performs a process of calculating confusion ellipse data for each of a plurality of preset pupil diameters for each accommodation level. Then, the confusion ellipse calculation unit 32 performs this processing for each adjustment level for each grid point.

The wavefront aberration calculation unit 33 calculates the difference surface between the wavefront calculated by the wavefront calculation by the wavefront calculation unit 31 and the ideal wavefront as a wavefront aberration. The ideal wavefront is a wavefront on which all light rays converge to one point, and is a spherical surface whose center is the object position and whose radius is the object distance (described later). In other words, the ideal wavefront is a spherical surface whose radius is the distance between the position of the wavefront and the lattice points. Since the ideal wavefront differs depending on the object distance, the wavefront aberration calculation unit 33 calculates the wavefront aberration for each object distance that changes for each grid point.

The PSF calculation unit 34 calculates a point spread function (PSF) based on the wavefront aberration calculated by the wavefront aberration calculation unit 33. For example, the PSF calculation unit 34 calculates the PSF by Fourier transforming the wavefront aberration calculated by the wavefront aberration calculation unit 33. The PSF calculation unit 34 stores the calculated PSF in the storage device 4. In the Fourier transform, sampling is performed at a value less than or equal to the minimum pixel width of a specific scene image, so that the influence of the pixel level can be reflected.

Specifically, the PSF calculation unit 34 first converts the wavefront aberration W(x,y) into a pupil function P(x,y), and performs a Fourier transform on the pupil function P(x,y) to calculate the PSF. can be found. For example, the pupil function P(x,y) is calculated using the following formula.

When performing Fourier transform on the pupil function P(x,y), the PSF calculation unit 34 determines parameters related to the Fourier transform. The relationship between the sampling interval Δ of the pupil function P(x,y) and the region width L of the pupil function P(x,y) is Δ=L/r. Note that r is the number of samplings.

The relationship between the basic spatial frequency V _base of the point spread function and the region width L of the pupil function is V _base =1/L. The sampling number r is determined to be a power of 2 so that the basic spatial frequency V _base is equal to or less than the minimum pixel width of the specific scene image. The result of Fourier transformation of the pupil function P(x,y) is a complex value. Therefore, the PSF calculation unit 34 stores the square value (real number) of the complex amplitude amount in the storage device 4 as PSF data.

Here, the PSF data is prepared for each grid point, each accommodation level, and each pupil diameter level. For example, wavefront calculation, wavefront aberration calculation, and point spread function calculation are performed as a series of processes, and the series of processes is executed for each of a plurality of preset pupil diameters. This series of processing for each of the plurality of pupil diameters is performed for each accommodation level. Therefore, when 10 accommodation levels are set, a series of processes for each of a plurality of pupil diameters is performed at each of the 10 accommodation levels, and is executed 10 times in total. Furthermore, this processing for each adjustment level is performed for each grid point. That is, a process (hereinafter referred to as "lattice point processing") for determining a point spread function by setting an arbitrary lattice point as a specific lattice point is executed. Grid point processing is performed for each grid point. Furthermore, in the lattice point processing, the above-described series of processing corresponding to each of a plurality of pupil diameters is executed for each accommodation level.

The image generation unit 20 executes appearance simulation processing. FIG. 10 is a schematic configuration diagram of the image generation section according to this embodiment. As shown in FIG. 10, the image generation section 20 includes, for example, a scene image acquisition section 40, an information acquisition section 41, an image filter generation section 42, and an image processing section 43.

The scene image acquisition unit 40 acquires a specific scene image. This specific scene image is created using computer graphics. FIG. 11 is a diagram illustrating a specific scene image. The specific scene image includes, for example, one or more objects. For example, the scene image acquisition unit 40 creates a two-dimensional image in a straight-view state obtained by perspectively projecting a three-dimensional scene as a specific scene image by executing a drawing process using computer graphics. Further, the scene image acquisition unit 40 acquires distance data of each pixel of the specific scene image. The distance data for each pixel corresponds to depth value data (hereinafter referred to as "depth data") recorded in a memory area called a depth buffer. In other words, the scene image acquisition unit 40 performs CG rendering of a three-dimensional scene (for example, a color image) using an ideal camera, and holds a CG rendered projection image and depth data of each pixel of the image.

The information acquisition unit 41 acquires the user's gaze direction and pupil diameter. For example, the user is wearing an eye tracking device. The information acquisition unit 41 acquires the gaze direction and pupil diameter of the user from the gaze tracking device worn by the user. Note that the information acquisition unit 41 may acquire the user's gaze direction and pupil diameter from a source other than the gaze tracking device. For example, the information acquisition unit 41 may acquire the information with the viewing direction set directly in front. That is, the user's line of sight direction may be set in advance as directly ahead. Further, the pupil diameter of the user may be set to a predetermined value in advance as a fixed value. Further, the information acquisition unit 41 may change the pupil diameter of the user according to the brightness of a specific scene or the distance to the gazed object, instead of setting the pupil diameter to a fixed value. That is, the information acquisition unit 41 may acquire the pupil diameter of the user by determining the value of the pupil diameter according to the brightness of the specific scene or the distance to the gazed object.

The information acquisition unit 41 acquires the object distance in the user's line of sight direction. The object distance is the distance from the reference position of the individual eyeball model Me to the spatial position corresponding to a specific pixel in the scene image. For example, the information acquisition unit 41 emits a light beam from the reference position of the eyeball of the individual eyeball model Me toward the line of sight direction. The information acquisition unit 41 determines the shortest point of intersection between the light ray and the object by determining a collision between the light ray and the object in the specific scene image. When the information acquisition unit 41 determines the intersection between the light ray and the object, the information acquisition unit 41 determines the object distance by determining the length between the intersection and the reference position of the eyeball.

The image filter generation unit 42 generates an image processing filter based on information including PSF data stored in the storage device 4. For example, the image filter generation unit 42 generates a discrete value image processing filter that matches the pixel width of a specific scene image from a continuous point spread function. FIG. 12 is a schematic diagram of an image processing filter. The image processing filter in Figure 12 extracts a ray spread range of 2.852 mm x 2.852 mm from PSF data of a 20 mm x 20 mm area, and creates an image of 55 x 55 numerical matrices so that the physical pixel width is 0.051855 mm. This is an example of creating a processing filter. An example of a method for creating an image processing filter in the image filter generation section 42 will be described below. FIG. 13 is a diagram illustrating a method for creating an image processing filter.

The image filter generation unit 42 determines one adjustment level from a plurality of adjustment levels when creating an image processing filter. For example, the image filter generation unit 42 calculates the accommodation force (hereinafter referred to as “required accommodation force”) P _accom required for the individual eyeball model Me to view an object at the object distance acquired by the information acquisition unit 41. . P _accom is calculated by the following formula. d _obj is the object distance.

If this required adjustment force P _accom is greater than or equal to the maximum adjustment force, the required adjustment force P _accom is set as the maximum adjustment force. If the required adjustment force P _acom and the adjustment level do not match, the data of the adjustment levels surrounding the required adjustment force P _acom are linearly interpolated and used.

For example, PSF data is prepared for each grid point, each accommodation level, and each pupil diameter level. In this case, the image filter generation unit 42 creates PSF data of eight points surrounding the object position Pobj shown in FIG. 13(a). As shown in the equation below, each of these eight points has a PSF data matrix [Ii] of q×m elements (i=0, 1,..., 7), where q is the number of accommodation levels and m is the number of pupil diameters. Assigned. i distinguishes the position of each grid point. Here, a ₀ to a _q in equation (5) indicate each adjustment level, and b ₀ to b _m indicate each pupil diameter level. For example, the suffix of adjustment level a distinguishes each adjustment level, and represents, for example, the number of the adjustment level in the ascending order of adjustment levels. For example, the subscript of pupil diameter level b distinguishes each pupil diameter level, and represents, for example, the number of the pupil diameter level counting from the lowest pupil diameter level.

The image filter generation unit 42 creates a PSF data array [I'i] of the current adjustment level from the PSF data matrix [Ii] shown above. The PSF data array [I'i] is expressed, for example, by the following equation.

Each element of the PSF data array [I'i] obtains an interval [a _k・a _k+1 ] including the current adjustment amount a _cur from the adjustment levels a ₀ , a ₁ ,...,a _q , and Ii( Ii (a _cur , b _j ) is calculated from a _k , b j ) and Ii (a _k+1 , b _{j )} by interpolation. In the case of linear interpolation, t=(acur-a _k )/(a _k+1 -a _k ) and the calculation is performed using the following formula. k is any number from 1 to q. j is any number from 1 to m.

Note that if the current adjustment amount a _cur is greater than or equal to the maximum adjustment amount, the PSF data of the adjustment level a _q is used as is, as shown in the following equation.

For example, if there is an object 300 mm in front of the eyes, an adjustment amount of 3.33D is required. When the maximum adjustment amount is 4.00D and the point spread function data for adjustment amounts of 0.00D, 2.00D, and 4.00D are held as a ₀ , a ₁ , and a ₂ , the image filter generation unit 42 is obtained by interpolating the PSF data of a ₁ and a ₂ corresponding to 2.00D and 4.00D. As an example, if 2.00D PSF data is Ii(a ₁ , b _j ) and 4.00D PSF data is Ii(a ₂ , b _j ), then t=(3.33-2)/(4. 00-2.00)=1.33/2=0.665, the PSF data corresponding to the current adjustment amount is expressed by the following formula.

When the maximum adjustment amount is 1.50D and the PSF data of adjustment amounts of 0.00D, 1.00D, and 1.50D are held as a ₀ , a ₁ , and a ₂ , the PSF data of a ₂ can be calculated using the following formula. Use as shown.

Next, the image filter generation unit 42 obtains PSF data Ii (a _cur , b _cur ) corresponding to the current pupil diameter b _cur from the PSF data array [I'i] of the current accommodation level. _{For example} , _an interval [ _{b k} . b _k+1 ] is obtained, and PSF data Ii (a _cur , b _cur ) is calculated from Ii (a _cur , b k ) and Ii ( _{a cur} , b _{k+1 )} by interpolation. In the case of this linear interpolation, t=(b _cur -b _k )/(b _k+1 -b _k ) and is calculated using the following formula.

If the current pupil diameter b _cur is greater than or equal to the maximum pupil diameter, the PSF data of the pupil diameter level b _m is used as is, as shown in the following equation.

Note that the PSF data at the pupil diameter level b ₀ corresponding to a pupil diameter of 0.0 mm may hold point imaging data having values only at the center.

The image filter generation unit 42 creates an image processing filter for convolution calculation from the PSF data Ii (a _cur , b _cur ). When the pixel size of the display surface is W (width) x H (height) with object distance d _obj and viewing angles θh and θv, the physical size of one pixel is the length in the X direction Δpx and the length in the Y direction When Δpy, it is expressed by the following formula.

It is assumed that the pixels are adjusted with viewing angles θh and θv so that they form a square (pΔx=pΔy). PSF data I is a distribution of continuous quantities. Therefore, the image filter generation unit 42 integrates a region of pixel size from the distribution of PSF data, and uses the integrated value as the kernel value of the image processing filter. The kernel physical size of the image processing filter, which is the spread range of the light beam, is the size of the circumscribed square of the confusion ellipse calculated in advance by confusion ellipse calculation. At this time, the image filter generation unit 42 performs linear interpolation to obtain confusion ellipse data from a data group of eight grid points surrounding the target pixel spatial position.

The image filter generation unit 42 calculates the number of pixels n of the image processing filter from the circumscribed square size Q _EoC and the one pixel size pΔx using the formula shown below.

When the number n of pixels of the image processing filter is an even number, the image filter generation unit 42 adds 1 to the number n of pixels to make it an odd number. The image filter generation unit 42 generates an image processing filter Fij by area-integrating each region within the number of pixels n×n of the image processing filter and setting a filter value. FIG. 14 is a diagram illustrating the image processing filter Fij.

The image processing unit 43 performs distortion processing to distort the specific scene image in accordance with the amount of distortion displacement. The image processing unit 43 performs distortion processing on the specific scene image (FIG. 15(a)) based on information including the amount of distortion displacement. For example, the image processing unit 43 performs distortion processing on the specific scene image and its distance data using the vector array data of the amount of distortion displacement and the distance data of the specific scene image. FIG. 15 is a diagram illustrating distortion processing according to this embodiment. For example, as shown in FIG. 15, by applying the distortion displacement amount, the image processing unit 43 transforms the pixel data Cz (u-Δx, v-Δy) of the specific scene image into the pixel data Cz' (u , v). That is, the image processing unit 43 samples the color of the distortion destination pixel of the specific scene image. As a result, a specific scene image (FIG. 15(b)) to which distortion processing has been applied is generated.

Furthermore, by applying the distortion displacement amount, the image processing unit 43 also replaces the distance data z (u−Δx, v−Δy) with the distortion destination distance z′ (u, v). Note that the amount of distortion displacement changes depending on the object distance. Therefore, the image processing unit 43 multiplies the amount of distortion displacement per unit distance by the object distance. From this, the distortion displacement amount is also applied to the distance data in the distortion image.

The image processing unit 43 performs blur processing on the specific scene image subjected to distortion processing based on information including the image processing filter. As a result, a simulation image is generated. FIG. 16 is a diagram illustrating blur processing according to this embodiment. As shown in FIG. 16, for example, the image processing unit 43 generates a simulation image by performing a convolution operation on a specific scene image that has been subjected to distortion processing using an image processing filter according to distance data. . The following equation is an example of blur processing using convolution calculation. Note that S is a preset number, for example, 16. R indicates pixel data of a specific scene image that has been subjected to distortion processing.

The simulation image generated by the image processing unit 43 is displayed on the display device 3 by the output device 6 as a still image or a moving image. When displayed as a video, by performing processing to generate a simulation image for each frame of the video and displaying it on the display device 3, it is possible to display the appearance through the eyeglass lenses on the display device 3 as a video. can. The simulation image is a frame of a moving image. With this, it is possible to reproduce the appearance when the point of gaze is moved.

The flow of the visual appearance simulation according to this embodiment will be described below. This simulation can be roughly divided into three types of processing: individual eyeball model creation processing, preliminary data creation processing, and real-time processing. First, with reference to FIG. 17, an example of the flow of the individual eyeball model creation process will be described.

The model construction unit 11 generates a standard eyeball model (step S101). When the model construction unit 11 acquires eyeball information regarding the user's eyeballs from the outside, it determines whether measurement data is included in the acquired eyeball information (step S102). If the eyeball information includes measurement data, the model construction unit 11 applies the measurement data to the standard eyeball model (step S103). After performing the process in step S103, the model construction unit 11 determines the axial length of the standard eyeball model (first eyeball model) to which the measurement data is applied based on the naked eye prescription (step S104). Furthermore, after determining the axial length of the first eyeball model, the model construction unit 11 sets posterior corneal toric parameters based on the naked eye prescription (step S105).

The model construction unit 11 applies the shape data of the free-form surface of the anterior surface of the cornea to the first eye model (second eye model) in which the axial length and the posterior corneal toric parameter are set, so that the anterior surface of the cornea is not reconstructed. It is made into a spherical surface (step S106). In addition, the model construction unit 11 adjusts the posterior corneal surface of the second eyeball model to have a free-form surface shape so that it matches the wavefront data at the anterior corneal apex position, which is one of the measured data (step S107). . Thereby, the model construction unit 11 aspherics the posterior surface of the cornea in the second eyeball model.

In step S102, if the measurement data is not included in the eyeball information, the model construction unit 11 determines whether or not to give priority to the axial length (step S108). Note that, for example, whether or not to give priority to the axial length is set in advance and can be switched by the administrator of the simulation device 2 or the like. Further, the process of step S108 can be omitted. When the model construction unit 11 determines that priority is given to the axial length, the axial length is set as the third variable parameter, and the average refractive index of the standard eyeball model matches the average refractive power determined from the glasses prescription. The third variable parameter is adjusted so as to (step S109). After step S109 is executed, the process moves to step S110. When the model construction unit 11 determines that the axial length is not prioritized, the process proceeds to step S110. In step S110, the model construction unit 11 sets the corneal front surface toric parameter as the fourth variable parameter so that the refractive power in each meridian direction of the standard eyeball model matches the target refractive power in each meridian direction determined from the glasses prescription. The anterior corneal toric parameters are adjusted as follows (step S110).

In step S109, the model construction unit 11 determines whether the average refractive index of the standard eyeball model matches the average refractive power determined from the eyeglass prescription (step S111). When the model construction unit 11 determines that the average refractive index of the standard eyeball model matches the average refractive power determined from the eyeglass prescription, the model construction unit 11 determines the axial length as the third variable parameter at that time. If the model construction unit 11 determines that the average refractive index of the standard eyeball model does not match the average refractive power determined from the eyeglass prescription, the process proceeds to step S109.

Furthermore, in step S111, the model construction unit 11 determines whether the refractive power of the standard eyeball model in each meridian direction matches the target refractive power in each meridian direction determined from the eyeglass prescription. In step S111, if the refractive power in each meridian direction of the standard eyeball model matches the target refractive power in each meridian direction determined from the glasses prescription, the model construction unit 11 sets the fourth variable parameter at that time to the corneal Decide on the front toric parameters. If the model construction unit 11 determines in step S111 that the refractive power in each meridian direction of the standard eyeball model does not match the target refractive power in each meridian direction determined from the glasses prescription, the process proceeds to step S109. do.

The model construction unit 11 repeats the loop process A from step S113 to step S115 until it is completed at each of the plurality of adjustment levels (steps S112 to S116). In step S112, the model construction unit 11 performs backward ray tracing of the ray passing through the center of the pupil. In step S113, the model construction unit 11 sets the curvature of the crystalline lens and the apex position of the crystalline lens as variable parameters, and uses the variable parameters until the average power at the apex position of the cornea obtained by backward ray tracing matches the target accommodation power. Adjust. Thereby, the parameters of the crystalline lens are updated and the individual eyeball model Me is constructed. In step S114, the model construction unit 11 stores the individual eyeball model Me corresponding to each accommodation force in the storage device 4 as a database.

Next, an example of the flow of preliminary data creation processing will be described with reference to FIG. 18. The preliminary data creation process is executed after the individual eyeball model creation process. The data calculation unit 12 sets a region of the view frustum 110 in the virtual space, and generates a plurality of lattice points within the view frustum 110 (step S201). Then, the data calculation unit 12 repeats the loop processing B from step S203 to step S212 until it is completed at each of all grid points (steps S202 to S213). That is, all the grid points are sequentially set as specific grid points, and loop processing B is executed.

In step S203, the data calculation unit 12 determines the line-of-sight direction when the light ray heads toward a specific grid point when the eyeglass lens is present, and determines the ray optical path when the eyeglass lens is not present in the line-of-sight direction ( Step S203). Then, the data calculation unit 12 calculates the difference between the intersection of the light beam path and the XY plane without the spectacle lens and the specific grid point as the amount of distortion displacement (step S204). As a result, a strain displacement amount is assigned to each grid point.

The data calculation unit 12 repeats the loop processing C from step S206 to step S211 until it is completed at each of all adjustment levels (steps S205 to S212).

In step S206, the data calculation unit 12 repeats the loop processing D from step S207 to step S210 until it is completed for each of all pupil diameter levels.

In step S207, the data calculation unit 12 calculates each of the multiple light rays emitted from the fovea of the retina of the individual eyeball model Me at the intersection position of the front surface of the cornea of the individual eyeball model Me or the front surface of the lens of the spectacle lens model Mg. The wavefront at the front surface of the eyeglass lens model Mg is calculated by fitting the distribution of points where the OPL of the light ray is the OPL of the chief ray using a Zernike polynomial surface.

In step S208, the data calculation unit 12 calculates the shape data of the confusion ellipse by calculating the intersection area (confusion ellipse) between the wavefront calculated by the wavefront calculation in step S207 and the plane in which the specific grid point is located. . As a result, shape data of the confusion ellipse is assigned to each grid point for each accommodation level and each pupil diameter.

In step S209, the data calculation unit 12 calculates the difference surface between the wavefront obtained by the wavefront calculation in step S207 and the ideal wavefront at the specific grid point as a wavefront aberration. In step S210, the data calculation unit 12 calculates a point spread function based on the wavefront aberration calculated in step S209. As a result, a point spread function is assigned to each grid point for each accommodation level and each pupil diameter.

Next, an example of the flow of real-time processing will be described with reference to FIG. 19. Real-time processing is performed after preliminary data creation processing. In this real-time processing, for example, a simulation image is displayed on the display device 3 in real time according to the line-of-sight direction or pupil diameter from a line-of-sight tracking device worn by the user. Here, real time may include the communication time between the eye tracking device and the simulation device 2 and the processing time of the simulation device 2. For example, when changing a simulation image according to the user's line of sight or pupil diameter, the change in the user's line of sight or pupil diameter and the change in the simulation image do not need to occur at the same time; It may also include a time delay corresponding to the above processing time.

The image processing unit 43 repeats the loop processing E from step S302 to step S306 for each frame of the moving image (steps S301 to S316).

In step S302, the image processing unit 43 acquires a specific scene image using computer graphics. In step S303, the image processing unit 43 acquires the user's gaze direction and pupil diameter from the gaze tracking device worn by the user.

In step S304, the image processing unit 43 calculates the object distance, which is the length between the object in the specific scene image in the line-of-sight direction acquired in step S303 and the reference position of the eyeball. In step S305, the image processing unit 43 determines, from among the plurality of accommodation levels, an accommodation level corresponding to the accommodation power necessary for the individual eyeball model Me to view an object at the object distance determined in step S304.

In step S306, the image processing unit 43 generates an image processing filter for performing blur processing based on the point spread function corresponding to the adjustment level determined in step S305. In step S307, the image processing unit 43 secures an image storage area of the same size as the specific scene image, and executes loop processing F from step S308 to step S309 for each pixel of the specific scene image of the same size.

In step S308, the image processing unit 43 acquires distance data of the target pixel from the depth buffer. In step S309, the image processing unit 43 performs distortion processing on the specific scene image and distance data using the distance data acquired in step S308 and the distortion vector array data.

In step S311, the image processing unit 43 executes loop processing G from step S312 to step S313 for each pixel of the specific scene image (steps S311 to S314).

In step S312, the image processing unit 43 acquires distance data of the target pixel from the depth buffer. In step S313, the image processing unit 43 uses the distance data acquired in step S312 and the image processing filter to perform blurring processing on the specific scene image that has been subjected to the distortion processing. In step S315, the output device 6 outputs the specific scene image subjected to the blurring process to the display device 3 as a simulation image.

As described above, the simulation device 2 according to the present embodiment reproduces how a specific scene is viewed by the user through the spectacle lens. The simulation device 2 generates a simulation image showing how a specific scene looks based on at least eyeball information regarding the user's eyeballs and design information of the spectacle lens. The simulation device 2 outputs the generated simulation image to the display device 3. Therefore, the accuracy of simulation is improved.

In addition, in the simulation of the appearance, calculations may be performed by taking into account aberrations between the eyeball and the correction spectacle lens by ray tracing. However, the amount of calculation required for ray tracing may be enormous, and using ray tracing in real-time processing causes an increase in delay. In the simulation device 2 according to this embodiment, since ray tracing is not used for real-time processing, delays in real-time processing can be suppressed.

The simulation device 2 generates an individual eyeball model Me that models the user's eyeball based on information including eyeball information, and generates an image representing the appearance of the scene seen from the individual eyeball model Me through the recording eyeglass lens as a simulation image. It may be generated as

The simulation device 2 may construct an optical system 100 in a virtual space that includes an individual eyeball model Me that is different for each individual and a spectacle lens model Mg that models a spectacle lens. In this case, the simulation device 2 obtains a specific scene image generated by computer graphics by backward ray tracing in which the optical system 100 traces rays from inside the individual eyeball model Me toward the spectacle lens model Mg. A simulation image may be generated by processing based on the information. Distortion processing is an example of processing. Blur processing is an example of processing.

The simulation device 2 may calculate the amount of distortion displacement in the line of sight direction caused by the eyeglass lens and the wavefront aberration at the front surface of the eyeglass lens by backward ray tracing, and calculate the PSF from the wavefront aberration. The simulation device 2 may perform distortion processing according to the amount of distortion displacement and blurring processing based on information including PSF on the specific scene image.

The simulation device 2 may calculate the PSF from the wavefront aberration using Fourier transform. The simulation device 2 also generates an image processing filter for performing blur processing based on the PSF, and performs blur processing on the specific scene image after distortion processing based on information including the image processing filter. You may do so.

The simulation device 2 performs distortion processing based on the amount of distortion displacement and distance data of each pixel of the scene image, and creates a specific scene image after the distortion processing based on the image processing filter and its distance data. Blur processing may be applied to the image.

The simulation device 2 may acquire information on at least one of the user's gaze direction and pupil diameter from the gaze tracking device, and further use this information to generate a simulation image.

The simulation device 2 may output a simulation image generated for each frame to the display device 3, thereby causing the display device 3 to output the appearance of a specific scene as a moving image.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the range described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the embodiments described above. It is clear from the claims that forms with such changes or improvements may also be included within the technical scope of the present invention.

In the embodiment described above, the simulation device 2 is configured as one device, but the simulation device 2 is not limited to this. For example, the simulation device 2 may be composed of a plurality of devices. For example, the data generation section 10 and the image generation section 20 may be different devices. That is, for example, the process of creating an individual eyeball model and the preliminary data creation process may be executed by a first device, and the real-time processing may be executed by a second device different from the first device. In this case, the network between the first device and the second device may be wired, wireless, or both. Furthermore, the process of creating an individual eyeball model and the process of creating preliminary data may be performed by separate devices.

FIG. 20 is a diagram showing a configuration example of a simulation system 1A according to this embodiment. The simulation system 1A includes a data transmission device 200, a model generation device 300, a storage device 400, a preliminary data generation device 500, an image generation device 600, and a display device 3.

For example, a data transmitting device 200 is provided in an eyeglass store ES. The data transmission device 200 includes, for example, an acquisition section 210 and a transmission section 220. The acquisition unit 210 acquires the user's eyeball information from a measurement device such as a wavefront aberration measurement device installed at an eyeglass store ES, for example. The acquisition unit 210 may acquire the user's eyeball information from a storage medium such as a server installed in the eyeglass store ES or managed by the eyeglass store ES. The transmitter 220 transmits the eyeball information acquired by the acquirer 210 to the model generation device 300 that generates different eyeball models for each individual. Here, when transmitting the eyeball information to the model generation device 300, the transmitter 220 may add identification information for identifying the user to the eyeball information and transmit the eyeball information. This identification information is not particularly limited as long as it can identify an individual, and may be, for example, a name or an ID number. The transmitter 220 is connected to the model generation device 300 via the communication network NW. The communication network NW may be wired, wireless, or both.

The model generation device 300 is managed by, for example, an eyeglass manufacturer EM. The model generation device 300 has the functions of the model construction section 11 described above, and executes the creation process of the individual eyeball model Me illustrated in FIG. 17. The model generation device 300 generates an individual eyeball model Me for reproducing the way a specific scene is viewed by a user through a spectacle lens. The model generation device 300 generates individual eyeball models Me that are different for each individual by modeling the user's eyeballs based on information including the user's eyeball information. The model generation device 300 stores the generated individual eyeball model Me in the storage device 400. Note that the storage device 400 may be provided within the model generation device 300, may be provided in the preliminary data generation device 500, or may be provided in another device. Furthermore, the storage device 400 may be composed of two or more storage media. Note that the individual eyeball model Me created by the model generation device 300 may be stored in the storage device 400 for each user's identification information. That is, the individual eyeball model Me may be stored in the storage device 400 in association with the user's identification information.

The preliminary data generation device 500 is managed, for example, by eyeglass manufacturer EM. The preliminary data generation device 500 is connected to the communication network NW. The prior data generation device 500 generates prior data for generating a simulation image that reproduces the way a specific scene is viewed by a user through a spectacle lens. Specifically, the preliminary data generation device 500 includes an optical system 100 that includes a user's individual eyeball model Me that is modeled using the user's eyeball information, and a spectacle lens model Mg that models a spectacle lens. Generate preliminary data using . For example, the preliminary data generation device 500 reads the user's individual eyeball model Me from the storage device 400. User information may be input into the preliminary data generation device 500 from the outside. For example, identification information of eyeglass lenses is input to the preliminary data generation device 500. The preliminary data generation device 500 extracts the design information of the spectacle lens from the storage device of the preliminary data generation device 500 or an external storage device from the input identification information of the spectacle lens, and based on the extracted design information, specifies the design information of the spectacle lens. A lens model Mg is generated. The preliminary data generation device 500 has the function of the data calculation section 12. The prior data generation device 500 generates prior data by executing the prior data creation process illustrated in FIG. 18 based on the generated individual eyeball model Me and spectacle lens model Mg. Note that the preliminary data generation device 500 does not necessarily need to generate the spectacle lens model Mg, and may acquire the spectacle lens model Mg from an external device. The preliminary data may be stored in the storage device 400, or may be transmitted to the image generation device 600, which will be described later. Further, the preliminary data may be stored in the storage device 400 in association with the user's identification information.

Image generation device 600 is connected to communication network NW. The image generation device 600 may be provided at an eyeglass store ES, or may be managed by an eyeglass manufacturer EM. The image generation device 600 generates a simulation image showing how a specific scene is viewed by a user through a spectacle lens. For example, the image generation device 600 reads the user's prior data stored in the storage device 400, generates a simulation image based on the read prior data, and displays the generated simulation image on the display device 3. Note that the image generation device 600 may not read the user's prior data stored in the storage device 400, but may directly transmit the prior data from the prior data generation device 500. The image generation device 600 has the function of the image generation section 20, and executes the real-time processing illustrated in FIG. 19 based on prior data. For example, when the user's identification information is input, the image generation device 600 reads prior data associated with the input identification information from the storage device 400, and executes real-time processing based on the read prior data. You can.

The execution order of each process such as operation, procedure, step, stage, etc. in the apparatus, system, program, and method shown in the claims, specification, and drawings is specifically defined as "before", "prior to", etc. It is not explicitly stated. Furthermore, it should be noted that each process can be executed in any order unless the output of a previous process is used in a subsequent process. Even if the claims, specifications, and operational flows in the drawings are explained using "first," "next," etc. for convenience, this does not mean that it is essential to carry out the operations in this order. It's not a thing.

1, 1A Simulation system 2 Simulation device 3 Display device 4 Storage device 5 Input device 6 Output device 7 Processor 10 Data generation section 11 Model construction section 12 Data calculation section 20 Image generation section 100 Optical system 200 Data transmission device 300 Model generation device 400 Storage device 500 Prior data generation device 600 Image generation device Me Eyeball model Mg Eyeglass lens model

Claims (16)

A simulation device that reproduces the appearance of a specific scene seen by a user through spectacle lenses,
a processing unit that generates a simulation image showing how the scene is viewed based on at least eyeball information regarding the user's eyeballs and design information of the spectacle lens;
an output unit that outputs the simulation image generated by the processing unit to a display device;
A simulation device equipped with. The processing unit includes:
a data generation unit that generates an eyeball model that models the user's eyeballs based on information including the eyeball information;
an image generation unit that generates, as the simulation image, a scene image representing how the scene is seen from the eyeball model through the eyeglass lens;
The simulation device according to claim 1, comprising: The data generation unit constructs in a virtual space an optical system having the eyeball model that is different for each individual and a spectacle lens model that models the spectacle lens,
The image generation unit processes the scene image generated by computer graphics based on information obtained by backward ray tracing that traces a ray from inside the eyeball model toward the eyeglass lens model in the optical system. generating the simulation image by
The simulation device according to claim 2. The data generation unit calculates a distortion displacement amount in the line of sight direction caused by the eyeglass lens by the backward ray tracing and a wavefront aberration at the front surface of the eyeglass lens, and calculates a point spread function from the wavefront aberration,
The image generation unit generates the simulation image by performing distortion processing to transform the scene image according to the distortion displacement amount and blurring processing to the scene image based on information including the point spread function. do,
The simulation device according to claim 3. The data generation unit calculates the point spread function from the wavefront aberration using Fourier transform.
The simulation device according to claim 4. The image generation unit generates an image processing filter for performing the blurring process based on the point spread function, and includes the image processing filter for the scene image after the distortion process is performed. performing the blurring process based on the information;
The simulation device according to claim 5. The image generation unit includes:
The image generation unit performs the distortion processing based on the distortion displacement amount and distance data of each pixel of the scene image, and the distortion processing is performed based on the image processing filter and the distance data. performing the blurring process on the image of the scene after
The simulation device according to claim 6. The processing unit acquires information on at least one of the user's gaze direction and pupil diameter from the gaze tracking device, and further uses the information to generate the simulation image.
The simulation device according to claim 1. The simulation image is a frame of a video,
The output unit outputs the simulation image generated for each frame to the display device, thereby causing the display device to output the appearance of the scene as a moving image.
A simulation device according to any one of claims 1 to 8. an acquisition unit that acquires eyeball information of the user;
a transmission unit that transmits the eyeball information acquired by the acquisition unit to a model generation device that generates a different eyeball model for each individual;
A data transmitting device comprising: A model generation device that generates an eyeball model for reproducing the appearance of a specific scene seen by a user through a spectacle lens,
A model generation device that generates a different eyeball model for each individual by modeling the user's eyeball based on information including eyeball information of the user. A prior data generation device that generates prior data for generating a simulation image that reproduces the appearance of a specific scene seen by a user through a spectacle lens,
Generating the prior data using an optical system having an eyeball model of the user modeled using eyeball information of the user and a spectacle lens model modeling the spectacle lens;
Advance data generation device. A point spread function and a distortion displacement amount in the direction of the line of sight caused by the eyeglass lens are generated as the prior data by performing backward ray tracing that traces a ray from inside the eyeball model toward the eyeglass lens model. do,
The advance data generation device according to claim 12. An image generation device that generates a simulation image showing how a specific scene is viewed by a user through a spectacle lens,
An image generation device that generates the simulation image based on a priori data generated based on eyeball information regarding the user's eyeballs and design information of the eyeglass lens, and displays the generated simulation image on a display device. A simulation method that reproduces the appearance of a specific scene seen by a user through spectacle lenses,
generating a simulation image showing how the scene is viewed based on at least eyeball information regarding the user's eyeballs and design information of the spectacle lens;
outputting the generated simulation image to a display device;
simulation methods including. A simulation program that causes a computer to function as a simulation device for reproducing the appearance of a specific scene seen by a user through spectacle lenses,
The computer,
a processing unit that generates a simulation image showing how the scene is viewed based on at least eyeball information regarding the user's eyeballs and design information of the spectacle lens;
A simulation program that functions as an output unit that outputs the simulation image generated by the processing unit to a display device.

Images