CN114545624A - Near-eye display system and near-eye display method - Google Patents
Near-eye display system and near-eye display method Download PDFInfo
- Publication number
- CN114545624A CN114545624A CN202011340223.4A CN202011340223A CN114545624A CN 114545624 A CN114545624 A CN 114545624A CN 202011340223 A CN202011340223 A CN 202011340223A CN 114545624 A CN114545624 A CN 114545624A
- Authority
- CN
- China
- Prior art keywords
- light
- phased array
- eye
- optical
- optical phased
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000003287 optical effect Effects 0.000 claims abstract description 136
- 238000001514 detection method Methods 0.000 claims abstract description 56
- 238000012545 processing Methods 0.000 claims abstract description 44
- 230000004075 alteration Effects 0.000 claims abstract description 41
- 210000001525 retina Anatomy 0.000 claims abstract description 10
- 230000001105 regulatory effect Effects 0.000 claims abstract description 8
- 230000001276 controlling effect Effects 0.000 claims abstract description 6
- 238000010586 diagram Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 claims description 10
- 238000005070 sampling Methods 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 4
- 238000013461 design Methods 0.000 description 12
- 230000004424 eye movement Effects 0.000 description 12
- 230000008901 benefit Effects 0.000 description 8
- 238000012937 correction Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000000737 periodic effect Effects 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 201000009310 astigmatism Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/0816—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements
- G02B26/0833—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light by means of one or more reflecting elements the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
-
- G—PHYSICS
- G02—OPTICS
- G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
- G02F1/29—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the position or the direction of light beams, i.e. deflection
- G02F1/292—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the position or the direction of light beams, i.e. deflection by controlled diffraction or phased-array beam steering
Abstract
The application relates to the technical field of retina scanning display, and provides a near-eye display system and a near-eye display method. The near-eye display system comprises a scanning display device for scanning and projecting a light source to human eyes; the scanning display device comprises an optical phased array, a detection unit and a processing unit; the optical phased array is arranged on a light path between the light source and the human eyes; the detection unit is used for receiving light reflected by human eyes and processing the light to generate a two-dimensional light spot pattern; the processing unit is used for calculating the wave front difference of the light according to the two-dimensional facula image; the optical phased array is used for regulating and controlling own pixels according to the wave front difference so as to compensate the aberration of human eyes. The near-eye display system and the near-eye display method can detect and compensate eye aberration of different individuals, so that image quality viewed by different users can be optimized.
Description
Technical Field
The application relates to the technical field of retina scanning display, in particular to a near-eye display system and a near-eye display method.
Background
The Retinal Scan projection technology (RSD) is a novel Display technology that directly projects a Display image onto the retina of a user in a light beam scanning manner, and since the Display technology does not require a solid Display surface and only generates and modulates required pixel points, the Display technology has the characteristics of small volume and low power consumption, and is very suitable for near-eye Display scenes such as AR (Augmented Reality), VR (Virtual Reality), MR (Mixed Reality), and the like.
Taking the AR visual display technology as an example, the display system generally includes two parts, namely an optical engine and an optical coupler, wherein the optical engine is used for generating a virtual image, and the optical coupler is used for mapping the virtual image into an image of the real world to form an augmented reality display effect.
However, since the human eyes have a certain field angle, the different rotation positions of the eyes can lead to different light receiving capabilities, which can lead to the deterioration of the image quality of the visual display when the user rotates the eyes to certain positions when the AR/VR/MR product at the present stage is worn; in addition, since the eye aberrations (diopter, astigmatism, higher order aberrations, etc.) of different individuals are unique and non-repetitive, the same AR/VR/MR product will cause the image quality viewed by different users to be different.
Therefore, in order to ensure the wearing display experience of different users, it is necessary to perform eye movement real-time tracking detection and individual eye aberration detection for each individual eye and perform image quality correction correspondingly, so as to ensure real-time optimization of the image quality viewed by different users.
In view of the foregoing, there is a need in the art for a near-eye display system that can detect eye aberrations for different individuals and automatically compensate for the aberrations.
Disclosure of Invention
In view of the above, the present application provides a near-eye display system and a near-eye display method, so as to solve the problem that the near-eye display system in the prior art cannot detect and compensate eye aberrations of different individuals.
The near-eye display system comprises a scanning display device for scanning and projecting a light source to human eyes;
the scanning display device comprises an optical phased array, a detection unit and a processing unit;
the optical phased array is arranged on an optical path between the light source and the human eye;
the detection unit is used for receiving the reflected light of the human eyes and processing the reflected light to generate a two-dimensional light spot pattern;
the processing unit is used for calculating the wave front difference of the reflected light according to the two-dimensional light spot pattern,
and the area array phase modulator in the optical phased array is used for regulating and controlling the pixels of the optical phased array according to the wave front difference so as to compensate the aberration of the human eyes.
The near-eye display system can scan and project a light source to human eyes through the scanning display device when in work, a detection unit in the scanning display device can synchronously receive reflected light of the human eyes and process the reflected light to generate a two-dimensional light spot image, a processing unit can calculate the wave front difference of the reflected light according to the two-dimensional light spot image, the wave front difference comprises the spherical lens difference, the cylindrical lens difference, the high-order aberration, the aberration of the scanning display system and the like of the individual human eyes, and a planar array phase modulator in the optical phased array can synchronously regulate and control the pixels of the planar array phase modulator according to the wave front difference to perform wave front difference compensation, so that the wave front difference generated by the spherical lens difference, the cylindrical lens difference, the high-order aberration, the aberration of the scanning display system and the like of the human eyes is compensated, and the image quality observed by different users can be optimized.
The near-eye display system solves the problem that the near-eye display system in the prior art cannot detect and compensate eye aberrations of different individuals.
In one possible design, the optical phased array includes a phased region that is pixel-wise modulatable;
the detection unit is also used for detecting the light intensity information of the reflected light;
the processing unit is also used for calculating the current gazing point position of the human eyes according to the light intensity information;
the optical phased array is used for regulating and controlling pixels of the phased area corresponding to the current gazing point position in the area array phase modulator.
The light intensity information of the reflected light of the human eyes is detected through the detection unit, the current watching point position of the human eyes is calculated through the processing unit according to the light intensity information, and the optical phased array correspondingly regulates and controls pixels of a phased area corresponding to the watching point position in the area array phase modulator, so that the near-eye display system can capture real-time eye movement of the human eyes and can compensate wave front differences of different individual human eyes to correspondingly correct the image quality of the eye movement.
In one possible design, the scanning display device further comprises a MEMS scanning galvanometer;
the MEMS scanning galvanometer is arranged in a light path between the light source and the optical phased array and used for scanning and displaying the light source to the human eyes and reflecting retina reflected light of the human eyes to the detection unit.
The MEMS scanning galvanometer is arranged in a light path between the light source and the optical phased array, the function of scanning and displaying a target picture to human eyes can be realized through periodic vibration of the MEMS scanning galvanometer, and the MEMS scanning galvanometer has the advantages of small volume and low power consumption.
In one possible design, the scanning display device further includes an optical coupler;
the optical coupler is positioned in front of the human eye and arranged in an optical path between the optical phased array and the human eye, and is used for reflecting the light source to the human eye.
The optical coupler may be a spherical, aspherical, free-form optical element or holographic film.
In one possible design, the scanning display device further comprises relay optics;
the relay optics are disposed in an optical path between the optical phased array and the optical coupler.
In one possible design, the scanning display device further includes a storage unit;
the storage unit is respectively electrically connected with the detection unit and the processing unit and can store the two-dimensional light spot pattern generated by the processing of the detection unit.
Because the detection unit receives light reflected by different human eyes, processes the light to generate a two-dimensional light spot pattern with individual difference, and the two-dimensional light spot pattern generated by the processing of the detection unit is stored by the storage unit, the two-dimensional light spot pattern data which are stored and recorded can be used for user identification, equipment unlocking and the like.
In one possible design, the scanning display device further includes a light splitting element;
the light splitting element is arranged in a light path between the light source and the MEMS scanning galvanometer;
the light splitting element is capable of partially transmitting the light source and reflecting light reflected by the human eye to the detection unit.
The light splitting element can partially transmit and project the light source to the retina of human eyes and reflect light reflected by the human eyes to the detection unit, so that an incident light path of the light source can be overlapped and shared with a reflection light path of the reflected light of the human eyes, and the scanning display device can be more compact in structure and smaller in size.
In one possible design, the optical phased array is a reflective optical phased array.
The reflection type optical phased array can greatly change the light path directions of the MEMS scanning galvanometer and the relay optical piece, so that the positions and the orientations of the MEMS scanning galvanometer and the relay optical piece can be more flexibly arranged.
In one possible design, the optical phased array is a transmissive optical phased array.
The transmission type optical phased array can only change the light path direction of the MEMS scanning galvanometer and the relay optical piece in a small range, so that the MEMS scanning galvanometer and the relay optical piece can be arranged in a parallel or approximately parallel direction, and the scanning display device is more compact in structure.
In one possible design, the optical phased array has a plurality of the phased regions, and the plurality of phased regions are arrayed in the optical phased array.
The optical phased array is provided with a plurality of phased areas arranged in an array manner, so that the optical phased array can independently adjust pixels of one or more phased areas according to the watching point position of human eyes at a certain time, and the advantage of reducing the power consumption of the optical phased array is achieved.
In one possible design, the detection unit comprises an imaging element and a photosensor array;
the imaging element is used for focusing the light reflected by the human eye to the photosensitive sensor array;
the photosensitive sensor array can detect light intensity information of the light at different moments and process the light to generate a two-dimensional light spot pattern.
The imaging element can better focus light reflected by human eyes on the photosensitive sensor array, and the photosensitive sensor array can detect light intensity information of the light at different moments and can process the light to generate a two-dimensional light spot pattern.
In addition, the application also provides a near-to-eye display method, which comprises the following steps:
the optical phased array receives light rays emitted by the light source and scans and projects the light rays to human eyes;
the detection unit receives the reflected light of the human eyes and processes the reflected light to generate a two-dimensional light spot pattern;
the processing unit calculates the wavefront difference of the reflected light according to the two-dimensional light spot pattern;
and the optical phased array regulates and controls the pixels of the area array phase modulator according to the wave front difference so as to compensate the aberration of the human eye.
The near-eye display method can be suitable for the near-eye display system, the aberration of the human eye can be calculated by utilizing the wavefront difference of the reflected light of the human eye, the phase of the light source can be adjusted and controlled by projecting the optical phased array to the human eye from the light source, so that the effect of compensating the aberration of the human eye is achieved, and the problem that the near-eye display system in the prior art cannot detect and compensate the aberration of different individuals is solved.
In one possible design, the near-eye display method further includes:
the detection unit detects light intensity information of the reflected light;
the processing unit calculates the current fixation point position of the human eyes according to the light intensity information;
the optical phased array adjusting and controlling the pixels of the area array phase modulator according to the wave front difference so as to compensate the aberration of the human eye specifically comprises: the optical phased array regulates and controls pixels of a phased area corresponding to the current gazing point in the area array phase modulator so as to compensate aberration of the human eyes.
Therefore, the near-eye display method can also realize the functions of capturing real-time eye movement of human eyes and realizing eye movement image quality correction, and pixels in a local phased area in the optical phased array are correspondingly regulated and controlled only aiming at a human eye gazing point position at a certain moment, so that the power consumption of the optical phased array can be minimized.
In a possible design, the detecting unit detects light intensity information of the reflected light, specifically including: the detection unit detects light intensity information of the reflected light at different moments according to set integration time and sampling intervals;
the step of calculating the current fixation point of the human eye according to the light intensity information by the processing unit specifically includes: and the processing unit records the time t with the maximum light intensity in different times and calculates the current fixation point position of the human eyes according to the light path of the reflected light at the time t.
Because the scanning path and the time of the light source have known correlation, the processing unit can calculate the current fixation point position of the human eye in a reverse mode according to the light path of the reflected light so as to realize the function of capturing the real-time eye movement of the human eye, and the method has the advantages of simple algorithm and small data operand.
Additional features and advantages of embodiments of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of embodiments of the present application. The objectives and other advantages of the embodiments of the application will be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
Fig. 1 is a schematic diagram of a first structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a second structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating a third structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram illustrating a fourth structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram illustrating a fifth structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram illustrating a sixth structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram illustrating a seventh structure of a near-eye display system according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of an optical phased array of a near-eye display system provided in an embodiment of the present application;
fig. 9 is a first flowchart of a near-eye display method according to an embodiment of the present disclosure;
fig. 10 is a second flowchart of a near-eye display method according to an embodiment of the present disclosure;
fig. 11 is a third flowchart of a near-eye display method according to an embodiment of the present disclosure.
Reference numerals:
1-a light source;
2-scanning the display device;
21-an optical phased array;
211-phased regions;
22-a detection unit;
221-an imaging element;
222-a photosensitive sensor array;
23-a processing unit;
24-MEMS scanning galvanometer;
25-an optical coupler;
26-relay optics;
27-a storage unit;
28-light splitting element.
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Detailed Description
For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.
It should be understood that the embodiments described are only some of the embodiments of the present application, and not all of the embodiments.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of association that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
It should be noted that the terms "upper", "lower", "left", "right", and the like used in the embodiments of the present application are described in terms of the angles shown in the drawings, and should not be construed as limiting the embodiments of the present application. In addition, in this context, it will also be understood that when an element is referred to as being "on" or "under" another element, it can be directly on "or" under "the other element or be indirectly on" or "under" the other element via an intermediate element.
Specific embodiments of the near-eye display system provided in the embodiments of the present application will be described below.
The embodiment of the application provides a near-eye display system, which comprises a scanning display device 2, a scanning light source device and a scanning light source device, wherein the scanning display device is used for scanning and projecting a light source 1 to human eyes 3; wherein the scanning display device 2 comprises an optical phased array 21, a detection unit 22, and a processing unit 23; the optical phased array 21 is arranged on an optical path between the light source 1 and the human eyes 3; the detection unit 22 is used for receiving the light reflected by the human eyes 3 and processing the light to generate a two-dimensional light spot pattern; the processing unit 23 is configured to calculate a wavefront difference of the light according to the two-dimensional speckle pattern; the area array phase modulator in the optical phased array 21 is used to adjust its own pixels according to the wavefront difference to compensate the aberration of the human eye 3.
Specifically, as shown in fig. 1, when the near-eye display system works, the light source 1 can be projected to the human eye 3 by scanning through the scanning display device 2, the detection unit 22 in the scanning display device 2 can synchronously receive the light reflected by the human eye 3 and process the light to generate a two-dimensional spot diagram, the processing unit 23 can calculate the wavefront difference of the light according to the two-dimensional spot diagram, the wavefront difference at this time includes the spherical aberration, the cylindrical aberration, the higher order aberration, the aberration of the scanning display system, and the like of the individual human eye, and the optical phased array 21 can further adjust and control the pixels of the area array phase modulator according to the wavefront difference to compensate the wavefront difference caused by the spherical aberration, the cylindrical aberration, the higher order aberration, and the like of the human eye 3, so that the image quality viewed by different users can be optimized.
The near-eye display system solves the problem that the near-eye display system in the prior art cannot detect and compensate eye aberrations of different individuals.
In an alternative of the present embodiment, the optical phased array 21 includes a phased region 211 that can be modulated on a pixel-by-pixel basis; the detection unit 22 is also used for detecting the light intensity information of the reflected light of the human eyes 3; the processing unit 23 is further configured to calculate a current gazing point of the human eye 3 according to the light intensity information, and the optical phased array 21 is configured to regulate and control pixels of the phased region 211 corresponding to the current gazing point.
Since the scanning display device 2 can scan and project the light source 1 to the human eye 3 according to a certain periodic frequency, the scanning path and time of the light source 1 have known correlation, and the human eye 3 has the strongest reflection effect on the light when the light is irradiated on the retina of the human eye 3 vertically through the pupil.
Therefore, when the scanning display device 2 scans and projects the light source 1 to the human eye 3 at a fixed scanning frequency, the detection unit 22 collects the light intensity of the light reflected by the human eye at different moments and establishes the relationship between the light intensity and the time at a certain integration time and sampling interval. If the light intensity of the reflected light reaches the peak value at the time t, the processing unit 23 may calculate the current field angle of the human eye 3 by reverse estimation according to the scanning path relationship of the scanning display device 2 at the time t, that is, capture the current gazing point of the human eye 3, so that the optical phased array 21 may implement the functions of tracking the eye movement in real time and correspondingly making image quality correction according to the pixel of the phased region 211 corresponding to the current gazing point of the human eye 3.
The light intensity information of the reflected light of the human eyes 3 is detected by the detection unit 22, the current gazing point position of the human eyes 3 is calculated by the processing unit 23 according to the light intensity information, and the pixels of the corresponding phased area 211 in the near-eye display system are correspondingly regulated and controlled by the optical phased array 21, so that the near-eye display system can compensate the wavefront difference of different individual human eyes 3 while capturing the real-time eye movement of the human eyes 3, and correspondingly make eye movement image quality correction, and the image quality seen by the human eyes 3 at any gazing position can be optimized.
In an alternative of this embodiment, the scanning display device 2 further comprises a MEMS scanning galvanometer 24; the MEMS scanning galvanometer 24 is disposed in the optical path between the light source 1 and the optical phased array 21, and is configured to scan and display the light source 1 to the human eye 3, and reflect the retina reflected light of the human eye 3 to the detection unit 22.
Specifically, as shown in fig. 2, the MEMS scanning galvanometer 24 is disposed in the optical path between the light source 1 and the optical phased array 21, and the scanning galvanometer 24 is vibrated along the x-axis and the y-axis of the space synchronously and periodically to realize the function of scanning the target image to the human eye 3, and in addition, the scanning frequency of the MEMS scanning galvanometer 24 can reach as high as 15KHz to 30KHz, which has the advantages of high scanning efficiency, small volume and low power consumption.
Of course, some other devices with scanning function may be provided or the light source 1 may be directly set as scanning light, so that the function of projecting light to the human eye 3 in periodic scanning may be achieved.
In an alternative of this embodiment, the scanning display device 2 further comprises an optocoupler 25; the optical coupler 25 is located in front of the human eye 3 and is disposed in an optical path between the optical phased array 21 and the human eye 3, for reflecting the light source 1 to the human eye 3.
Specifically, as shown in fig. 3, an optical coupler 25, which may be a spherical surface, an aspherical surface, a free-form optical element, or a holographic film, is provided in the optical path between the optical phased array 21 and the human eye 3.
In addition, when the near-eye display system is applied to head-mounted products such as AR/VR/MR, the optical coupler 25 can be arranged on the lens, so that the head-mounted products such as AR/VR/MR can be more compact in structure and better in image quality display effect.
In an alternative of this embodiment, the scanning display device 2 further comprises relay optics 26; relay optics 26 are disposed in the optical path between optical phased array 21 and optical coupler 25.
Specifically, as shown in fig. 4, by providing the relay optical element 26, the light source 1 can be balanced, enhanced, modulated, etc. by the relay optical element 26 before the light source 1 projects onto the optical coupler 25, so that the image display effect of the light source 1 finally projected onto the human eye 3 is better.
In an alternative of the present embodiment, the scanning display device 2 further includes a storage unit 27; the storage unit 27 is electrically connected to the detection unit 22 and the processing unit 23, respectively, and can store the two-dimensional light spot pattern generated by the processing of the detection unit 22.
Specifically, as shown in fig. 5, since the spherical aberration, the cylindrical aberration, and the high-order aberration of different human eyes 3 have uniqueness, the detection unit 22 receives the light reflected by different human eyes 3 and processes the light to generate a specific individual difference of the two-dimensional spot diagram, and the two-dimensional spot diagram generated by the detection unit 22 is stored in the storage unit 27, so that the two-dimensional spot diagram data recorded in the storage unit can be used for user identification, device unlocking, and the like.
In an alternative of this embodiment, the scanning display device 2 further includes a light splitting element 28; the light splitting element 28 is arranged in the light path between the light source 1 and the MEMS scanning galvanometer 24; the light splitting element 28 is capable of partially transmitting the light source 1 and of reflecting light reflected by the human eye 3 towards the detection unit 22.
Specifically, as shown in fig. 6, the light splitting element 28 is specifically but not limited to a half-transparent mirror, and by providing the light splitting element 28, when the light source 1 partially transmits through the light splitting element 28 and projects onto the retina of the human eye 3, and reflects the light reflected by the human eye 3 to the detection unit 22, the incident light path of the light source 1 and the reflected light path of the reflected light of the human eye can be overlapped and shared, so that the scanning display device 2 can be more compact in structure and smaller in size.
In an alternative of this embodiment, the optical phased array 21 is a reflective optical phased array.
Specifically, as shown in fig. 6, the optical phased array 21 is configured as a reflection-type optical phased array, and in this case, the optical phased array 21 can change the optical path directions of the MEMS scanning galvanometer 24 and the relay optical element 26 to a relatively large extent, so that the positions and orientations of the MEMS scanning galvanometer 24 and the relay optical element 26 can be more flexibly arranged.
In an alternative of this embodiment, the optical phased array 21 is a transmissive optical phased array.
Specifically, as shown in fig. 7, the optical phased array 21 is configured as a transmissive optical phased array, and the optical phased array 21 can only change the optical path directions of the MEMS scanning galvanometer 24 and the relay optical element 26 by a small amount, so that the MEMS scanning galvanometer 24 and the relay optical element 26 can be arranged in a parallel or approximately parallel direction, so as to make the structure of the scanning display device 2 more compact.
In an alternative of this embodiment, the optical phased array 21 has a plurality of phased regions 211, and the plurality of phased regions 211 are arrayed in the optical phased array 21, and each phased region 211 can change its own pixel under the control of the processing unit 23.
Specifically, as shown in fig. 8, 3 × 3 rectangular array arrangement phased regions 211 are exemplarily set in the optical phased array 21, and may respectively correspond to a head-up point, a head-down point, a left-view point, a right-view point, a head-up point, a head-down point, and a right-down point of the human eye 3, so that when the human eye 3 is located at one of the points at a certain moment, the optical phased array 21 may regulate and control pixels of the corresponding phased region 211 to perform image quality correction.
The adjacent phased regions 211 may be partially overlapped as shown in fig. 8, and the respective phased regions 211 may be arranged in a closely spaced manner without gaps.
In addition, the optical phased array 21 may be provided with more phased regions 211, for example, 4 × 4, 5 × 5, 6 × 6, and the like, and the optical phased array 21 may adjust and control pixels of a plurality of adjacent phased regions 211 at a time, so that the optical phased array 21 can perform image quality correction in a wider range.
The above-described manner of installing the optical phased array 21 enables the optical phased array 21 to individually adjust the pixels of a certain phased region 211 only for the fixation point of the human eye 3 at a certain time, and has an advantage of low power consumption.
In an alternative of this embodiment, the detection unit 22 includes an imaging element 221 and a photosensor array 222; the imaging element 221 is used for focusing the light reflected by the human eye 3 on the photosensor array 222; the photosensor array 222 can detect light intensity information at different moments of the light and process the light to generate a two-dimensional light spot pattern.
Specifically, as shown in any one of fig. 1 to 7, the imaging element 221 may be configured as a microlens array, and the photosensor array 222 may be configured as a high-speed CCD array (Charge Coupled Device) or a CMOS array (Complementary Metal Oxide Semiconductor).
The imaging element 221 can better focus the reflected light of the human eye 3 on the photosensitive sensor array 222, and the photosensitive sensor array 222 can detect the light intensity information of the reflected light at different moments and process the light to generate a two-dimensional light spot pattern.
It should be noted that the light source 1 may be specifically configured as a display light source and a detection light source, and the display light source may be specifically red, green and blue light, and the detection light source may be specifically infrared light. Therefore, various colored pictures can be projected and displayed to the human eyes 3 through the display light source, and because the human eyes 3 cannot see infrared light, the photosensitive cells on the retina cannot absorb the infrared light, the detection light source is set to be the infrared light, so that the human eyes can better reflect the detection light source, and the detection unit 22 can receive the reflected light of the human eyes 3 conveniently.
In addition, the embodiment of the application also provides a first near-eye display method, and the near-eye display method can be adapted to the near-eye display system.
Specifically, as shown in fig. 9, the near-eye display method includes:
step S1, the optical phased array 21 receives the light emitted by the light source 1 and scans and projects the light to the human eye 3;
step S2, the detection unit 22 receives the reflected light of the human eye 3 and processes the reflected light to generate a two-dimensional light spot pattern;
step S3 — the processing unit 23 calculates the wavefront difference of the reflected light according to the two-dimensional light spot pattern;
step S4 — the optical phased array 21 adjusts and controls the pixels of the area array phase modulator according to the wavefront difference to compensate the aberration of the human eye 3.
Through the above steps S2, S3 and S4, the aberration of the human eye can be calculated through the wavefront difference of the reflected light of the human eye 3, and the pixels of the area array phase modulator can be regulated and controlled through the optical phased array 21, so as to achieve the effect of compensating the aberration of the human eye, thereby solving the problem that the near-eye display system in the prior art cannot detect and compensate the aberration of the eyes of different individuals.
In an alternative of this embodiment, the near-eye display method further includes:
step S21 — the detection unit 22 detects the light intensity information of the reflected light;
step S31, the processing unit 23 calculates the current fixation point of the human eye 3 according to the light intensity information;
the optical phased array 21 adjusts and controls the pixels of the area array phase modulator according to the wave front difference so as to compensate the aberration of the human eye 3, and specifically includes: the optical phased array 21 modulates the pixels of the phased region 211 corresponding to the current gazing point in the area array phase modulator.
Specifically, as shown in fig. 10, step S21 may be executed in synchronization with step S2, step S31 may be executed in synchronization with step S3, and step S21, step S31, and step S4 may capture real-time eye movement of the human eye 3, and may adjust and control the pixels of the local phased region 211 of the area array phase modulator in the optical phased array 21 only in correspondence with the gaze point of the human eye 3 at a certain time, thereby achieving eye movement image quality correction and minimizing power consumption of the optical phased array 21.
In an alternative of this embodiment, the detecting unit 22 detects the light intensity information of the reflected light specifically includes: the detection unit 22 detects light intensity information of the reflected light at different moments with set integration time and sampling intervals; the calculating, by the processing unit 23, the current gazing point of the human eye 3 according to the light intensity information specifically includes: the processing unit 23 records the time t with the maximum light intensity in different times, and calculates the current gazing point of the human eye according to the light path of the reflected light at the time t.
Specifically, as shown in fig. 11, the sampling interval of the detection unit 22 should be much smaller than the scanning projection period of the light source 1, for example, the sampling interval of the detection unit 22 may be set to be one tenth of the scanning projection period of the light source 1, so that light intensity information of ten times can be collected in one scanning projection period of the light source 1, and the processing unit 23 can obtain the time t with the maximum light intensity by comparing the ten light intensity information, because the scanning path and the time of the light source 1 have known correlation, so that the processing unit 23 can calculate the current gaze point of the human eye by back-stepping according to the light path of the reflected light, so that the function of capturing the real-time eye movement of the human eye 3 can be realized, and the advantages of simple algorithm and small data computation amount are obtained.
It is noted that a portion of this patent application contains material which is subject to copyright protection. The copyright owner reserves the copyright rights whatsoever, except for making copies of the patent files or recorded patent document contents of the patent office.
The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (14)
1. A near-eye display system comprising a scanning display device (2) for scanning a light source (1) for projection onto a human eye (3);
wherein the scanning display device (2) comprises an optical phased array (21), a detection unit (22), and a processing unit (23);
the optical phased array (21) is arranged in the optical path between the light source (1) and the human eye (3);
the detection unit (22) is used for receiving the reflected light of the human eyes (3) and processing the reflected light to generate a two-dimensional light spot pattern;
the processing unit (23) is used for calculating the wavefront difference of the reflected light according to the two-dimensional light spot diagram;
the area array phase modulator in the optical phased array (21) is used for regulating and controlling own pixels according to the wave front difference so as to compensate the aberration of the human eye (3).
2. A near-eye display system as claimed in claim 1 wherein the optical phased array (21) comprises a phased region (211) that is pixel-wise modulatable;
the detection unit (22) is also used for detecting the light intensity information of the reflected light;
the processing unit (23) is further configured to calculate a current gazing point of the human eye (3) according to the light intensity information;
the optical phased array (21) is used for regulating and controlling pixels of the phased area (211) corresponding to the current gazing point position in the area array phase modulator.
3. A near-to-eye display system as claimed in claim 2 wherein the scanning display device (2) further comprises a MEMS scanning galvanometer (24);
the MEMS scanning galvanometer (24) is arranged in a light path between the light source (1) and the optical phased array (21) and is used for scanning and displaying the light source (1) to the human eye (3) and reflecting retina reflected light of the human eye (3) to the detection unit (22).
4. A near-eye display system as claimed in claim 3 wherein the scanning display device (2) further comprises an optical coupler (25);
the optical coupler (25) is located in front of the human eye (3) and is arranged in the optical path between the optical phased array (21) and the human eye (3) for reflecting the light source (1) to the human eye (3).
5. The near-to-eye display system of claim 4 wherein the scanning display device (2) further comprises relay optics (26);
the relay optics (26) are arranged in the optical path between the optical phased array (21) and the optical coupler (25).
6. The near-eye display system of claim 5 wherein the scanning display device (2) further comprises a storage unit (27);
the storage unit (27) is electrically connected with the detection unit (22) and the processing unit (23) respectively, and can store the two-dimensional light spot pattern generated by the processing of the detection unit (22).
7. A near-to-eye display system as claimed in claim 3 wherein the scanning display device (2) further comprises a light splitting element (28);
the light splitting element (28) is arranged in an optical path between the light source (1) and the MEMS scanning galvanometer (24);
the light-splitting element (28) is capable of partially transmitting the light source (1) and of reflecting light reflected by the human eye (3) towards the detection unit (22).
8. A near-to-eye display system according to claim 2 wherein the optical phased array (21) is a reflective optical phased array.
9. A near-eye display system as claimed in claim 2 wherein the optical phased array (21) is a transmissive optical phased array.
10. The near-eye display system of claim 2 wherein the optical phased array (21) has a plurality of the phased regions (211), and the plurality of phased regions (211) are arrayed in the optical phased array (21).
11. The near-eye display system of claim 2 wherein the detection unit (22) comprises an imaging element (221) and a photosensor array (222);
the imaging element (221) is used for focusing the light reflected by the human eye (3) to the photosensitive sensor array (222);
the photosensor array (222) can detect light intensity information of the light at different moments and process the light to generate a two-dimensional light spot pattern.
12. A near-eye display method, comprising:
the optical phased array (21) receives light rays emitted by the light source (1) and scans and projects the light rays to human eyes (3);
the detection unit (22) receives the reflected light of the human eyes (3) and processes the reflected light to generate a two-dimensional light spot diagram;
the processing unit (23) calculates the wavefront difference of the reflected light according to the two-dimensional light spot diagram;
the optical phased array (21) modulates the pixels of the area array phase modulator according to the wave front difference to compensate the aberration of the human eye (3).
13. The near-eye display method of claim 12, further comprising:
the detection unit (22) detects light intensity information of the reflected light;
the processing unit (23) calculates the current gazing point position of the human eye (3) according to the light intensity information;
the optical phased array (21) adjusts and controls the pixels of the area array phase modulator according to the wave front difference so as to compensate the aberration of the human eye (3) and specifically comprises the following steps: the optical phased array (21) modulates pixels of a phased region (211) of the area array phase modulator corresponding to the current gaze point location to compensate for aberrations of the human eye (3).
14. The near-eye display method of claim 12, wherein the detecting unit (22) detects light intensity information of the reflected light, specifically comprising: the detection unit (22) detects light intensity information of the reflected light at different moments with set integration time and sampling intervals;
the processing unit (23) calculates the current gazing point of the human eye (3) according to the light intensity information, and specifically comprises: the processing unit (23) records the time t with the maximum light intensity in different times, and calculates the current fixation point position of the human eyes according to the light path of the reflected light at the time t.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011340223.4A CN114545624A (en) | 2020-11-25 | 2020-11-25 | Near-eye display system and near-eye display method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011340223.4A CN114545624A (en) | 2020-11-25 | 2020-11-25 | Near-eye display system and near-eye display method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114545624A true CN114545624A (en) | 2022-05-27 |
Family
ID=81659222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011340223.4A Pending CN114545624A (en) | 2020-11-25 | 2020-11-25 | Near-eye display system and near-eye display method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114545624A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140313484A1 (en) * | 2013-03-15 | 2014-10-23 | Amo Groningen B.V. | Wavefront generation for ophthalmic applications |
| CN105934902A (en) * | 2013-11-27 | 2016-09-07 | 奇跃公司 | Virtual and augmented reality systems and methods |
| CN106291923A (en) * | 2016-10-25 | 2017-01-04 | 吉林大学 | A kind of projection helmet objective designing method of corrective ophthalmic optical aberration |
| US20170039907A1 (en) * | 2015-08-03 | 2017-02-09 | Oculus Vr, Llc | Display with a Tunable Mask for Augmented Reality |
| CN108061968A (en) * | 2018-01-05 | 2018-05-22 | 京东方科技集团股份有限公司 | A kind of head-up display device and display image correcting method |
| CN109565554A (en) * | 2017-06-15 | 2019-04-02 | 谷歌有限责任公司 | The near-eye display with frame rendering for eyes characterization based on reflection wavefront analysis |
| US10317680B1 (en) * | 2017-11-09 | 2019-06-11 | Facebook Technologies, Llc | Optical aberration correction based on user eye position in head mounted displays |
| US10656707B1 (en) * | 2018-04-10 | 2020-05-19 | Facebook Technologies, Llc | Wavefront sensing in a head mounted display |
-
2020
- 2020-11-25 CN CN202011340223.4A patent/CN114545624A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140313484A1 (en) * | 2013-03-15 | 2014-10-23 | Amo Groningen B.V. | Wavefront generation for ophthalmic applications |
| US20180271361A1 (en) * | 2013-03-15 | 2018-09-27 | Amo Groningen B.V. | Wavefront generation for ophthalmic applications |
| CN105934902A (en) * | 2013-11-27 | 2016-09-07 | 奇跃公司 | Virtual and augmented reality systems and methods |
| CN107193126A (en) * | 2013-11-27 | 2017-09-22 | 奇跃公司 | Virtual and augmented reality System and method for |
| US20170039907A1 (en) * | 2015-08-03 | 2017-02-09 | Oculus Vr, Llc | Display with a Tunable Mask for Augmented Reality |
| CN106291923A (en) * | 2016-10-25 | 2017-01-04 | 吉林大学 | A kind of projection helmet objective designing method of corrective ophthalmic optical aberration |
| CN109565554A (en) * | 2017-06-15 | 2019-04-02 | 谷歌有限责任公司 | The near-eye display with frame rendering for eyes characterization based on reflection wavefront analysis |
| US10317680B1 (en) * | 2017-11-09 | 2019-06-11 | Facebook Technologies, Llc | Optical aberration correction based on user eye position in head mounted displays |
| CN108061968A (en) * | 2018-01-05 | 2018-05-22 | 京东方科技集团股份有限公司 | A kind of head-up display device and display image correcting method |
| US10656707B1 (en) * | 2018-04-10 | 2020-05-19 | Facebook Technologies, Llc | Wavefront sensing in a head mounted display |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108474939B (en) | Image projection system | |
| EP3607381B1 (en) | Wide field of view scanning display | |
| CN108107579B (en) | Holographic light field large-view-field large-exit-pupil near-to-eye display system based on spatial light modulator | |
| EP3610318B1 (en) | Foveated mems scanning display | |
| US20180084232A1 (en) | Optical See-Through Head Worn Display | |
| US6201517B1 (en) | Stereoscopic image display apparatus | |
| JP5156875B1 (en) | Display device | |
| US5467104A (en) | Virtual retinal display | |
| EP2212735B1 (en) | Pupil scan apparatus | |
| CN110914786A (en) | Method and system for registration between an external scene and a virtual image | |
| US20120280956A1 (en) | Display apparatus | |
| CN109633905B (en) | Multi-focal-plane display system and apparatus | |
| JP2017515161A (en) | Eye projection system | |
| EP3729182B1 (en) | Eye tracking for head-worn display | |
| WO2021096447A1 (en) | Augmented reality head-up display with steerable eyebox | |
| EP3514606A1 (en) | Eye tracking for head-worn display | |
| CN112189161A (en) | Light field display adapted to human vision | |
| US20240019710A1 (en) | Optical system for a virtual retinal scan display and method for projecting image contents onto a retina | |
| US20230077212A1 (en) | Display apparatus, system, and method | |
| CN114545624A (en) | Near-eye display system and near-eye display method | |
| CN216485801U (en) | Optical imaging system, image display device and augmented reality display equipment | |
| CN107272319A (en) | Projection arrangement and image projecting method | |
| US20230080420A1 (en) | Display apparatus | |
| JP2000214408A (en) | Picture display device | |
| JP3444577B2 (en) | Display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220527 |
|
| RJ01 | Rejection of invention patent application after publication |