CN117311514A - VR interactive display integrated screen structure with eye tracking - Google Patents
VR interactive display integrated screen structure with eye tracking Download PDFInfo
- Publication number
- CN117311514A CN117311514A CN202311458585.7A CN202311458585A CN117311514A CN 117311514 A CN117311514 A CN 117311514A CN 202311458585 A CN202311458585 A CN 202311458585A CN 117311514 A CN117311514 A CN 117311514A
- Authority
- CN
- China
- Prior art keywords
- infrared
- light source
- screen
- substrate
- integrated screen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 21
- 210000001508 eye Anatomy 0.000 claims abstract description 32
- 238000003491 array Methods 0.000 claims abstract description 4
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 4
- 239000000758 substrate Substances 0.000 claims description 32
- 230000001681 protective effect Effects 0.000 claims description 6
- 239000011521 glass Substances 0.000 claims description 5
- 239000012780 transparent material Substances 0.000 claims description 3
- 238000011161 development Methods 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 5
- 230000003287 optical effect Effects 0.000 abstract description 2
- 230000004424 eye movement Effects 0.000 description 14
- 238000001514 detection method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000034 method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention discloses a VR interactive display integrated screen structure with eye tracking, relates to VR devices, and provides a scheme for solving the problem of heavy and huge in the prior art. The integrated screen comprises an integrated screen and a transparent protection plate arranged on the front side of the integrated screen; the integrated screen is formed by arranging a plurality of screen unit arrays; each screen unit is integrated with three color-developing light sources, an infrared photodiode and an infrared light source matched with the infrared photodiode; the infrared photodiode is used for receiving light information of light emitted by an infrared light source after eyeball reflection. The device has the advantages that the camera shooting and the light source are integrated integrally, and the angle full coverage is realized; the optical path is simple, and the infrared light source serving as the transmitting end, the eyes of the reflecting end and the center of the infrared photodiode of the receiving end almost keep the same straight line; the power consumption is low, and the whole device is light and small.
Description
Technical Field
The invention relates to a VR screen structure, in particular to a VR interactive display integrated screen structure with eye movement tracking.
Background
Eye tracking is a mature technique widely used to study human cognition. Video-based systems have become mainstream using computer vision techniques. Since devices such as smartphones are equipped with small high quality cameras, they now have a lightweight and convenient eye tracking system, and can even be integrated into VR headset or portable glasses. These devices are capable of monitoring eye movement quickly and accurately, providing a large amount of data.
Unlike conventional eye tracking, VR eye tracking can more easily define regions of interest and track the points in time at which these regions are viewed. Virtual reality eye tracking has the advantage of providing more natural stimulus, more realistic movements, a controlled environment, and controlled data collection.
The prior VR eye tracking scheme mainly adopts a separated structure, wherein the eye tracking is realized by the cooperative work of an infrared LED and a camera, and meanwhile, a tracking module is separated from a VR screen. In this design, the VR screen is responsible for presenting the virtual world, while the infrared camera is used to receive the image of the infrared LED light source reflected by the eye surface, and transmit the captured data to the controller. The controller reacts to the data by adjusting the display on the VR screen to match the eye movements of the user in the VR. Most of the existing eye tracking adopts a mode of combining a front infrared camera and an infrared light source (LED or other radiation source) module: the infrared light source emits invisible infrared light to irradiate the surface of the eyes of the user, part of the light is reflected or scattered by the surface of the eyes, the infrared camera in the VR glasses captures the infrared light reflected by the surface of the eyes of the user, the captured data can be processed by a complex algorithm to determine the position and the point of gaze of the eyes of the user, and the result of eye movement tracking is fed back to the system.
VR eye tracking based on this approach has several technical problems: the front-facing camera is separated from the light source, so that the eye movement detection angle is limited by the volume of the VR device, and theoretically, the larger the detection angle is, the larger the volume of the VR device can be used for accommodating more front-facing cameras; the light path from the light source to the camera through the eye is complex, and the infrared light needs to be processed by a complex algorithm; a plurality of infrared front cameras are generally needed to track eye movement, so that the power consumption is high; the display screen is separated from the eye tracking module, and the device is heavy and huge.
Disclosure of Invention
The present invention is directed to a VR interactive display integrated screen structure with eye tracking to solve the above-mentioned problems of the prior art.
The VR interactive display integrated screen structure with the eye movement tracking comprises an integrated screen and a transparent protective plate arranged on the front side of the integrated screen; the integrated screen is formed by arranging a plurality of screen unit arrays; each screen unit is integrated with three color-developing light sources, an infrared photodiode and an infrared light source matched with the infrared photodiode; the infrared photodiode is used for receiving light information of light emitted by an infrared light source after eyeball reflection.
The screen unit comprises a first substrate and a second substrate; the second substrate is arranged on the front side of the first substrate and is made of transparent materials; the infrared photodiode and the infrared light source are disposed in a first substrate, and the three color-developing light sources are disposed in a second substrate.
The screen unit comprises a first substrate, and the infrared photodiode, the infrared light source and the three color development light sources are all arranged in the first substrate.
The transparent protective plate is a glass plate. The infrared light source is an infrared LED.
The VR interactive display integrated screen structure with eye tracking has the advantages that the integration of shooting and light sources is realized, and the full angle coverage is realized; the optical path is simple, and the infrared light source serving as the transmitting end, the eyes of the reflecting end and the center of the infrared photodiode of the receiving end almost keep the same straight line; the power consumption is low, and the whole device is light and small.
Drawings
Fig. 1 is a schematic structural diagram of a VR interactive display integrated screen structure according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a VR interactive display integrated screen structure according to a second embodiment of the present invention.
Reference numerals:
10-integrated screen, 11-first substrate, 12-second substrate, 13-color light source, 14-infrared photodiode, 15-infrared light source;
20-a transparent protective plate.
Detailed Description
Example 1
As shown in fig. 1, the VR interactive display integrated screen structure with eye tracking according to the present invention includes an integrated screen 10 and a transparent protective plate 20 disposed at a front side of the integrated screen 10; the integrated screen 10 is formed by arranging a plurality of screen unit arrays; each screen unit is integrated with three color-developing light sources 13, one infrared photodiode 14 and an infrared light source 15 matched with the infrared photodiode 14; the infrared photodiode 14 is configured to receive light information of light emitted from the infrared light source 15 after the light is reflected by the eyeball. The screen unit includes a first substrate 11 and a second substrate 12; the second substrate 12 is disposed on the front side of the first substrate 11, and the second substrate 12 is made of transparent material; an infrared photodiode 14 and an infrared light source 15 are provided in the first substrate 11, and the three color-developing light sources 13 are provided in the second substrate 12. The transparent protection plate 20 is a glass plate. The infrared light source 15 is an infrared LED. The three color light sources 13 are respectively used for displaying three kinds of visible light of RGB, and play a role of screen display.
The planar elements of the array arrangement are such that the infrared photodiodes 14 likewise have the form of an array arrangement, all of the infrared photodiodes 14 constituting a camera function. And each infrared photodiode 14 is integrated very compactly from the corresponding adapted infrared light source 15, so that the infrared emission end and the infrared receiving end of the whole VR interactive display integrated screen structure are maintained in almost parallel simple light paths. The structural advantage of this embodiment is that the infrared photodiode 14 and the chromogenic light source 13 are manufactured by processing on different substrates, and even the array of infrared photodiodes 14 can be manufactured using a CMOS substrate, the compatibility is stronger, and the circuit design is simpler by the separate substrate.
The basic principle is as follows: the infrared light source 15 irradiates eyes of a person, the reflected infrared light is received by the adaptive infrared photodiode 14, and the captured image is calculated by the algorithm module to judge eye movement information, so that the eye movement information is fed back to the integrated screen 10 to display corresponding contents.
The VR device with the VR interactive display integrated screen structure specifically comprises the following steps:
1. the integrated screen 10 displays: initially, the integrated screen 10 displays the desired image or interface.
2. The controller determines an eye tracking mode: the controller monitors eye movement of the user and determines whether to turn on the eye movement tracking mode. The eye movement and gaze point may be detected by an algorithm.
3. Turning on an eye tracking mode: if the controller decides to turn on the eye tracking mode, the associated hardware and software operations are triggered.
4. The infrared light source 15 emits light: once the eye-tracking mode is on, the infrared light source 15 begins to illuminate for eye-tracking.
5. The array of infrared photodiodes 14 captures the signal: in eye tracking mode, the user's eye will reflect infrared light and the array of infrared photodiodes 14 will capture these reflected signals. This array forms a two-dimensional coordinate matrix.
6. Gaze point detection: by analyzing the infrared light signals received by the infrared light photodiodes 14 of different coordinates, the controller can determine the gaze point of the user, i.e. the position at which the user is looking.
7. Interactive feedback: the controller may take different interactions depending on the gaze point of the user. For example, the user gazes at the button "back to home" position on the screen, and the system sets up a gazing time of up to three seconds, the controller will trigger a feedback action.
8. Feedback to the integrated screen 10: after the controller calculates the gaze point of the user, it transmits feedback information to the integrated screen 10 to trigger a corresponding image or interface change.
9. Interactive display: the integrated screen 10 displays a corresponding image or performs an interactive operation, such as returning to a home page or zooming in a partial image, according to feedback from the controller.
Integrated screen 10, continuous tracking: the eye movement tracking mode will continuously track the eye movement of the user in order to achieve a real-time interactive display.
Example two
As shown in fig. 2, the main difference from the first embodiment is that the screen unit includes a first substrate 11, and the infrared photodiode 14, the infrared light source 15, and three color-developing light sources 13 are all disposed in the first substrate 11. The structural advantage of this embodiment is that the cost is lower by processing and manufacturing the infrared photodiode 14 and the chromogenic light source 13 on the same substrate.
It will be apparent to those skilled in the art from this disclosure that various other changes and modifications can be made which are within the scope of the invention as defined in the appended claims.
Claims (5)
1. VR interactive display integrated screen structure with eye tracking, characterized in that it comprises an integrated screen (10) and a transparent protective plate (20) arranged on the front side of said integrated screen (10);
the integrated screen (10) is formed by arranging a plurality of screen unit arrays;
each screen unit is integrated with three color development light sources (13), an infrared photodiode (14) and an infrared light source (15) matched with the infrared photodiode (14); the infrared photodiode (14) is used for receiving light information of light emitted by the infrared light source (15) after the eyeball reflects.
2. The VR interactive display integrated screen structure with eye-tracking of claim 1, characterized in that said screen unit comprises a first substrate (11) and a second substrate (12); the second substrate (12) is arranged on the front side of the first substrate (11), and the second substrate (12) is made of transparent materials; an infrared photodiode (14) and an infrared light source (15) are arranged in the first substrate (11), and the three color-developing light sources (13) are arranged in the second substrate (12).
3. The VR interactive display integrated screen structure with eye tracking of claim 1, characterized in that said screen unit comprises a first substrate (11), said infrared photodiode (14), infrared light source (15) and three chromogenic light sources (13) are all arranged in the first substrate (11).
4. The VR interactive display integrated screen structure with eye tracking of claim 1 wherein said transparent protective plate (20) is a glass plate.
5. The VR interactive display integrated screen structure with eye-tracking of claim 1, characterized in that said infrared light source (15) is an infrared LED.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311458585.7A CN117311514A (en) | 2023-11-03 | 2023-11-03 | VR interactive display integrated screen structure with eye tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311458585.7A CN117311514A (en) | 2023-11-03 | 2023-11-03 | VR interactive display integrated screen structure with eye tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117311514A true CN117311514A (en) | 2023-12-29 |
Family
ID=89237386
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311458585.7A Pending CN117311514A (en) | 2023-11-03 | 2023-11-03 | VR interactive display integrated screen structure with eye tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117311514A (en) |
-
2023
- 2023-11-03 CN CN202311458585.7A patent/CN117311514A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10319266B1 (en) | Display panel with non-visible light detection | |
| US8398239B2 (en) | Wearable eye tracking system | |
| US10656731B2 (en) | Peripheral device for head-mounted display | |
| EP3065623B1 (en) | Optical eye tracking | |
| CN106663183B (en) | Eye tracking and user response detection | |
| US8933912B2 (en) | Touch sensitive user interface with three dimensional input sensor | |
| US9360935B2 (en) | Integrated bi-sensing optical structure for head mounted display | |
| CN108700933A (en) | It can carry out the wearable device of eye tracks | |
| KR102669768B1 (en) | Event camera system for pupil detection and eye tracking | |
| US10663729B2 (en) | Peripheral device for head-mounted display | |
| US10007331B2 (en) | Wearable intelligent systems and interaction methods thereof | |
| WO2016118369A1 (en) | Applying real world scale to virtual content | |
| Bohme et al. | Remote eye tracking: State of the art and directions for future development | |
| JPWO2015012280A1 (en) | Gaze detection device | |
| WO2009121638A1 (en) | System and a method for tracking input devices on lc-displays | |
| US10551500B2 (en) | Infrared optical element for proximity sensor system | |
| CN112639687A (en) | Eye tracking using reverse biased light emitting diode devices | |
| WO2024114258A1 (en) | Eye movement tracking apparatus and method | |
| CN116601541A (en) | System with supplemental illumination | |
| CN117311514A (en) | VR interactive display integrated screen structure with eye tracking | |
| US20130120361A1 (en) | Spatial 3d interactive instrument | |
| CN114660806A (en) | Eye tracking optical device, head-mounted display equipment and eye tracking method | |
| KR20220148922A (en) | Geospatial image surfacing and selection | |
| CN111700586A (en) | Eye movement tracking device and electronic device using same | |
| US20240201502A1 (en) | Wearable terminal device, program, and display method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |