CN113325585B - Display device, wearable display equipment and method for determining position of fixation point - Google Patents
Display device, wearable display equipment and method for determining position of fixation point Download PDFInfo
- Publication number
- CN113325585B CN113325585B CN202110586822.2A CN202110586822A CN113325585B CN 113325585 B CN113325585 B CN 113325585B CN 202110586822 A CN202110586822 A CN 202110586822A CN 113325585 B CN113325585 B CN 113325585B
- Authority
- CN
- China
- Prior art keywords
- display device
- light
- photoelectric sensing
- area
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
Abstract
The application discloses a display device, wearable display equipment and a method for determining the position of a fixation point, and relates to the technical field of virtual reality. Because the display device has higher processing efficiency on the electric signals sent by each photoelectric sensing assembly, the display device can quickly determine the position of the fixation point of the eyes of the user on the display panel based on the electric signals sent by each photoelectric sensing assembly, thereby improving the image display efficiency of the display panel and having higher refresh rate of the display panel.
Description
Technical Field
The application relates to the technical field of virtual reality, in particular to a display device, a wearable display device and a method for determining a position of a fixation point.
Background
A Virtual Reality (VR) device refers to a device that is capable of creating a virtual environment by means of a displayed image, and immersing a user in the virtual environment.
In the related art, a VR device includes a display panel, a camera, a processor, and a driving circuit. The camera is used for shooting the eye image of the user. The processor determines the position of a fixation point of a user on the display panel according to the eye image, and locally renders the display image required to be displayed according to the position of the fixation point. The driving circuit drives the display panel to display based on the received partially rendered display image sent by the processor. Because the processor can only locally render the region where the position of the gazing point in the display image is located, and does not need to globally render the display image, the load of the processor can be reduced, and the display effect of the display panel can be ensured.
However, the processor in the related art is inefficient in determining the position of the gazing point according to the eye image captured by the camera, which in turn results in inefficient display of the display panel.
Disclosure of Invention
The embodiment of the application provides a display device, wearable display equipment and a method for determining the position of a fixation point. The problem that the efficiency of determining the position of the fixation point is low in the prior art can be solved, and the technical scheme is as follows:
in one aspect, there is provided a display device including:
the display panel is provided with a display area and a peripheral area surrounding the display area, and the peripheral area is provided with a plurality of strip-shaped detection areas;
a plurality of photoelectric sensing assemblies positioned in each detection area;
the optical structure is positioned on one side, far away from the display panel, of the photoelectric sensing assemblies, and is provided with a light shielding area and a plurality of light transmission areas in one-to-one correspondence with the detection areas, and the orthographic projection of each light transmission area on the display panel is positioned in the corresponding detection area;
each photoelectric sensing assembly is used for receiving an optical signal which is reflected by the eyes of the user and penetrates through a light-transmitting area in the optical structure, and converting the optical signal into an electric signal.
Optionally, the shape of the light-transmitting region is circular.
Optionally, the display device further includes: a first lens located within the light-transmitting region.
Optionally, the light-transmitting region is slit-shaped.
Optionally, the plurality of strip-shaped detection areas include: a first detection region extending in a first direction and a second detection region extending in a second direction, the first direction intersecting the second direction;
the extending direction of the light-transmitting area corresponding to the first detection area is intersected with the extending direction of the light-transmitting area corresponding to the second detection area.
Optionally, the display device further includes: a lenticular lens located within the light transmissive region.
Optionally, the first direction is perpendicular to the second direction; the plurality of strip-shaped detection areas include: two of the first detection areas and two of the second detection areas; the two first areas are arranged along the second direction and are respectively positioned at two sides of the display area; the two second areas are arranged along the first direction and are respectively positioned on two sides of the display area.
Optionally, the extending directions of the two light-transmitting regions corresponding to the two first detection regions are parallel, and the extending directions of the two light-transmitting regions corresponding to the two second detection regions are parallel.
Optionally, in the length direction of each detection region, the maximum width of the light-transmitting region corresponding to the detection region is in a range from 10 micrometers to 100 micrometers.
Optionally, the plurality of photoelectric sensing assemblies each have a strip-shaped photosensitive area, and an extending direction of the detection area is perpendicular to an extending direction of the photosensitive area of the photoelectric sensing assembly located in the detection area.
Optionally, a vertical distance T between the optical structure and the photoelectric sensing assembly satisfies: t = dxl s /L e Said D represents the vertical distance between the user's eye and the optical structure, said L s Width of photosensitive area representing the photoelectric sensing assembly, L e Representing the minimum distance the user's eye can determine to move.
Optionally, the display device further includes: the processing circuit is connected with each photoelectric sensing assembly and used for determining the position of the gazing point of the eyes of the user on the display panel based on the magnitude of the signal value of the electric signal sent by each photoelectric sensing assembly and the position of at least one photoelectric sensing assembly.
Optionally, the display device further includes: a control circuit; each of the photoelectric sensing assemblies includes: a switching transistor and a photodiode;
wherein the photodiode includes: the pixel electrode, the P-type material layer, the photosensitive material layer, the N-type material layer and the common electrode are sequentially stacked along the direction far away from the substrate;
the first pole of the switch transistor is electrically connected with the pixel electrode, the second pole of the switch transistor is electrically connected with the processing circuit, the control pole of the switch transistor is electrically connected with the control circuit, and the control circuit is used for controlling the on and off of the switch transistor.
In another aspect, a wearable display apparatus is provided, the wearable display apparatus including: the display device according to the above aspect, a second lens located on a display side of the display device, and a plurality of light emitting elements located at edges of the lens;
wherein the emitting directions of the plurality of light emitting elements are deviated from the display device.
In another aspect, there is provided a method for determining a position of a gaze point, the method being applied to the display device of the above aspect, the method including: receiving a light signal reflected by a user's eye;
converting the optical signal into an electrical signal;
and determining the position of the fixation point of the eyes of the user on the display panel based on the magnitude of the signal value of the electric signal and the position of the at least one photoelectric sensing assembly.
The beneficial effects that technical scheme that this application embodiment brought include at least:
the embodiment of the application provides a display device, a wearable display device and a method for determining the position of a fixation point, wherein the display device has higher processing efficiency on electric signals sent by each photoelectric sensing assembly, so that the display device can determine the position of the fixation point of eyes of a user on a display panel based on the electric signals sent by each photoelectric sensing assembly, the image display efficiency of the display panel can be further improved, and the refresh rate of the display panel is higher. The optical structure with the light transmission area is arranged on the side, far away from the display panel, of the photoelectric sensing assembly, so that the photoelectric sensing assembly can only receive light reflected by the eyes of a user, and the accuracy of determining the position of the fixation point of the eyes of the user on the display panel is effectively improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a display device according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of another display device provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of another display device provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of another display device provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of another display device provided in an embodiment of the present application;
fig. 6 is a schematic structural diagram of another display device provided in an embodiment of the present application;
fig. 7 is a schematic structural diagram of an optoelectronic sensing assembly according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a photoelectric sensing assembly, a processing circuit and a control circuit according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a wearable display device provided in an embodiment of the present application;
fig. 10 is a flowchart of a method for determining a fixation point according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, the following detailed description of the embodiments of the present application will be made with reference to the accompanying drawings.
The terminology used in the description of the embodiments section of the present application is for the purpose of explanation only of the examples of the present application and is not intended to be limiting of the present application. Unless otherwise defined, technical or scientific terms used in the embodiments of the present application should have the ordinary meaning as understood by those having ordinary skill in the art to which the present application belongs. The use of "first," "second," "third," and similar terms in the description and claims of this application do not denote any order, quantity, or importance, but rather the terms are used to distinguish one element from another. Also, the use of the terms a, an, etc. do not denote a limitation of quantity, but rather denote the presence of at least one. The word "comprise" or "comprises", and the like, means that the element or item listed before "comprises" or "comprising" covers the element or item listed after "comprising" or "comprises" and its equivalents, and does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, which may also change accordingly when the absolute position of the object being described changes.
Fig. 1 is a schematic structural diagram of a display device according to an embodiment of the present application. As can be seen with reference to fig. 1, the display device 01 may include: a display panel 011, a plurality of photoelectric sensing elements 012, and an optical structure 013.
The display panel 011 has a display region 011a and a peripheral region 011b surrounding the display region 011a, the peripheral region 011b having a plurality of stripe-shaped detection regions a. A plurality of photoelectric sensing elements 012 are respectively located in each detection area a. The optical structure 013 is located at a side of the plurality of photoelectric sensing assemblies 012 away from the display panel 011. The optical structure 013 has a light-shielding region 013a and a plurality of light-transmitting regions 013b corresponding to the plurality of detection regions a one to one, and an orthogonal projection of each light-transmitting region 013b on the display panel 011 is located within the corresponding detection region a.
Each of the photoelectric sensing elements 012 is configured to receive an optical signal reflected by the eyes of the user and transmitted through the light-transmitting region 013b in the optical structure 013, and convert the optical signal into an electrical signal.
In the embodiment of the present application, the display device 01 may store the positions of the photoelectric sensing units 012 in advance. The photoelectric sensing element 012 receives different light signals reflected by different areas of the human eye because the different areas of the human eye have different reflectivities to light (e.g., infrared light). The photoelectric sensing element 012 has different signal values of the electrical signal converted based on different optical signals, so that the display device 01 can determine the position of the gazing point of the user's eyes on the display panel 011 based on the signal values of the electrical signal and the position of the photoelectric sensing element 012.
In general, the display device 01 has a high processing efficiency for electric signals as compared with an image because the amount of data of electric signals is small and the amount of data of an image is large. In the embodiment of the present application, the processing efficiency of the display device 01 on the electrical signals sent by the photoelectric sensing assemblies 012 is high, the position of the gaze point of the eyes of the user on the display panel 011 can be determined quickly, the efficiency of displaying images by the display panel 011 can be further improved, and the refresh rate of the display panel 011 is high.
Furthermore, by providing the optical structure 013 having the light transmitting region 013b on the side of the photoelectric sensing element 012 away from the display panel, the photoelectric sensing element 012 can only receive the light reflected by the eyes of the user, thereby effectively improving the accuracy of the display device 01 in determining the gazing point position of the eyes of the user on the display panel 011.
Furthermore, since the plurality of photoelectric sensing elements 012 are located in the peripheral area 011b of the display panel 011, the plurality of photoelectric sensing elements 012 do not affect the normal display of the display panel 011, and the display effect of the display panel 011 is better.
To sum up, the embodiment of the present application provides a display device, because display device is higher to the processing efficiency of the signal of telecommunication that each photoelectric sensing subassembly sent, consequently display device can be based on the position of the quick determination user's eyes fixation point on display panel of the signal of telecommunication that each photoelectric sensing subassembly sent, and then can improve the efficiency that display panel shows the image, display panel's refresh rate is higher. The optical structure with the light transmission area is arranged on the side, far away from the display panel, of the photoelectric sensing assembly, so that the photoelectric sensing assembly can only receive light reflected by the eyes of a user, and the accuracy of determining the position of the fixation point of the eyes of the user on the display panel is effectively improved.
In the present embodiment, there are many possible implementations of the shape of the light-transmitting region 013b in the optical structure 013, and the present embodiment is schematically illustrated by taking the following two possible implementations as examples.
In a first possible implementation manner, please refer to fig. 1 and fig. 2, fig. 2 is a schematic structural diagram of another display device provided in an embodiment of the present application, and a shape of the light-transmitting region 013b in the optical structure 013 is a circle. In this manner, the light transmitting region 013b can image the user's eye using the pinhole imaging principle, so that the photo-sensing element 013 in the display device 01 can receive only the light signal reflected by the user's eye through the light transmitting region 013b and convert the light signal into an electrical signal.
In the present application, please refer to fig. 3, wherein fig. 3 is a schematic structural diagram of another display device provided in the embodiment of the present application. The display device 01 may further include: and a first lens 014 positioned within the light transmission region 013 b. The first lens 014 can increase the amount of light signals transmitted through the light transmitting region 013b in the optical structure 013, and improve the effect of imaging the light transmitting region 013b on the user's eyes. Meanwhile, in the wearable display apparatus including the display device 01, a second lens is usually disposed between the user's eyes and the display device 01, and therefore, an image formed by the user's eyes may be significantly distorted at the edge of the second lens, which affects the accuracy of the display device 01 in determining the gazing point position of the user's eyes on the display panel 011. In this way, the first lens 014 in the display device 01, which is located in the light transmitting region 013b, may form a lens group with the second lens in the wearable display apparatus, so as to reduce the distortion degree of the image formed by the eyes of the user at the edge of the second lens, thereby improving the accuracy of the display device 01 in determining the gazing point position of the eyes of the user on the display panel 011.
Alternatively, the first lens 014 may be prepared by a process such as a photoresist thermal reflow or a glass grinding.
In a second possible implementation manner, please refer to fig. 4, fig. 4 is a schematic structural diagram of a display device according to an embodiment of the present application, and a shape of a light-transmitting region 013b in an optical structure 013 is a slit shape. The slit-shaped light transmission region 013b can increase the amount of the optical signal transmitted through the optical structure 013, so that the photoelectric sensing element 012 receives more optical signals reflected by the eyes of the user, thereby improving the accuracy of the display device 01 in determining the gazing point position of the eyes of the user on the display panel 011.
In the present application, as shown in fig. 4, the plurality of stripe-shaped detection regions a in the display panel 011 can include: a first detection area A1 extending in the first direction X and a second detection area A2 extending in the second direction Y. Wherein the first direction X intersects the second direction Y. The extending direction of the light-transmitting region 013b corresponding to the first detection region A1 intersects the extending direction of the light-transmitting region 013b corresponding to the second detection region A2. In this way, the plurality of photoelectric sensing elements 012 located in the first detection area A1 can receive the first optical signal reflected by the eyes of the user and in the direction parallel to the extending direction of the light-transmitting area 013b corresponding to the first detection area A1, and convert the first optical signal into the first electrical signal. The photoelectric sensing element 012 located in the second detection area A2 can receive a second optical signal reflected by the eyes of the user and in a direction parallel to the extending direction of the light-transmitting area 013b corresponding to the second detection area A2, and convert the second optical signal into a second electrical signal. That is, the plurality of photoelectric sensing elements 012 located in the first detection area A1 and the plurality of photoelectric sensing elements 012 located in the second detection area A2 can respectively receive optical signals reflected by the eyes of the user in two intersecting directions, and the display device 01 can determine the gazing point position according to the electrical signal converted from the optical signals reflected by the eyes of the user in the two intersecting directions, thereby improving the accuracy of the display device 01 in determining the gazing point position of the eyes of the user on the display panel 011.
In the present application, as shown in fig. 5, fig. 5 is a schematic structural diagram of another display device provided in an embodiment of the present application. The display device 01 may further include: and a lenticular lens 015 positioned within the light-transmitting region 013 b. This cylinder 015 can form the lens battery with the second lens in the wearable display device to reduce the degree of distortion of the image that user's eyes were formed at the edge of second lens, and then improve display device 01 and confirm the rate of accuracy of user's eyes gazing point position on display panel 011.
Alternatively, the lenticular lens 015 may be prepared by a process such as thermal reflow of a photoresist or glass grinding.
In the present application, please refer to fig. 4 and 5, the first direction X is perpendicular to the second direction Y. The first direction X may be a pixel row direction of the display panel 011, and the second direction Y may be a pixel column direction of the display panel 011. The plurality of detection areas a may include two first detection areas A1 and two second detection areas A2. The two first detection regions A1 may be arranged along the second direction Y and located at both sides of the display region 011a, respectively. The two second detection regions A2 may be arranged along the first direction X and located at both sides of the display region 011a, respectively. In this way, the display device 01 can determine the position of the gaze point of the user's eyes on the display panel 011 based on the photoelectric sensing elements 012 located in the two first detection areas A1 and the photoelectric sensing elements 012 located in the two second detection areas A2, and can improve the accuracy of the determined position of the gaze point.
In the embodiment of the present application, as shown in fig. 4, the extending directions of the two light-transmitting regions 013b corresponding to the two first detection regions A1 are parallel, and the extending directions of the two light-transmitting regions 013b corresponding to the two second detection regions A2 are parallel. In this way, the photoelectric sensing elements 012 located in the two first detection areas A1 can receive the optical signals of the light reflected by the eyes of the user in the same direction, and the photoelectric sensing elements 012 located in the two second detection areas A2 can receive the optical signals of the light reflected by the eyes of the user in the same direction, so as to ensure the accuracy of the determined position of the fixation point.
In the present application, the width of the light transmitting area 013b in the longitudinal direction of each detection area a influences the amount of signals received by the photoelectric sensing device 012 and the imaging quality of the user's eyes by the light transmitting area 013 b. When the width of the light transmitting region 013b in the longitudinal direction of each detection region a is increased, the amount of light transmitted through the light transmitting region 013b is increased, and the requirements for the sensitivity and the signal-to-noise ratio of the photoelectric sensor unit 012 can be reduced accordingly. However, when the width of the light transmitting region 013b in the longitudinal direction of each detection region a is larger, the collection angle of the light transmitting region 013b is increased, and the quality of imaging of the user's eyes with the light transmitting region 013b is poor. In this way, the maximum width of the light transmission region 013b corresponding to the detection region a in the longitudinal direction of each detection region a may be in the range of 10 micrometers to 100 micrometers.
In the embodiment of the present application, as shown in fig. 6, fig. 6 is a schematic structural diagram of another display device provided in the embodiment of the present application, and in order to more clearly show the structure of the display device, fig. 6 does not show an optical structure 013 in the display device 01. The display device 01 may further include: a processing circuit 016, the processing circuit 016 being connected to each of the photoelectric sensing elements 012, the processing circuit 016 being configured to determine the position of the gazing point of the user's eyes on the display panel 011 based on the magnitude of the signal value of the electric signal transmitted by each of the photoelectric sensing elements 012 and the position of at least one of the photoelectric sensing elements 012.
In the present application, as shown in fig. 1 to fig. 6, each of the plurality of photoelectric sensing elements 012 in the display device 01 has a strip shape, and an extending direction of the detection area a is perpendicular to an extending direction of the photoelectric sensing elements 012 located in the detection area a. For example, the plurality of photoelectric sensing elements 012 located in the first detection area A1 extending along the first direction X all have an extending direction along the second direction Y, and the plurality of photoelectric sensing elements 012 are uniformly arranged along the first direction X; the extending directions of the plurality of photoelectric sensing elements 012 located in the second detection area A2 extending along the second direction Y are all the first direction X, and the plurality of photoelectric sensing elements 012 are uniformly arranged along the second direction Y. In this way, the plurality of photoelectric sensing elements 012 located in the first detection area A1 can receive the first optical signal reflected by the user's eyes and in the direction parallel to the second direction Y, and convert the first optical signal into the first electrical signal. The plurality of photoelectric sensing elements 012 located in the second detection area A2 can receive the second optical signal reflected by the eyes of the user and in the direction parallel to the first direction X, and convert the second optical signal into a second electrical signal.
In this embodiment, the processing circuit 016 can receive the electrical signal transmitted by each of the plurality of optoelectronic sensing assemblies 012 located in the first detection area A1, and determine at least one target first optoelectronic sensing assembly. The processing circuit 016 can also receive the electrical signal sent by each of the plurality of photoelectric sensing elements 012 located in the second detection area A2, and determine at least one target second photoelectric sensing element. The final processing circuit 016 can determine the position of the gazing point of the user's eyes on the display panel 011 based on the position of the at least one target first electro-optical sensing component and the position of the at least one target second electro-optical sensing component.
The signal value of the electrical signal sent by the target first photoelectric sensing assembly can be smaller than or equal to a first threshold value, and the signal value of the electrical signal sent by the target second photoelectric sensing assembly can be smaller than or equal to a second threshold value. The first threshold and the second threshold may be equal to or unequal to each other, which is not limited in this application.
The user eyes comprise pupils, sclera and irises, and the position of the gaze point of the user eyes on the display panel 011 is the position of the gaze point of the pupils on the display panel 011. The light signal reflected by the pupil is minimal because the pupil is darkest in color. Further, the electrical signal converted from the optical signal reflected by the pupil is minimized. Thus, based on the signal value of the electric signal transmitted by the target first photoelectric sensing element being less than or equal to the first threshold value and the signal value of the electric signal transmitted by the target second photoelectric sensing element being less than or equal to the second threshold value, the position of the gazing point of the pupil of the user's eye on the display panel 011 can be determined.
Alternatively, the first threshold and the second threshold may be fixed values stored in the processing circuit 016 in advance. Alternatively, the first threshold may be determined by the processing circuit 016 according to the signal values of the received electrical signals of the plurality of photoelectric sensing elements 012 located in the first detection area A1. The second threshold may be determined by the processing circuit 016 according to the signal values of the received electrical signals of the plurality of photoelectric sensing elements 012 located in the second detection area A2.
For example, the processing circuit 016 may arrange the signal values of the N electrical signals transmitted by the N photoelectric sensing elements 012 located in the first detection area A1 in descending order, and may determine the signal value located in the nth bit as the first threshold. Wherein N is an integer greater than 1, and N is an integer greater than 1 and less than N/2. The processing circuit 016 may arrange the signal values of the M electrical signals transmitted by the M photoelectric sensing elements 012 located in the first detection area A1 in descending order, and may determine the signal value located at the mth bit as the second threshold. Wherein M is an integer greater than 1, and M is an integer greater than 1 and less than M/2.
Alternatively, the processing circuit 016 determines the signal value with the smallest signal value among the received electrical signals from the plurality of photoelectric sensor modules 012 located in the first detection area A1 as the first threshold value, and determines the signal value with the smallest signal value among the received electrical signals from the photoelectric sensor modules 012 located in the second detection area A1 as the second threshold value.
In the embodiment of the present application, the processing circuit 016 can determine the first coordinate value of the target first photoelectric sensing element with the smallest signal value of the electric signals transmitted by the photoelectric sensing elements 012 located in the first detection area A1, and can determine the second coordinate value of the target second photoelectric sensing element with the smallest signal value of the electric signals transmitted by the photoelectric sensing elements 012 located in the second detection area A2. The processing circuit 016 may determine the position of the gaze point of the user's eyes on the display panel 011 based on the first coordinate value and the second coordinate value.
In the present application, it is assumed that the vertical distance between the user's eye and the optical structure 013 is D (this distance D may also be referred to as the object distance), the light sensitivity of the photo-sensor unit 012The width of the region is L s The minimum distance that the display device 01 can determine that the user's eyes move is L e (the distance L e Also referred to as tracking accuracy). The vertical distance T (which may also be referred to as the image distance) between the optical structure 013 and the photo-sensing element 012 satisfies:
T＝D×L s /L e (1)
illustratively, when the object distance D is 6000 microns, the tracking accuracy L e 500 microns, and the width L of the photosensitive area of the photoelectric sensing element 012 s At 5 microns, the image distance T is 600 microns.
From equation (1), it can be seen that for object distance D and tracking accuracy L e Defining an image distance T of the display device 01 and a width L of a photosensitive area of the photoelectric sensing element 012 s In positive correlation, that is, as the image distance T increases, the width L of the photosensitive area of the photoelectric sensing element 012 s The manufacturing difficulty of the photoelectric sensing element 012 is reduced. In order to provide a larger image distance T to reduce the manufacturing difficulty of the photoelectric sensing assembly 012, in the present application, the image distance T may be increased by using backplane glass and color filter glass in the peripheral area 011b of the display panel 011.
In the embodiment of the present application, the display device 01 may further include a control circuit. Fig. 7 is a schematic structural diagram of an optoelectronic sensing assembly according to an embodiment of the present disclosure. As can be seen with reference to fig. 7, the photoelectric sensing assembly 012 may include: a switching transistor 0121 and a photodiode 0122.
Alternatively, the photodiode 0122 may be a hydrogenated amorphous silicon diode or an Organic Photodiode (OPD). Among them, the organic photodiode responds better to infrared light than a hydrogenated amorphous silicon diode.
In the embodiment of the present application, the photodiode 0122 is an organic photodiode as an example. Referring to fig. 7, a photodiode 0122 includes: a substrate 01221, and a pixel electrode 01222, a p-type material layer 01223, a photosensitive material layer 01224, an n-type material layer 01225, and a common electrode 01226 which are positioned at one side of the substrate 01221 and are sequentially stacked in a direction away from the substrate 01221.
Referring to fig. 8, fig. 8 is a schematic diagram of a photoelectric sensing assembly, a processing circuit and a control circuit according to an embodiment of the present disclosure. The first electrode of the switching transistor 0121 is electrically connected to the pixel electrode 01222, the second electrode of the switching transistor 0121 is electrically connected to the processing circuit 016, the control electrode of the switching transistor 0121 is electrically connected to the control circuit 017, and the control circuit 017 is used for controlling the on and off of the switching transistor 0121. In FIG. 8, only the first electrode of the switching transistor 0121 is electrically connected to the pixel electrode 01222, and the switching transistor 0121 is not electrically connected to the processing circuit 016 and the control circuit 017.
In the embodiment of the present application, the display panel 011 may be an LCD display panel. The liquid crystal display panel comprises an array substrate, a color film substrate and a liquid crystal layer positioned between the array substrate and the color film substrate. In general, an orthographic projection of the color film substrate on the array substrate and an orthographic projection of the liquid crystal layer on the array substrate are both located in the array substrate, and the array substrate further includes a region which is not overlapped with the color film substrate and the liquid crystal layer.
In other possible implementation manners, the display panel 011 can also be an OLED display panel or an LED display panel, and the embodiment of the present application does not limit the type of the display panel 011. When the display panel 011 is an OLED display panel or an LED display panel, the switching transistor 0121 of the photo sensor module 012 is only required to be located in the peripheral area 011b of the display panel 011.
Alternatively, the switching transistor 0121 of the photo sensor module 012 may be an amorphous silicon (a-si) thin film transistor, a Low Temperature Polysilicon (LTPS) thin film transistor, or an oxide (oxide) thin film transistor.
In the embodiment of the present application, the photo sensing assemblies 012 include the switching transistors 0121 and the photodiodes 0122, and the switching transistors 0121 may be integrated in the display panel 011 or located on a flexible circuit board attached to the display panel 011, and the arrangement of the switching transistors 0121 is not fixed, so the arrangement of the photo sensing assemblies 012 generally refers to the arrangement of the photodiodes 0122 in the photo sensing assemblies 012.
For example, the arrangement of the plurality of photoelectric sensing elements 012 along the first direction X means: the photodiodes 0122 in the plurality of photo sensor modules 012 are arranged along the first direction X. The arrangement of the plurality of photoelectric sensing elements 012 along the second direction Y means: the photodiodes 0122 in the plurality of photoelectric sensing elements 012 are arranged along the second direction Y.
In embodiments of the present application, the thickness of the optical structure 013 can range from: 0.1 to 10 microns.
Alternatively, the material of the light-shielding region 013a in the optical structure 013 can be a black photoresist (i.e., a black matrix material), or a metal material, or another material that can shield infrared light. The optical structure 013 can be formed directly on the display panel 011 by deposition, or the optical structure 013 can be formed by deposition on the substrate before the optical structure 013 is attached to the display panel 011.
To sum up, the embodiment of the present application provides a display device, because display device is higher to the processing efficiency of the signal of telecommunication that each photoelectric sensing subassembly sent, consequently display device can be based on the position of the quick determination user's eyes fixation point on display panel of the signal of telecommunication that each photoelectric sensing subassembly sent, and then can improve the efficiency that display panel shows the image, display panel's refresh rate is higher. The optical structure with the light transmitting area is arranged on one side, far away from the display panel, of the photoelectric sensing assembly, so that the photoelectric sensing assembly can only receive light reflected by the eyes of a user, and the accuracy of determining the position of the fixation point of the eyes of the user on the display panel is effectively improved.
Fig. 9 is a schematic structural diagram of a wearable display device provided in an embodiment of the present application. As can be seen with reference to fig. 9, the wearable display device 00 may include: the display device 01 provided in the above embodiment, the second lens 02 located on the display side of the display device 01, and the plurality of light emitting elements 03 located at the edge of the second lens 02. Here, the emission direction of the plurality of light-emitting elements 03 is away from the display device.
In the embodiment of the present application, the light emitted from each light emitting element 03 can be irradiated to the eyes of the user, and the light emitted from the light emitting element 03 is reflected by the eyes of the user and then is irradiated to the photo sensor element 012 through the light transmitting area 013b in the optical structure 013. This allows the photoelectric sensing unit 012 to receive the optical signal reflected by the user's eyes.
Referring to fig. 9, the wearable display apparatus 00 further includes: and the lens frame 04 is positioned at the edge of the second lens 02 and plays a role in fixing the second lens 02. The plurality of light emitting elements 03 may be fixed to a side of the lens frame 03 away from the display panel 011.
Optionally, the light emitting elements 03 may be uniformly arranged on one side of the lens frame 04 away from the display panel 011, so as to ensure uniformity of light irradiated to the eyes of the user and ensure accuracy of the display device 01 for determining the position of the fixation point of the eyes of the user on the display panel 011 based on the electrical signal sent by each of the photoelectric sensing components 012.
Optionally, the light emitting elements 03 are all infrared light emitting diodes. Since the difference between the reflectivity of the infrared light of the pupil, the sclera and the iris of the user's eye is relatively large, the light-emitting element 03 is designed as an infrared light-emitting diode, so that the difference between the light signal of the infrared light reflected by the pupil, the light signal of the infrared light reflected by the sclera and the light signal of the infrared light reflected by the iris, which are received by the photoelectric sensing component 012, is relatively large, which facilitates the processing circuit 016 to determine the position of the gazing point of the user's eye (pupil) on the display panel 011.
To sum up, the embodiment of the present application provides a wearable display device, because the display device of wearable display device is higher to the processing efficiency of the signal of telecommunication that each photoelectric sensing subassembly sent, therefore display device can be based on the signal of telecommunication that each photoelectric sensing subassembly sent is very fast confirms the position of user's eyes gazing point on display panel, and then can improve the efficiency that display panel shows the image, display panel's refresh rate is higher. The optical structure with the light transmission area is arranged on the side, far away from the display panel, of the photoelectric sensing assembly, so that the photoelectric sensing assembly can only receive light reflected by the eyes of a user, and the accuracy of determining the position of the fixation point of the eyes of the user on the display panel is effectively improved.
Fig. 10 is a flowchart of a method for determining a fixation point according to an embodiment of the present application. The method can be applied to the display device provided in the above embodiment. Referring to fig. 10, the method may include:
In the embodiment of the present application, the display device includes a display panel 011, a plurality of photoelectric sensing elements 012, and an optical structure 013. The display panel 011 has a display region 011a and a peripheral region 011b surrounding the display region 011 a. The peripheral region 011b has a plurality of stripe-shaped detection regions a. A plurality of photoelectric sensing elements 012 may be located in each detection area a. An optical structure 013 is located on a side of the plurality of photoelectric sensor assemblies 012 away from the display panel 011, the optical structure 013 has a light shielding region 013a and a plurality of light transmitting regions 013b corresponding to the plurality of detection regions a one to one, and an orthogonal projection of each light transmitting region 013b on the display panel 011 is located in the corresponding detection region a. Each of the photoelectric sensing elements 012 can receive an optical signal reflected by the eyes of the user and transmitted through the light-transmitting region 013b in the optical structure 013.
In the embodiment of the present application, after each of the photoelectric sensing elements 012 receives the optical signal reflected by the eyes of the user and transmitted through the light transmitting area 013b in the optical structure 013, the received optical signal can be converted into an electrical signal.
And 103, determining the position of the fixation point of the eyes of the user on the display panel by the display device based on the magnitude of the signal value of the electric signal and the position of the at least one photoelectric sensing assembly.
In the embodiment of the present application, the display device 01 is capable of receiving the electrical signal transmitted by each of the photoelectric sensing assemblies 012. After receiving the electric signals transmitted by the respective photoelectric sensing components 012, the display device 01 may determine the position of the gazing point of the eyes of the user on the display panel 011 based on the electric signals transmitted by the respective photoelectric sensing components 012.
In the embodiment of the present application, the display device 01 may store the positions of the photoelectric sensing units 012 in advance. The photoelectric sensing element 012 receives different light signals reflected by different areas of the human eye because the different areas of the human eye have different reflectivities to light (e.g., infrared light). The photoelectric sensing element 012 converts the electrical signal into a different signal value based on different optical signals, so that the display device can determine the position of the gazing point of the user's eyes on the display panel 011 based on the signal value of the electrical signal and the position of the photoelectric sensing element 012.
In general, the display device 01 has a high processing efficiency for electric signals as compared with an image because the amount of data of electric signals is small and the amount of data of an image is large. In the embodiment of the present application, the processing efficiency of the display device 01 on the electrical signals sent by the photoelectric sensing assemblies 012 is high, the position of the gaze point of the eyes of the user on the display panel 011 can be determined quickly, the efficiency of displaying images by the display panel 011 can be further improved, and the refresh rate of the display panel 011 is high.
In addition, by providing the optical structure 013 having the light transmitting region 013b on the side of the photoelectric sensing element 012 away from the display panel, the photoelectric sensing element 012 can only receive the light reflected by the user's eyes, thereby effectively improving the accuracy of the display device 01 in determining the gazing point position of the user's eyes on the display panel 011.
Furthermore, since the plurality of photoelectric sensing elements 012 are located in the peripheral area 011b of the display panel 011, the plurality of photoelectric sensing elements 012 do not affect the normal display of the display panel 011, and the display effect of the display panel 011 is better.
In summary, the embodiment of the present application provides a method for determining a position of a gaze point, because the display device has higher processing efficiency on electrical signals sent by each photoelectric sensing assembly, the display device can determine the position of a gaze point of a user's eyes on a display panel faster based on the electrical signals sent by each photoelectric sensing assembly, and thus can improve the efficiency of the display panel for displaying images, and the refresh rate of the display panel is higher. The optical structure with the light transmission area is arranged on the side, far away from the display panel, of the photoelectric sensing assembly, so that the photoelectric sensing assembly can only receive light reflected by the eyes of a user, and the accuracy of determining the position of the fixation point of the eyes of the user on the display panel is effectively improved.
The embodiment of the application provides a computer-readable storage medium, and instructions are stored in the computer-readable storage medium and executed by a display device to implement the method for determining the position of the gazing point provided by the method embodiment.
The above description is intended to be exemplary only, and not to limit the present application, and any modifications, equivalents, improvements, etc. made within the spirit and scope of the present application are intended to be included therein.
Claims (11)
1. A display device, characterized in that the display device comprises:
the display panel is provided with a display area and a peripheral area surrounding the display area, and the peripheral area is provided with a plurality of strip-shaped detection areas;
a plurality of photoelectric sensing assemblies positioned in each detection area;
the optical structure is positioned on one side, far away from the display panel, of the plurality of photoelectric sensing assemblies, and is provided with a light shading area and a plurality of light transmission areas in one-to-one correspondence with the plurality of strip-shaped detection areas, and the orthographic projection of each light transmission area on the display panel is positioned in the corresponding detection area;
each photoelectric sensing assembly is used for receiving an optical signal which is reflected by the eyes of a user and penetrates through a light-transmitting area in the optical structure, and converting the optical signal into an electric signal;
the light-transmitting region has a slit shape, and the display device further includes: a lenticular lens located within the light transmissive region.
2. The display device according to claim 1, wherein the plurality of stripe-shaped detection regions include: a first detection region extending in a first direction and a second detection region extending in a second direction, the first direction intersecting the second direction;
the extending direction of the light-transmitting area corresponding to the first detection area is intersected with the extending direction of the light-transmitting area corresponding to the second detection area.
3. The display device according to claim 2, wherein the first direction is perpendicular to the second direction; the plurality of strip-shaped detection areas include: two of the first detection areas and two of the second detection areas; the two first detection areas are arranged along the second direction and are respectively positioned on two sides of the display area; the two second detection areas are arranged along the first direction and are respectively positioned at two sides of the display area.
4. The display device according to claim 3, wherein two light-transmitting regions corresponding to the two first detection regions extend in parallel, and wherein two light-transmitting regions corresponding to the two second detection regions extend in parallel.
5. The display device according to any one of claims 1 to 4, wherein a maximum width of the light transmitting region corresponding to the detection region in a length direction of each of the detection regions is in a range of 10 micrometers to 100 micrometers.
6. The display device according to any one of claims 1 to 4, wherein each of the plurality of photo sensor elements has a strip-shaped photosensitive area, and the extension direction of the detection area is perpendicular to the extension direction of the photosensitive area of the photo sensor element located in the detection area.
7. The display device of claim 6, wherein a vertical distance between the optical structure and the photo-sensing componentTSatisfies the following conditions:
said D representing the vertical distance between the user's eye and the optical structure, saidL s A width of a photosensitive region representing the photoelectric sensing assemblyL e Representing the minimum distance the user's eye can determine to move.
8. The display device according to any one of claims 1 to 4, characterized in that the display device further comprises: the processing circuit is connected with each photoelectric sensing assembly and used for determining the position of the gazing point of the eyes of the user on the display panel based on the magnitude of the signal value of the electric signal sent by each photoelectric sensing assembly and the position of at least one photoelectric sensing assembly.
9. The display device according to claim 8, further comprising: a control circuit; each of the photoelectric sensing assemblies includes: a switching transistor and a photodiode;
wherein the photodiode includes: the pixel electrode, the P-type material layer, the photosensitive material layer, the N-type material layer and the common electrode are sequentially stacked on one side of the substrate along a direction far away from the substrate;
the first pole of the switch transistor is electrically connected with the pixel electrode, the second pole of the switch transistor is electrically connected with the processing circuit, the control pole of the switch transistor is electrically connected with the control circuit, and the control circuit is used for controlling the on and off of the switch transistor.
10. A wearable display device, comprising: the display device according to any one of claims 1 to 9, a second lens located on a display side of the display device, and a plurality of light emitting elements located at edges of the lens;
wherein the emitting directions of the plurality of light emitting elements are deviated from the display device.
11. A method for determining a position of a gaze point, the method being applied to the display device of any one of claims 1 to 9, the method comprising:
receiving a light signal reflected by a user's eye;
converting the optical signal into an electrical signal;
and determining the position of the fixation point of the eyes of the user on the display panel based on the magnitude of the signal value of the electric signal and the position of the at least one photoelectric sensing assembly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110586822.2A CN113325585B (en) | 2021-05-27 | 2021-05-27 | Display device, wearable display equipment and method for determining position of fixation point |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110586822.2A CN113325585B (en) | 2021-05-27 | 2021-05-27 | Display device, wearable display equipment and method for determining position of fixation point |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113325585A CN113325585A (en) | 2021-08-31 |
| CN113325585B true CN113325585B (en) | 2023-03-28 |
Family
ID=77421817
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110586822.2A Active CN113325585B (en) | 2021-05-27 | 2021-05-27 | Display device, wearable display equipment and method for determining position of fixation point |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113325585B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022246740A1 (en) * | 2021-05-27 | 2022-12-01 | 京东方科技集团股份有限公司 | Display apparatus, wearable display device, and method for determining position of gaze point |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8752963B2 (en) * | 2011-11-04 | 2014-06-17 | Microsoft Corporation | See-through display brightness control |
| NZ719204A (en) * | 2012-01-24 | 2017-11-24 | Univ Arizona | Compact eye-tracked head-mounted display |
| KR20180025524A (en) * | 2016-08-31 | 2018-03-09 | 엘지디스플레이 주식회사 | Display device for personal immersion apparatus and driving method thereof |
| WO2018119276A1 (en) * | 2016-12-22 | 2018-06-28 | Magic Leap, Inc. | Systems and methods for manipulating light from ambient light sources |
| CN108733204A (en) * | 2017-04-20 | 2018-11-02 | 上海耕岩智能科技有限公司 | A kind of method and apparatus of eyeball tracking operation |
| WO2019055839A2 (en) * | 2017-09-15 | 2019-03-21 | Trustees Of Dartmouth College | View-through sensors and apparatuses for tracking eye movement, and methods and software therefor |
| US10417784B1 (en) * | 2018-06-29 | 2019-09-17 | Facebook Technologies, Llc | Boundary region glint tracking |
-
2021
- 2021-05-27 CN CN202110586822.2A patent/CN113325585B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113325585A (en) | 2021-08-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106773273B (en) | Display device and driving method of display device | |
| US9749562B2 (en) | Liquid crystal display and infrared image sensor on silicon | |
| JP5329882B2 (en) | Display device | |
| CN109860270B (en) | Display panel and display device | |
| US20110285680A1 (en) | Image display device, electronic apparatus, image display system, method of acquiring method, and program | |
| EP3822738B1 (en) | Display screen assembly, electronic device, and control method for electronic device | |
| US20130278646A1 (en) | Image display device having imaging device | |
| EP2827185A1 (en) | Display device with infrared photosensors | |
| US20220086378A1 (en) | Electronic device and imaging method thereof | |
| CN113325585B (en) | Display device, wearable display equipment and method for determining position of fixation point | |
| US20090073345A1 (en) | Liquid crystal display device | |
| US10302484B2 (en) | Optical sensor module | |
| CN113325572B (en) | Wearable display device and method for determining position of gaze point | |
| CN110969146B (en) | Fingerprint identification component, display substrate, display panel and fingerprint identification method | |
| US20230343292A1 (en) | Light emitting device, display device, photoelectric conversion device, electronic apparatus, illumination device, and moving body | |
| CN113540155A (en) | Display screen and electronic equipment | |
| US20220293055A1 (en) | Display apparatus, photoelectric conversion apparatus and electronic device | |
| CN115867881A (en) | Display device, wearable display equipment and method for determining position of fixation point | |
| CN113966485B (en) | Display screen, terminal equipment and under-screen camera imaging control method | |
| JP2011257496A (en) | Image display device, electric equipment, image display system, image display method, illumination light distribution correction device, illumination light distribution correction method, and program | |
| JP2023032244A (en) | Display apparatus, photoelectric conversion apparatus, electronic apparatus, illumination apparatus, moving body, and wearable device | |
| JP2022085258A (en) | Light-emitting device, display device, photoelectric conversion device, electronic appliance, illumination device, and mobile body | |
| WO2022246743A1 (en) | Display apparatus, wearable display device, and method for determining point-of-gaze position | |
| WO2022246742A1 (en) | Wearable display device and method for determining position of fixation point | |
| CN112055134B (en) | Image acquisition device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |