TW202202898A - Eye tracking structure, electronic device and smart glasses - Google Patents

Abstract

An embodiment of the present invention provides an eye tracking structure, which includes a transparent substrate, a plurality of infrared micro LEDs arranged on the substrate, a plurality of micro optical sensing elements arranged on the substrate, and a micro integrated circuit on the substrate. The infrared micro LEDs and the micro optical sensing elements are electrically connected to the micro integrated circuit. The infrared micro LEDs are used to emit infrared light to a user's eyeball. The micro optical sensing elements are used to sense the infrared light reflected by the user's eyeball and collect an image of the user's eyeball. The micro integrated circuit is used to determine a rotation position of the user's eyeball and perform eye iris recognition based on the image of the user's eyeball. Another embodiment of the present invention provides an electronic device and smart glasses.

Description

Eye tracking structure, electronic device and smart glasses

The present invention relates to the field of display technology, and in particular, to an eye tracking structure, an electronic device and smart glasses.

Eye tracking technology is a technology that uses mechanical, electronic, optical and other detection methods to obtain the user's current "gazing direction". With the rapid development of computer vision, artificial intelligence technology and digital technology, eye tracking technology has become a hot research field and has a wide range of applications in the field of human-computer interaction, such as virtual reality, augmented reality, vehicle assisted driving, user Plural fields such as experience, cognitive impairment diagnosis, etc.

Of course, the conventional eye tracking structure has a single function.

An embodiment of the present invention provides an eye tracking structure, which includes: transparent substrate; a plurality of infrared micro-LEDs arranged in an array on the substrate; a plurality of miniature optical sensing elements, arrayed on the substrate; and a micro-integrated circuit, disposed on the substrate, the infrared micro-LED and the micro-optical sensing element are electrically connected to the micro-integrated circuit; Wherein, the infrared micro-LED is used to emit infrared light to the user's eyeball; The micro-optical sensing element is used for sensing the infrared light reflected back by the user's eyeball and collecting the image of the user's eyeball; The micro integrated circuit is used for determining the rotational position of the user's eyeball according to the image of the user's eyeball and performing human iris recognition according to the image of the user's eyeball.

An embodiment of the present invention further provides an electronic device, which includes a body and an eye tracking structure disposed on the body, wherein the eye tracking structure is the above-mentioned eye tracking structure.

An embodiment of the present invention also provides smart glasses, including a lens, and the above-mentioned eye tracking structure is mounted on the lens.

The eye tracking structure, the electronic device and the smart glasses have at least both an eye tracking function and a human iris recognition function, and have various functions.

FIG. 1 is a schematic plan view of an eye tracking structure 10 according to an embodiment of the present invention. As shown in FIG. 1 , the eye tracking structure 10 includes a transparent substrate 11 , a plurality of infrared micro LEDs 12 arranged in an array on the substrate 11 , a plurality of micro optical sensing elements 13 arranged in an array on the substrate 11 , and A micro integrated circuit 14 (micro integrated circuit, referred to as micro IC or μIC) on the substrate 11 . The infrared micro LED 12 and the micro optical sensing element 13 are electrically connected to the micro integrated circuit 14 .

The infrared micro-LED 12 is used to emit infrared light to the user's eyeball. The micro-optical sensing element 13 is used for sensing the infrared light reflected back by the user's eyeball and collecting the image of the user's eyeball.

The micro-integrated circuit 14 is used for determining the position of the user's eyeball rotation according to the image of the user's eyeball, so as to realize the eye tracking function. The micro integrated circuit 14 is also used to perform human iris recognition according to the image of the user's eyeball. That is, the eye tracking structure 10 has at least both an eye tracking function and a human iris recognition function, and has various functions. In addition, in the eye tracking structure 10 , the realization of the human iris recognition function can be realized based on the hardware of the eye tracking function (eg, the micro optical sensing element 13 and the micro integrated circuit 14 ), and the device has a high degree of integration.

In one embodiment, the micro integrated circuit 14 can determine the user's eye movement, such as eye movement, gaze, smooth tracking, blinking, etc., according to the position of the user's eyeball movement.

In one embodiment, the realization of the human iris recognition function of the eye tracking structure 10 includes the following steps.

S1: image acquisition.

S2: iris liveness detection.

S3: Segmentation and normalization.

S4: Feature extraction and encoding.

S5: feature matching (feature comparison), and calculate the similarity value (similarity value).

In step S1, the micro-optical sensing element 13 collects the image of the user's eyeball. In one embodiment, the image of the eyeball may include some or all of the eye, as shown in FIG. 3A .

In step S2, the micro integrated circuit 14 performs iris biometric detection according to the image of the user's eyeball. In one embodiment, the step S1 may perform biometric detection based on pupil grayscale. Since the pupil is the clear aperture of the human eye, the light from the pupil passes through the lens and is then imaged in the retina, and the light entering the pupil is rarely reflected out of the pupil, so the pupil is black when observed under different brightnesses of the outside world. Corresponding to a non-living body (such as a fake eye printed on paper), the pupil area of the fake eye will show different grayscales under different brightness. By using the characteristic that the pupil area of the living human eye does not reflect external light, the living body detection is performed, and the detection speed is fast. If in step S2, the living body detection is passed, then step S3 is performed.

In step S3, the micro integrated circuit 14 acquires the annular iris recognition area according to the image of the user's eyeball.

In one embodiment, as shown in FIG. 2 , the inter-ocular distance (IOD) between the centers of the two pupils of the user is defined as the inter-ocular distance D. It is defined that the center of the pupil in the image of the user's eyeball is the center of the circle, and the circular area with the diameter R1 is the area of the human eye. It is defined that the center of the pupil in the image of the user's eyeball is the center of the circle, and the circular area with the diameter R3 is the pupil area. Define an annular area with the center of the pupil in the image of the user's eyeball as the center of the circle, the diameter of the inner circle as R3, and the diameter of the outer circle as R2 as the annular iris recognition area.

In one embodiment, the diameter R1 of the human eye region is 0.25D, the outer diameter R2 of the annular iris recognition region is 0.1D, and the diameter R3 of the pupil region is 0.05D. That is, due to the difference in the size of human eyes and the occlusion effect of the upper and lower eyelids, part of the iris information cannot be used. In this embodiment, an annular area with inner and outer circle radii of 0.05D and 0.1D is selected as the iris identification area, and reserved for comparison The part close to the pupil to meet the needs of iris recognition.

In other embodiments, since the center of the iris and the center of the pupil may not necessarily coincide, in the annular iris identification area, the center of the inner and outer circles may not necessarily be the center of the pupil.

As shown in FIG. 3A , step S3 further includes cutting the annular iris identification area. In FIG. 3A , the annular iris recognition area is divided into four equal parts.

As shown in FIG. 3B , step S3 further includes converting the circular iris recognition area into a rectangular iris recognition area. Wherein, starting from the horizontal axis on the right side of the center of the iris, and clockwise around the center of the iris, the rectangular iris recognition area is obtained by expanding in turn. The upper boundary of the rectangular iris recognition area corresponds to the outer circular boundary of the iris, and the inner circular boundary of the iris (ie, the pupil boundary) is located at the lower boundary of the rectangular iris recognition area.

In step S3, it also includes normalizing the rectangular iris recognition area. For example, the height and lower boundary of the rectangular iris recognition area are corrected to normalize them.

In step S4, the coding can be performed by a method of logical judgment, for example, the feature points are marked as logical "1", the non-feature points are marked as logical "0", and 0 and 1 constitute the feature information of the iris.

In step S5, the feature matching may include, for example, preliminary matching, fine matching, and the like.

Thereby, the eye tracking structure 10 can realize the human iris recognition function based on the hardware of the eye tracking function (eg, the micro optical sensing element 13 and the micro integrated circuit 14 ), and the device has a high degree of integration.

In addition, in the eye tracking structure 10 , the infrared micro LED 12 , the micro optical sensing element 13 , and the micro integrated circuit 14 are all micron-scale, the substrate 11 is transparent, and the light transmittance of the eye tracking structure 10 is very high. In one embodiment, the size of the infrared micro LEDs 12 ranges from 1 micron to 100 microns. The size of the miniature optical sensing element 13 ranges from 1 micrometer to 100 micrometers.

In one embodiment, the miniature optical sensing element 13 is a Complementary Metal Oxide Semiconductor (CMOS) element.

In one embodiment, the infrared micro LED 12 and the micro optical sensing element 13 are electrically connected to the micro integrated circuit 14 through the lead wire 16 . The lead 16 is a transparent conductive material, such as indium tin oxide (Indium Tin Oxide, ITO).

In one embodiment, the substrate 11 is made of transparent glass or transparent plastic, such as polyimide (PI). Wherein, when the eye tracking structure 10 is installed on the lens 40 of the smart glasses 100 and used, the material of the substrate 11 is transparent glass with certain bending properties or transparent plastic with certain bending properties.

In one embodiment, the eye tracking structure 10 can be applied to virtual reality (VR), augmented reality (AR), and mixed reality (MR) products.

In one embodiment, the eye tracking structure 10 further includes a plurality of pixel micro-LEDs (not shown) arranged on the substrate 11 in an array. The pixel micro-LED is electrically connected to the micro-integrated circuit 14 and performs image display under the control of the micro-integrated circuit 14 . Thereby, the eye tracking structure 10 has the functions of eye tracking, iris recognition and image display.

In one embodiment, the size of the pixel micro-LED ranges from 1 micron to 100 microns, which has the advantages of high brightness, low power consumption, high reliability, and short response time.

In one embodiment, the eye tracking structure 10 further includes a light guide 50 (as shown in FIGS. 6-8 ). The light guide 50 is attached to the side of the substrate 11 where the infrared micro LEDs 12 are arranged. The infrared light emitted by the infrared micro LED 12 to the user's eyeball passes through the light guide 50 and reaches the user's eyeball, and the infrared light reflected by the user's eyeball passes through the light guide 50 and reaches the micro-optical sensing element 13. Thereby, the infrared light emitted by the infrared micro LED 12 can be coupled to the user's eyeball through the light guide 50 , and the infrared light reflected by the user's eyeball can be coupled to the micro optical sensing element 13 through the light guide 50 .

In one embodiment, the eye tracking structure 10 can be incorporated into an electronic device (not shown). The electronic device includes a body and an eye-tracking structure 10 disposed on the body. Wherein, the electronic device may be a non-contact smart terminal, which on the one hand performs preset operations by tracking the user's eyeballs, and on the other hand, also has an iris recognition function.

In one embodiment, the electronic device may be a smart phone, which performs a specific screen display by tracking the movement of the user's eyeballs, and also has an iris recognition function. In one embodiment, the electronic device can also be an access control device, which can perform eye tracking and iris recognition without contact.

4 is a schematic structural diagram of an eye tracking structure 20 according to another embodiment of the present invention, which is different from the eye tracking structure 10 shown in FIG. different. In the eye tracking structure 20 , a plurality of the micro optical sensing elements 13 surround the micro integrated circuit 14 to form an inner circle, and a plurality of the infrared micro LEDs 12 are arranged around the inner circle. Thereby, the accuracy and speed of eye tracking are further improved.

FIG. 5 is a schematic perspective view of the smart glasses 100 according to an embodiment of the present invention. As shown in FIG. 5 , the smart glasses 100 include a frame 30 and a lens 40 disposed on the frame 30 . The frame 30 includes a frame 32 , a temple 34 and a connecting portion 36 . The temple 34 is connected to the frame 32 by the connecting portion 36 , and can be folded relative to the frame 32 . The above-mentioned eye tracking structure 10 ( 20 ) is mounted on the lens 40 .

In one embodiment, the eye tracking structure 10 ( 20 ) is located in the lens 40 (as shown in FIG. 6 ), or attached to the surface of the lens 40 close to the eyeball of the user (as shown in FIG. 7 ) shown), or located on the side of the lens 40 close to the user's eyeball without fitting with the lens 40 (as shown in FIG. 8 ). Wherein, if the eye tracking structure 10 ( 20 ) is located on the side of the lens 40 close to the eyeball of the user and does not fit with the lens 40 , the eye tracking structure 10 ( 20 ) can fit or Installed on other components of the smart glasses 100 . For convenience of description, only the substrate 11 is schematically drawn in FIGS. 6 to 8 , and the micro optical sensing element 13 , the infrared micro LED 12 , and the micro integrated circuit 14 on the substrate 11 are omitted.

As shown in FIGS. 6 to 8 , when the eye tracking structure 10 ( 20 ) includes a light guide 50 , the light guide 50 is closer to the user's eyeball than the substrate 11 . In this way, the light guide 50 can couple the infrared light emitted by the infrared micro LED 12 to the user's eyeball, and the infrared light reflected by the user's eyeball can also be coupled to the micro optical sensing element 13 through the light guide 50 . In addition, when the eye tracking structure 10 ( 20 ) has a display function, the visible light emitted by the micro pixel LED can also be coupled to the user's eyeball through the light guide 50 so as to be viewed by the user.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced. without departing from the spirit and scope of the technical solutions of the present invention.

10, 20: Eye Tracking Architecture 11: Substrate 12: Infrared Micro LED 13: Miniature Optical Sensing Elements 14: Micro integrated circuits 16: Leads 100: Smart Glasses 30: Frames 32: Frame 34: temples 36: Connection part 40: Lens 50: Light guide

FIG. 1 is a schematic plan view of an eye tracking structure according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of an annular iris recognition area.

FIG. 3A is a schematic diagram of cutting a circular iris recognition area.

FIG. 3B is a schematic diagram of a rectangular iris recognition area.

FIG. 4 is a schematic structural diagram of an eye tracking structure according to another embodiment of the present invention.

FIG. 5 is a schematic perspective view of smart glasses according to an embodiment of the present invention.

6 is a schematic diagram of the positions of the lenses and the eye tracking structure in the smart glasses according to an embodiment of the present invention.

7 is a schematic diagram of the positions of the lenses and the eye tracking structure in the smart glasses according to another embodiment of the present invention.

8 is a schematic diagram of the positions of the lenses and the eye tracking structure in the smart glasses according to still another embodiment of the present invention.

10: Eye Tracking Architecture

11: Substrate

12: Infrared Micro LED

13: Miniature Optical Sensing Elements

14: Micro integrated circuits

Claims (10)

An eye-tracking structure is improved by including: transparent substrate; a plurality of infrared micro-LEDs arranged in an array on the substrate; a plurality of miniature optical sensing elements, arrayed on the substrate; and a micro-integrated circuit, disposed on the substrate, the infrared micro-LED and the micro-optical sensing element are electrically connected to the micro-integrated circuit; Wherein, the infrared micro-LED is used to emit infrared light to the user's eyeball; The micro-optical sensing element is used for sensing the infrared light reflected back by the user's eyeball and collecting the image of the user's eyeball; The micro-integrated circuit is used for determining the rotational position of the user's eyeball according to the image of the user's eyeball and performing human iris recognition according to the image of the user's eyeball. The eye tracking structure according to claim 1, wherein the micro integrated circuit can acquire a circular iris recognition area according to the image of the user's eyeball, and convert the circular iris recognition area into a rectangle The iris recognition area is normalized, and then iris feature points are extracted, encoded and iris matched. The eye tracking structure according to claim 2, wherein the distance between the eyes of the user is defined as D, the center of the circular iris recognition area is the center of the pupil in the image of the user's eyeball, The inner diameter of the annular iris identification area is 0.05D, and the outer diameter is 0.1D. The eye tracking structure according to claim 2, wherein before the micro integrated circuit acquires the annular iris recognition area, it is further used to perform living body detection according to the image of the user's eyeball. The eye-tracking structure according to claim 1, wherein a plurality of the micro-optical sensing elements surround the micro-integrated circuit to form an inner circle, and a plurality of the infrared micro-LEDs surround the inner circle formed by the micro-optical sensing element. Circle arrangement. The eye-tracking structure according to claim 1, further comprising a plurality of pixel micro-LEDs arranged in an array on the substrate, the pixel micro-LEDs being electrically connected to the micro-integrated circuit, and connected to the micro-integrated circuit. image display under the control of . The eye tracking structure according to claim 1, further comprising a light guide, the light guide is attached to the side of the substrate on which the infrared micro LEDs are arranged, and the infrared micro LEDs emit infrared rays to the user's eyeballs. The light reaches the eyeball of the user after passing through the light guide, and the infrared light reflected by the eyeball of the user reaches the micro optical sensing element after passing through the light guide. An electronic device includes a body and an eye-tracking structure disposed on the body, wherein the eye-tracking structure is the eye-tracking structure described in any one of claim 1 to 7. A kind of smart glasses, comprising lenses, wherein the eye tracking structure according to any one of claim 1 to 7 is mounted on the lenses. The smart glasses according to claim 9, wherein the eye tracking structure is located in the lens, or is attached to the surface of the lens close to the user's eyeball, or is located in the lens close to the user side of the eyeball without fitting with the lens.

Images