WO2022199580A1 - 电子设备及其控制方法 - Google Patents

电子设备及其控制方法 Download PDF

Info

Publication number
WO2022199580A1
WO2022199580A1 PCT/CN2022/082282 CN2022082282W WO2022199580A1 WO 2022199580 A1 WO2022199580 A1 WO 2022199580A1 CN 2022082282 W CN2022082282 W CN 2022082282W WO 2022199580 A1 WO2022199580 A1 WO 2022199580A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
camera module
light source
electronic device
display screen
Prior art date
Application number
PCT/CN2022/082282
Other languages
English (en)
French (fr)
Inventor
朱帅帅
曾以亮
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP22774236.8A priority Critical patent/EP4293412A4/en
Priority to US18/552,553 priority patent/US20240171846A1/en
Publication of WO2022199580A1 publication Critical patent/WO2022199580A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/30—Polarising elements
    • G02B5/3025—Polarisers, i.e. arrangements capable of producing a definite output polarisation state from an unpolarised input state
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera

Definitions

  • the present application relates to the technical field of electronic devices, and in particular, to an electronic device and a control method thereof.
  • eye movement calibration needs to be performed every time before using the head-mounted display device, so as to calibrate the visual axis of the eyeball.
  • the eye movement calibration time is long, the user cannot use the device during the calibration time, and the user experience is poor.
  • Embodiments of the present application provide a control method for an electronic device and an electronic device.
  • the control method can effectively improve user experience.
  • a control method of an electronic device includes a light source, a lens group, a display screen, and a camera module.
  • the light source is used to emit first light to the eyeball of the user, the first light is reflected by the eyeball, and enters the camera through the lens group.
  • the display screen controls the display screen to display according to a display period, and the display period includes alternately arranged display periods and black-insertion periods;
  • controlling the camera module to expose during the black insertion period to obtain the first light, and stop exposing during the display period;
  • An eyeball image is formed according to the first light.
  • the light source, lens group and camera module of the present application constitute an identification device to identify the user, and can register the eyeball image information when the user uses the electronic device for the first time, and calibrate the parameters of his eye movement. Save it in the personal file.
  • the system will directly call the user's eye movement calibration parameters after confirming the user's identity through scleral recognition. The user does not need to perform eye movement calibration again, which saves the user's time and greatly improves the user experience.
  • the present application controls the camera module to expose when the display screen is in black, and stop exposure when the display screen is in the display period, so that during the identification process, the second light of the display screen will not be sensed by the camera module, preventing Stray light from the display screen enters the camera module, effectively improving the accuracy of identification.
  • control method before controlling the exposure of the camera module, the control method further includes lighting a light source, so that the camera module can receive the first light during exposure.
  • the lighting of the light source includes lighting the light source during a lighting period, and the lighting period covers at least one of the display periods and at least one of the black-insertion periods; or the lighting period covers at least one of the black insertion periods. That is to say, the light source can be in a long bright state, or it can only be displayed during the black-out period, that is, the light source is turned on when the camera module is exposed, and turned off when the camera module is not exposed, which can effectively save energy.
  • the light source is a visible light source
  • the eyeball image is used for sclera identification. Since the eye pattern of the eyeball has a high contrast under visible light illumination, the use of visible light illumination for scleral identification can improve the accuracy of identification.
  • the light source includes a visible light source and a near-infrared light source
  • the camera module is a near-infrared light and a visible light double-pass camera module
  • the control method also includes:
  • the visible light source is controlled to emit the first light, and the camera module is exposed to obtain a first eyeball image, and the first eyeball image is used for sclera identification;
  • the near-infrared light source is controlled to emit the first light, and the camera module is exposed to obtain a second eyeball image, and the second eyeball image is used for eye tracking.
  • the camera module can be a near-infrared light and visible light dual-pass camera module, and the same camera module can be used for eye tracking and user identification, which effectively saves cost and space for electronic equipment.
  • control method further includes: comparing the first eyeball image with the pre-stored image; if the two match, calling the pre-stored eye movement calibration data, and the pre-stored eye movement calibration data is the same as the pre-stored eye movement calibration data. corresponding to the pre-stored images.
  • the control method provided in this application can register the sclera information of the user when the user uses the electronic device for the first time, and save the parameters of the eye movement calibration in the personal file by identifying the user.
  • the system passes After scleral recognition confirms the user's identity, the user's eye movement calibration parameters are directly called, so that the user does not need to perform eye movement calibration again, which saves the user's time and greatly improves the user experience.
  • identity recognition can also be used in scenarios in which electronic devices are in use, such as replacing password payments with scleral recognition payments.
  • control method is applied when the electronic device is turned on and in the boot interface, the control method further includes: comparing the first eyeball image with the pre-stored image; if the two match, unlocking the electronic device, so that the electronic device enters the main interface.
  • the control method of the present application can also be the same as the unlocking of the electronic device, and the unlocking speed through scleral identification is fast, which improves the user experience.
  • control method further includes: if the first eyeball image does not match the pre-stored image, calibrating the user's eye movement, obtaining eye movement calibration data, and calibrating the eye movement.
  • the data is stored in the eye movement calibration database for next use.
  • an electronic device in a second aspect, includes a light source, a lens group, a camera module, a display screen, and a controller, the display screen is arranged on the light-emitting side of the lens group, and the light source is arranged on a side of the lens group away from the display screen.
  • the light incident side of the camera module faces the lens group;
  • the display period of the display screen includes alternately arranged display periods and black-insertion periods;
  • the light source is used to emit first light to the user's eyeball, The first light is reflected by the eyeball and enters the camera module through the lens group; part of the second light emitted by the display screen is reflected or transmitted through the lens group and enters the camera module;
  • the light source, the The camera module and the display screen are both electrically connected with the controller, and the controller is used to control the camera module to be exposed during the black-insertion period to obtain the first light, and the display The exposure is stopped during the period, and an eyeball image is formed according to the first light.
  • the electronic device has an identification function.
  • the controller controls the camera module to be exposed during the black insertion period, and stops exposure when the display screen is in the display period, so that during the identification process, the second light of the display screen is It will not be sensed by the camera module, effectively preventing stray light from the display screen from entering the camera module, so that the obtained eyeball image is clearer, and the accuracy of identification is effectively improved.
  • the light source is a visible light source. Since the eye pattern of the eyeball has a high contrast under visible light illumination, the use of visible light illumination for scleral identification can improve the accuracy of identification.
  • the light source includes a visible light source and a near-infrared light source
  • the camera module is a near-infrared light and visible light dual-pass camera module, so that both eye tracking and user identification can use the same camera module. group, effectively saving cost and space of electronic equipment.
  • the lens group includes a lens and a semi-reflective and semi-transparent film, and the semi-reflective and semi-transparent film is provided on the light-emitting side of the lens.
  • Half of the second light from the display screen is reflected by the transflective film, and half is transmitted through the transflective film, resulting in folding in the lens group, which increases the optical path of the second light but does not increase the thickness of the electronic device, which is beneficial to electronic Miniaturization of equipment.
  • the transmittance of the transflective film in the first wavelength band is greater than 95%, and the reflectivity is less than 1%, wherein the first wavelength band is a frequency band where near-infrared light is located.
  • the first light is near-infrared light.
  • this solution can effectively reduce the light reflected by the transflective film and finally enter the camera module, reducing the ghost image formed, and on the other hand, it can reduce the loss of the intensity of the first light.
  • the camera module can identify the clear first light, which is beneficial to the algorithm technology of the eye tracking device, improves the accuracy of eye tracking, and further improves the user experience. .
  • the lens group further includes a reflective polarizer and a first wave plate, the first wave plate is provided on the side of the lens facing away from the transflective film, and the reflective polarizer is The film is arranged on the side of the first wave plate facing away from the lens;
  • the camera module further includes a first linear polarizer and a second wave plate, and the first linear polarizer is located between the second wave plate and the second wave plate.
  • the polarization direction of the reflective polarizer and the polarization direction of the first linear polarizer are the same or perpendicular to each other, and the first wave plate and the second wave plate are both four.
  • One-wave plate is one-wave plate.
  • the reflective polarizer, the first wave plate and the transflective film can be understood as the multi-layer film structure in the Pancake lens group, the purpose is to make the light fold between the film layers.
  • the camera module includes a first linear polarizer and a second wave plate, which can effectively eliminate the light emitted after the lens group is folded and greatly reduce the brightness of ghost images.
  • the lens group further includes a reflective polarizer and a first wave plate, the first wave plate is provided on the side of the lens facing away from the transflective film, and the reflective polarizer is The film is arranged on the side of the first wave plate facing away from the lens;
  • the camera module further includes a first circular polarizer, and the first circular polarizer has the same rotation direction as the first wave plate, so The polarization directions of the first circular polarizer and the reflective polarizer are the same or perpendicular to each other.
  • the reflective polarizer, the first wave plate and the transflective film can be understood as the multi-layer film structure in the Pancake lens group, the purpose is to make the light fold between the film layers.
  • the camera module includes a first circular polarizer, which can effectively eliminate the light emitted after the lens group is folded, and greatly reduce the brightness of ghost images.
  • the electronic device further includes a second linear polarizer and a third wave plate, the second linear polarizer is arranged on the light-emitting side of the display screen, and the third wave plate is arranged on the between the second linear polarizer and the lens group.
  • the second linear polarizer and the third wave plate between the display screen and the lens group, the second light emitted from the display screen can be intercepted and returned to the display screen after being reflected by the lens group.
  • the ghosting phenomenon caused by light allows users to see a clearer picture.
  • FIG. 1 is a schematic diagram of calibration of a head-mounted display device in the related art
  • FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of an augmented reality component of the structure shown in FIG. 2;
  • Fig. 4 is the schematic diagram of eyeball
  • FIG. 5 is a schematic diagram of a propagation path of a first light ray of the augmented reality component shown in FIG. 3;
  • FIG. 6 is a schematic diagram of a propagation path of another light of the augmented reality component shown in FIG. 3;
  • FIG. 7 is a schematic diagram of a propagation path of the second light of the structure shown in FIG. 3;
  • FIG. 8 is a schematic diagram of another propagation path of the second light of the structure shown in FIG. 3;
  • FIG. 9 is a schematic flowchart of a control method for an electronic device provided by an embodiment of the present application.
  • FIG. 10 is a schematic display diagram of the display screen of the augmented reality component shown in FIG. 3;
  • Fig. 11 is the control schematic diagram of the control method shown in Fig. 9;
  • Fig. 12 is another control schematic diagram of the control method shown in Fig. 9;
  • Fig. 13 is another control schematic diagram of the control method shown in Fig. 9;
  • Fig. 14 is another control schematic diagram of the control method shown in Fig. 9;
  • FIG. 15 is a schematic diagram of part of the operation flow of the control method shown in FIG. 9 .
  • connection may be detachable connection, or It is a non-removable connection; it can be a direct connection or an indirect connection through an intermediate medium.
  • fixed connection refers to connection with each other and the relative positional relationship after connection remains unchanged.
  • Rotationally connected means connected to each other and capable of relative rotation after connection.
  • FIG. 1 is a schematic diagram of calibration of a head-mounted display device in the related art.
  • the head-mounted display device in the related art can track the user's eye movement, and the user's eye movement needs to be calibrated before using the head-mounted display device.
  • the general calibration process is to display 5 to 9 points in sequence on the display screen, and prompt the user to look at these points in sequence. Specifically, as shown in Fig. 1, 5 points are displayed on the display screen, and different colors are used to identify the points to guide the user to look at. For example, the point in the middle of The dots are changed to red to prompt the user to look.
  • the time for each point to turn red is about 2 seconds, plus the instruction tutorial before calibration, the entire eye movement calibration time is about 20 to 40 seconds. Since the user needs to perform eye movement calibration every time before wearing the electronic device, the user cannot use the electronic device within half a minute after wearing the electronic device, and the user experience is poor.
  • Embodiments of the present application provide an electronic device for wearing on a user's head.
  • the electronic device may be a head-mounted display device, for example, the electronic device may be augmented reality (AR) glasses, AR helmets, mixed reality (MR) glasses or MR helmets, etc. that combine digital content with real scenes. electronic product.
  • the electronic device may also not be worn on the head.
  • the electronic device of the present application solves the problem of poor user experience of the electronic device in the related art.
  • the electronic device is an example of AR glasses for specific description.
  • FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device 100 includes a frame 10 and an augmented reality component 20 mounted on the frame 10 .
  • an augmented reality component 20 mounted on the frame 10 .
  • there are two augmented reality components 20 and the two augmented reality components 20 are installed on the glasses frame 10 at intervals.
  • the frame 10 includes a mirror body 11 and a temple 12 connected to the mirror body 11 . There are two temples 12 , and the two temples 12 are connected to opposite ends of the mirror body 11 . It should be noted that, in other embodiments, the mirror frame 10 may also include a mirror body 11 and a fixing band connected to the mirror body 11 , which is not specifically limited in this application.
  • the lens body 11 may include a mounting surface 110 and an eyecup 111 .
  • the eyecup 111 is disposed on the mounting surface 110 and forms a receiving cavity 112 with the mounting surface 110 .
  • the number of the accommodating cavities 112 is two, and the two accommodating cavities 112 are respectively used for accommodating the electronic components of the electronic device 100 .
  • the position of the eye mask 111 between the two accommodating cavities 112 forms a nose pad for easy wearing.
  • the mirror body 11 may be an integral structure to ensure the overall strength of the mirror body 11 .
  • the material of the mirror body 11 includes but is not limited to metal, plastic, resin or natural materials. It should be understood that the lens body 11 is not limited to the structure shown in FIG. 2 , but can also be a half-frame or rimless spectacle frame, as long as it can accommodate the electronic components of the electronic device 100 .
  • the two temples 12 are rotatably connected to opposite ends of the mirror body 11 .
  • the two temples 12 are rotated relative to the mirror body 11 to face each other.
  • the two temples 12 of the electronic device 100 can be respectively erected on the two temples of the user.
  • the nose bracket is arranged on the bridge of the user's nose so as to be worn on the user's head.
  • the user's two eyeballs are respectively opposite to their corresponding accommodating cavities 112 , so as to facilitate viewing of the picture presented by the electronic components located in the accommodating cavities 112 .
  • the two temples 12 are rotated relative to the mirror body 11 to at least partially overlap each other and are accommodated inside the mirror body 11 . At this time, the electronic device 100 can be accommodated.
  • the eye mask 111 may have a certain flexibility.
  • the eye mask 111 When the user wears the electronic device 100, the eye mask 111 is in contact with the user's skin to prevent external light from passing through the gap between the user's skin and the eye mask 111, thereby affecting the user's viewing experience.
  • the eye mask 111 is flexible, which can buffer the gravity of the electronic device 100 and bring a good wearing experience to the user.
  • the eyecup 111 may not be flexible.
  • the two temples 12 may be fixedly connected to opposite ends of the mirror body 11 respectively.
  • the two temple legs 12 may be integrally formed with the mirror body 11 , that is, the electronic device 100 is always in an unfolded state, which is not specifically limited in this application.
  • the interior of the temple 12 may also be provided with a receiving cavity, and the receiving cavity of the temple 12 may also accommodate the electronic components of the electronic device 100 .
  • the terms “inside”, “outside” and other orientations are mainly described based on the orientation of the electronic device 100 when the electronic device 100 is worn on the head of the user.
  • the inner side is close to the user's head, and the outer side is away from the user's head, which does not limit the orientation of the electronic device 100 in other scenarios.
  • the length direction of the electronic device 100 as the x-axis direction
  • the width direction of the electronic device 100 as the y-axis direction
  • the thickness direction of the electronic device 100 as the z-axis direction
  • the x-axis direction is perpendicular to each other two by two.
  • the x-axis direction is the direction from one containing cavity 112 to the other containing cavity 112 in the containing cavity 112
  • the z-axis direction is the direction in which the mirror body 11 faces the temple 12 .
  • the structures of the two augmented reality components 20 are the same. Specifically, the two augmented reality components 20 are respectively installed in the corresponding accommodating cavities 112 of the mirror body 11 . That is, the two augmented reality components 20 are arranged side by side along the x-axis direction. When the electronic device 100 is worn on the user's head, one augmented reality component 20 corresponds to the user's left eye, and the other augmented reality component 20 corresponds to the user's right eye. At this time, the user's eyes can view virtual reality through the two augmented reality components 20 . scene and real scene. It should be noted that, in other embodiments, the structures of the two augmented reality components 20 may also be different, which is not specifically limited in this application.
  • FIG. 3 is a schematic structural diagram of the augmented reality component 20 of the structure shown in FIG. 2 .
  • the augmented reality assembly 20 includes a light source 21 , a lens group 22 , a display screen 23 , a camera module 24 and a controller 25 .
  • the camera module 24 is located between the lens group 22 and the display screen 23.
  • the light incident side of the camera module 24 faces the lens group 22.
  • the light source 21 is arranged on the side of the lens group 22 away from the camera module 24.
  • the eyeball of the device emits a first light ray, and the first light ray is reflected by the eyeball and enters the camera module 24 through the lens group 22 .
  • Part of the second light emitted by the display screen 23 is reflected by the lens group 22 and enters the camera module 24 .
  • the light source 21 , the camera module 24 and the display screen 23 are all electrically connected to the controller 25 , and the controller 25 is used to control the light source 21 and the camera module 24 according to the display period of the display screen 23 .
  • each of the two augmented reality components 20 has a display screen 23 , or the two augmented reality components 20 share a display screen 23 .
  • the light source 21, the lens group 22, the camera module 24 and the controller 25 constitute an eye tracking device or an identification device
  • the display screen 23 and the lens group 22 constitute a display device.
  • the eye tracking device/identification device and the display device share the lens group 22 .
  • Sclera recognition also known as eye pattern recognition, is based on the distribution of capillaries in the sclera area of the user's eyeball for identification.
  • Figure 4 is a schematic diagram of the eyeball, in which the gray curved short lines are the capillaries in the sclera region, also known as eye lines.
  • the augmented reality component 20 in this embodiment can be used to simulate a virtual environment through a display device.
  • a stereoscopic image may be displayed on the display screen 23 to simulate an illusion of depth, ie, to create a virtual environment so that the user has the illusion of being in the virtual environment.
  • the augmented reality component 20 can in turn estimate which part of the virtual environment the user is viewing through the eye tracking device. That is to say, while the display device of the augmented reality component 20 can present the virtual environment for the user, the eye tracking device can detect the position where the user's gaze is fixed, that is, the specific position of the virtual environment where the user's gaze is fixed. Better adjust the virtual environment presented to the user and improve the user experience.
  • the augmented reality component 20 can also identify the user through the identification device, so as to reduce the related calibration steps and improve the user experience.
  • the light source 21 includes a visible light source 211 and a near-infrared light source 212
  • the visible light source 211 , the lens group 22 , the camera module 24 and the controller 25 constitute an identification device.
  • the near-infrared light source 212 , the lens group 22 , the camera module 24 and the controller 25 constitute an eye tracking device. Since eye tracking and sclera recognition both require the user's eyeball image, eye tracking and sclera recognition can share one camera module 24 , which effectively saves cost and space of the electronic device 100 .
  • the controller 25 controls the near-infrared light source 212 to light up, and the first light is near-infrared light, which is sensed by the camera module 24 to form an eyeball image. Since eye tracking may accompany the experience process of most of the electronic devices 100, the near-infrared light source 212 is used during eye tracking, which is invisible to the user, thereby improving user experience.
  • the controller 25 controls the visible light source 211 to light up, and the first light is visible light, which is sensed by the camera module 24 to form an eyeball image. Since the eye pattern of the eyeball has a high contrast under visible light illumination, the use of visible light illumination for scleral identification can improve the accuracy of identification.
  • the light source of the eye tracking device may also be a visible light source
  • the light source of the identity recognition device may be a near-infrared light source.
  • the light sources of the eye tracking device and the identification device are both visible light sources.
  • the light sources of the eye tracking device and the identification device are both near-infrared light sources.
  • the visible light source 211 and the near-infrared light source 212 may be two separate light sources.
  • the visible light source 211 and the near-infrared light source 212 can be disposed on the side of the lens group 22 that faces away from the display screen 23, so as to emit light to the user's eyeball, so as to perform eye tracking or identification of the user.
  • the light source 21 can be fixed to the mirror body 11 or the temple 12, or fixed to the mirror body 11 or the temple 12 through an intermediate connecting piece (FIG. 2).
  • the number of visible light sources 211 and near-infrared light sources 212 may be one or more.
  • the visible light source and the near-infrared light source can also be an integrated light source 21, that is, the light source can emit both visible light and near-infrared light, and the controller can control the light source to emit visible light or near-infrared light.
  • the light source is only a visible light source.
  • the augmented reality component 20 only shows one visible light source 211 and one near-infrared light source 212 .
  • the number of visible light sources 211 and near-infrared light sources 212 may also be multiple.
  • the single light source 21 of the multiple visible light sources 211 and the near-infrared light sources 212 The luminous intensity can be set to be weaker, which can reduce the risk of damage to the user's eyeballs due to excessive light exposure.
  • the visible light sources 211 and the near-infrared light sources 212 may also be disposed in other positions of the augmented reality component 20 , and the present application does not limit the number and installation positions of the visible light sources 211 and the near-infrared light sources 212 .
  • the electronic device 100 of the present application can register the sclera information when the user uses the electronic device 100 for the first time by adding an identification device, and save the parameters of the eye movement calibration in the personal file.
  • the system After confirming the user's identity through scleral recognition, the user's eye movement calibration parameters are directly called, so that the user does not need to perform eye movement calibration again, which saves the user's time and greatly improves the user experience.
  • the identification device can also be used in a scenario in which the electronic device 100 is in use, for example, payment with scleral identification is used instead of password payment.
  • the lens group 22 is a pancake lens group, and the lens group 22 includes a reflective polarizer 221 , a first wave plate 222 , a lens 223 and a Beam Splitter (BS) 224 .
  • the film 222 is arranged on the light incident side of the lens 223. It can be understood that the light incident side of the lens 223 is the side where the light of the light source 21 enters the lens 223.
  • the transflective film 224 is disposed on the light-emitting side of the lens 223 , that is, the side of the lens 223 facing away from the first wave plate 222 , and is located between the lens 223 and the display screen 23 .
  • the reflective polarizer 221 , the first wave plate 222 , the lens 223 and the transflective film 224 are arranged in sequence.
  • the number of lenses 223 is one.
  • the number of lenses 223 may also be multiple, and the number of the reflective polarizer 221 , the first wave plate 222 and the transflective film 224 may not be limited to one.
  • the lens group 22 may be directly fixed to the lens body 11 of the lens frame 10 ( FIG. 2 ), or may be indirectly fixed to the lens body 11 of the lens frame 10 .
  • the lens group 22 may also be other types of lens groups 22 other than the Pancake lens group.
  • the first wave plate 222 is attached to the surface of the lens 223 on the light-incident side
  • the transflective film 224 is attached to the surface of the lens 223 on the light-exit side
  • the reflective polarizer 221 is attached to the first wave plate 222 faces away from the surface of lens 223 . That is to say, the reflective polarizer 221 , the first wave plate 222 and the transflective film 224 are directly or indirectly fixed to the lens 223 , and the lens 223 is fixed to the mirror body 11 , avoiding the need to use other brackets to fix the reflective polarizer 221 , the first wave plate 222 and the semi-reflective and semi-transparent film 224 are beneficial to the miniaturization of the lens group 22 .
  • the reflective polarizer 221 , the first wave plate 222 and the transflective film 224 can be understood as the multi-layer film structure in the Pancake lens group, the purpose is to make the light fold between the film layers.
  • the first wave plate 222 , the reflective polarizer 221 and the transflective film 224 may also be fixed to the lens body 11 through other fixing structures.
  • the reflective polarizer 221 is a reflective polarizer (Reflective Polarizer, RP), and the first wave plate 222 is a quarter wave-plate (Quarter Wave-plate, QWP).
  • the function of the polarized reflective film is to transmit polarized light whose polarization direction is perpendicular to the x-axis direction, and reflect polarized light whose polarization direction is parallel to the x-axis direction, that is, the reflective polarizer 221 can reflect the polarized light in the x-axis direction and transmit the y-axis direction. of polarized light. That is, the polarization direction of the reflective polarizer 221 is along the y-axis direction.
  • a quarter wave plate is a birefringent single crystal wave plate with a certain thickness.
  • the fast axis direction of the first wave plate 222 is 45 degrees from the y axis.
  • the transmittance of the transflective film 224 in the first wavelength band is greater than 95%, and the reflectivity is less than 1%, wherein the first wavelength band is the frequency band where the near-infrared light is located, that is, the frequency band of the first wavelength band is 850 nm ⁇ 20nm. That is to say, when the near-infrared light source 212 emits light, that is, when the first light is near-infrared light, when the first light passes through the transflective film 224, most of the light can be emitted from the transflective film 224, and the light A very small part of it will be reflected.
  • this solution can effectively reduce the light reflected by the transflective film 224 and finally enter the camera module 24, thereby reducing the ghost image formed, and on the other hand, it can reduce the loss of the first light intensity, so that the final camera module
  • the intensity of the first light rays received by the group 24 is large enough that the camera module 24 can identify the clear first light rays, which is beneficial to the algorithm technology of the eye tracking device, improves the accuracy of eye tracking, and further improves the user experience.
  • the transflective film 224 is transflective in the second wavelength band, that is, the transmissivity of the transflective film 224 in the second wavelength band is 50%, and the reflectivity is 50%.
  • the second wavelength band is the wavelength band where the visible light (or the light emitted by the display screen 23 ) is located, that is, the frequency band of the second wavelength band is 400 nm ⁇ 720 nm.
  • Half of the first light from the visible light source 211 is transmitted through the transflective film 224 and enters the camera module 24 .
  • Half of the second light from the display screen 23 is reflected by the transflective film 224, and half is transmitted through the transflective film, resulting in a fold in the lens group 22, which increases the optical path of the second light but does not increase the strength of the augmented reality component 20.
  • the thickness is beneficial to the miniaturization of the augmented reality component 20 .
  • the camera module 24 is a near-infrared light and visible light double-pass camera module 24, that is, the camera module 24 can shoot near-infrared light and can also shoot visible light.
  • the camera module 24 may include a lens 241 , an image sensor 242 for sensing the first light, and an optical element 243 .
  • the optical element 243 is provided on the light incident side of the lens 241 .
  • the image sensor 242 is a photosensitive chip, and the camera module 24 is fixed to the lens body 11 .
  • the camera module 24 can be directly fixed to the mirror body 11 , or can be indirectly fixed to the mirror body 11 .
  • the optical element 243 includes a first linear polarizer 2431 and a second wave plate 2432 , the first linear polarizer 2431 is arranged on the light incident side of the lens 241 , and the second wave plate 2432 is arranged on the back of the first linear polarizer 2431 To the side of the lens 241. That is, the first linear polarizer 2431 is located between the lens 241 and the second wave plate 2432 .
  • the polarization direction of the reflective polarizer 221 is perpendicular to the polarization direction of the first linear polarizer 2431 .
  • the first linear polarizer 2431 is a polarizing film.
  • the second wave plate 2432 is a quarter wave plate.
  • the first linear polarizer 2431 can be fixed to the surface of the lens 241 on the light incident side by bonding
  • the second wave plate 2432 can be fixed to the surface of the first linear polarizer 2431 facing away from the lens 241 by bonding.
  • first linear polarizer 2431 and the second wave plate 2432 may also be fixed on the light incident side of the lens 241 through a snap ring.
  • first linear polarizer 2431 and the second wave plate 2432 can also be screwed and fixed to the lens 241 through an intermediate piece.
  • first linear polarizer 2431 and the second wave plate 2432 may also be fixed to the lens body 11 of the lens frame 10 .
  • the polarization direction of the reflective polarizer 221 is the same as the polarization direction of the first linear polarizer 2431 .
  • the optical element 243 includes a first circular polarizer, the first circular polarizer and the first wave plate 222 have the same rotation direction, and the polarization of the first circular polarizer and the reflective polarizer 221 The directions are the same or perpendicular to each other. That is, the first circular polarizer can replace the first linear polarizer 2431 and the second wave plate 2432 .
  • the optical axis direction of the camera module 24 is defined as the z' axis
  • the opposite direction of the x axis is defined as the x' axis
  • the direction perpendicular to the optical axis direction of the camera module 24 is defined as y ' axis
  • the x' axis direction, the y' axis direction and the z' axis direction are perpendicular to each other in pairs.
  • the optical axis of the first linear polarizer 2431 is along the y' axis direction, the angle between the fast axis direction of the second wave plate 2432 and the polarization direction of the first linear polarizer 2431 is 45 degrees, that is, the fast axis of the second wave plate 2432 Orientation is 45 degrees from the x' axis.
  • the process may be an eyeball image captured by eye tracking, or an eyeball image captured by identity recognition.
  • FIG. 5 shows the polarization states of the first light 210 a reflected by the eyeball after passing through different elements in the process of reaching the image sensor 242 of the camera module 24 .
  • the first light 210a may be visible light emitted by the visible light source 211 , or may be near-infrared light emitted by the near-infrared light source 212 .
  • the light emitted by the visible light source or the near-infrared light source is natural light, and the natural light includes polarized light vibrating in various directions.
  • the first light 210a After passing through the reflective polarizer 221, the first light 210a is modulated into linearly polarized light whose polarization direction is perpendicular to the x-axis.
  • the direction of the fast axis of the first wave plate 222 is 45 degrees from the x-axis.
  • the polarization state of the first light 210 a is modulated into left-handed polarized light.
  • the first light ray 210 a is still left-handed polarized light after passing through the transflective film 224 .
  • the left-handed polarized light is modulated by the second wave plate 2432, it becomes linearly polarized light with the polarization direction along the x' axis direction.
  • the light transmission axis of the first linear polarizer 2431 is along the x' axis direction, so the linearly polarized light will pass through the first linear polarizer 2431 and be focused on the image sensor 242 through the lens 2412 of the camera module 24, that is, the camera module 24
  • the light reflected by the user's eyeball 200 is captured to form an image.
  • the transflective film 224 in this embodiment has high transmission and low reflection in the first wavelength band (near-infrared light 850nm ⁇ 20nm). Therefore, when the first light 210a is near-infrared light, the transflective film 224 is equivalent to a parallel flat plate , the stray light or ghost image caused by the surface reflection of the transflective film 224 is relatively slight.
  • the transflective film 224 is transflective in the second wavelength band (visible light 400nm-720nm), and when the first light 210a is visible light, the first light 210a is half transmitted through the transflective film 224 and enters the camera module 24 .
  • this embodiment can also effectively eliminate the light emitted after the lens group 22 is folded, thereby greatly reducing the brightness of ghost images.
  • the light emitted by the visible light source or the near-infrared light source is natural light, and after being reflected by the user's eyeball 200 , the light 210 b is formed, which is still natural light.
  • the light 210b After passing through the reflective polarizer 221, the light 210b is modulated into linearly polarized light whose polarization direction is perpendicular to the x-axis.
  • the direction of the fast axis of the first wave plate 222 is 45 degrees from the x-axis.
  • the polarization state of the light 210 b is modulated to be left-handed polarized light. Since the transflective film 224 does not change the polarization state of the light 210b, part of the light 210b is reflected by the transflective film 224 to form right-handed polarized light. After that, the light 210b passes through the second wave plate 2432 and is modulated into polarized light. Linearly polarized light oriented at 90° to the x' axis direction.
  • the light 210b is absorbed by the first linear polarizer 2431, that is, the folded light 210b will be absorbed by the first linear polarizer 2431 and will not be absorbed by the first linear polarizer 2431.
  • the image captured on the image sensor 242 of the camera module 24 cannot be superimposed with the image formed by the first light 210a, which can effectively eliminate ghost images and improve the accuracy of eye tracking algorithms or identity recognition.
  • the first linear polarizer 2431 if the light transmission axis of the first linear polarizer 2431 is the same as the light transmission axis of the reflective polarizer 221, the first light 210a will be blocked by the first linear polarizer 2431 and will not be blocked by the camera module
  • the group 24 senses, and the light 210b passes through the first linear polarizer 2431 and is sensed by the camera module 24 . Therefore, the light 210b reflected by the eyeball is folded once between the reflective polarizer 221 and the transflective film 224. Therefore, the camera module 24 can be tilted at a smaller angle than the case where the user's eyeball is shot directly through the lens group 22. The angle of the user's eyeball is captured, which facilitates the camera module 24 to capture the light 210b, reduces the aberration and distortion of the camera module 24, and improves the accuracy of eye tracking or identification.
  • the display screen 23 includes but is not limited to a fast liquid crystal display (Fast liquid crystal display, Fast LCD), a Micro-OLED, a liquid crystal on silicon (Liquid Crystal on Silicon, LCOS), a digital light processor (Digital Light Processing, DLP), Micro-LED, etc.
  • the display screen 23 will be inserted into black processing, that is, the display screen 23 will be in the lit state only about 20% of the time in the display period, and the period is: During the display period, the display screen 23 is in a closed state at other times, and this period is a black insertion period.
  • the controller 25 is electrically connected to the display screen 23 to obtain whether the display screen 23 is in the black insertion period or the display period.
  • the controller 25 can control the visible light source 211 or the near-infrared light source 212 to light up, and when the display screen 23 is in the black-insertion period, control the camera module 24 to perform Exposure is performed to obtain the first light.
  • the controller 25 controls the camera module 24 to stop exposure, and forms an eyeball image according to the first light.
  • the camera module 24 Since the camera module 24 is exposed when the display screen 23 is in the black insertion period, and stops being exposed when it is in the display period, the first light will be sensed by the camera module 24, and the second light will not be sensed by the camera module 24, which is effective The stray light from the display screen 23 is prevented from entering the camera module 24, so that the obtained eyeball image is clearer, and the accuracy of eye tracking or identification is improved.
  • the augmented reality component 20 further includes a second linear polarizer 26 and a third wave plate 27 , the second linear polarizer 26 is arranged on the light-emitting side of the display screen 23 , and the third wave plate 27 is arranged on the second linear polarizer 26 and the half-wave plate 27 . between the reverse semipermeable membrane 224 .
  • the second linear polarizer 26 is a polarizing film
  • the third wave plate 27 is a quarter wave plate.
  • the second linear polarizer 26 can be glued and fixed to the display surface of the display screen 23
  • the third wave plate 27 can be glued and fixed to the surface of the second linear polarizer 26 facing away from the display screen 23 .
  • the second linear polarizer 26 and the third wave plate 27 may also be sequentially fixed to the side of the display screen 23 toward which the display surface faces through an intermediate member.
  • the augmented reality component 20 further includes a second circular polarizer, and the second circular polarizer is disposed between the transflective film 224 and the display screen 23 , that is, the second circular polarizer can replace the second linear polarizer 26 and the third wave plate 27.
  • FIG. 7 is a schematic diagram of a second light path of the structure shown in FIG. 3 .
  • FIG. 7 shows the polarization states of the second light emitted by the display screen 23 after passing through different elements in the process of reaching the user's eyeball.
  • the second light from the display screen 23 is folded in the lens group 22 and finally exits into the user's eyeball.
  • the second light 230 emitted from the display screen 23 first passes through the second linear polarizer 26 and is then modulated into linearly polarized light.
  • the polarization direction can be set along the y-axis direction.
  • the second light 230 becomes right-handed polarized light, and the direction of the fast axis of the third wave plate 27 is 45 degrees from the y-axis.
  • the second light 230 reaches the transflective film 224, a part of the second light 230 is reflected, the other part of the second light 230 is transmitted, and passes through the first wave plate 222 to reach the reflective polarizer 221.
  • the fast axis direction of the first wave plate 222 is the same as that of the third wave plate 27.
  • the second light beam 230 is re-modulated into linearly polarized light, and the polarization direction is along the x-axis direction.
  • the reflective polarizer 221 can reflect the polarized light in the x-axis direction and transmit the polarized light in the y-axis direction. Therefore, the second light 230 is reflected, and reaches the transflective film 224 through the first wave plate 222 .
  • the second light 230 is right-handed polarized light. As before, a portion of the second light ray 230 is transmitted, and another portion of the second light ray 230 is reflected.
  • the polarization state of the second light ray 230 becomes left-handed polarized light, and after passing through the first wave plate 222, the second light ray 230 is modulated into linearly polarized light again, and the polarization direction is along the y-axis direction. According to the characteristics of the reflective polarizer 221, the second light 230 will be emitted through the reflective polarizer 221 and finally enter the eyeball.
  • the second light ray 230 emitted by the display screen 23 is folded in the lens group 22 and finally enters the eyeball, and passes through the lens group 22 to fold the second light ray 230 emitted by the display screen 23 .
  • the optical path of the second light ray 230 is increased without increasing the thickness of the augmented reality component 20 , which is beneficial to the miniaturization of the augmented reality component 20 .
  • the second light 230 when the second light 230 reaches the transflective film 224, a part of the second light 230 is transmitted and finally enters the eyeball.
  • the third wave plate 27 forms linearly polarized light, and the polarization direction is along the x-axis direction. According to the characteristics of the second linear polarizer 26, the second light 230 is intercepted by the second linear polarizer 26 (the part of the light path represented by the dotted arrow in FIG. 7).
  • the second linear polarizer 26 and the third wave plate 27 between the display screen 23 and the lens group 22, the second light 230 emitted from the display screen 23 can be intercepted and returned to the display screen 23 after being reflected by the lens group 22. part, the ghost phenomenon caused by the reflected light in the augmented reality component 20 is reduced, so that the user can watch a clearer picture.
  • the second light 230 formed by the reflection of the transflective film 224 is also partially reflected to the second wave plate 2432, and modulated by the second wave plate 2432 to become linearly polarized light with the polarization direction along the x' axis direction, the first The transmission axis of the linear polarizer 2431 is along the x′ axis direction, so the linearly polarized light will enter the camera module 24 through the first linear polarizer 2431 .
  • the controller 25 controls the camera module 24 to expose when the display screen 23 is in the black insertion period, and stops exposure when the display screen 23 is in the display period, so in the process of eye tracking and identification, the first position of the display screen 23
  • the two light rays 230 will not be sensed by the camera module 24 , which effectively prevents stray light from the display screen 23 from entering the camera module 24 and improves the accuracy of eye tracking or identification.
  • the camera module can also be disposed on the light incident side of the lens group and disposed toward the lens group.
  • the second light part is transmitted through the lens group and enters the camera module.
  • the camera module can also be embedded inside the display screen and set towards the lens group. In this case, when the display screen is in the display period, the light emitted by the display screen will not enter the camera module, and the augmented reality device may not.
  • the controller can not control the exposure of the camera module during the black insertion period, and stop the exposure during the display period to prevent the stray light from the display screen from entering the camera module.
  • FIG. 9 is a schematic flowchart of a control method of an electronic device 100 provided by an embodiment of the present application.
  • the control method is applied to the electronic device 100 shown in FIG. 3 .
  • the control method of the electronic device 100 includes the following S110 to S130.
  • S110 Control the display screen 23 to display according to a display period, and the display period includes alternately arranged display periods and black insertion periods.
  • the display screen 23 in order to eliminate the smear phenomenon of the screen when the head is moved, the display screen 23 is subjected to black insertion processing, that is, the display screen 23 is only in the lit state for about 20% of the time in the display period, This period is a display period, and the display screen 23 is in a closed state at other times, and this period is a black insertion period.
  • this embodiment takes a display screen with a frame rate of 90fps as an example, the interval between two Vsync signals is the displayed frame interval, which is about 11.1ms, and within 8.9ms after the Vsync signal, the display screen 23 In the off state, the display screen 23 is lit for about 2.2ms of the last frame interval, and then enters the next frame period. That is to say, the display screen 23 has a black insertion period and a lighting period in sequence in one frame, and then the next frame also has a black insertion period and a lighting period in sequence, and the cycle repeats.
  • S120 Control the camera module 24 to expose during the black insertion period to obtain the first light, and stop the exposure during the display period.
  • the light source is turned on.
  • the controller 25 responds to the user identification instruction or to the eye tracking instruction.
  • the user identification instruction and the eye tracking instruction may be issued to the controller 25 through the central processing unit of the electronic device 100 .
  • the controller 25 may also acquire the state of the electronic device 100 , and enable the user identification instruction or the eye tracking instruction according to the state of the electronic device 100 .
  • the controller 25 when the controller 25 responds to the user identification instruction, it controls the visible light source 211 to emit first light, the camera module 24 is exposed during the black insertion period, and stops exposure during the display period.
  • the controller 25 controls the near-infrared light source 212 to emit the first light in response to the eye tracking instruction, the camera module 24 is exposed during the black insertion period and stops exposure during the display period.
  • the visible light source 211 and the near-infrared light source 212 are lit during the lighting period, and the lighting period covers the display period and the black insertion period, that is, , the visible light source 211 is in a long bright state when responding to the identification instruction, and the near-infrared light source 212 is in a long bright state when responding to the eye tracking command.
  • the lighting period covers at least one display period and at least one black insertion period.
  • the light source is lit during the lighting period, and the lighting period covers at least one black insertion period.
  • the light source can also be turned on when the camera module 24 is exposed, that is, the lighting of the light source and the exposure of the camera module are performed simultaneously.
  • the controller 25 when the controller 25 responds to the user identification instruction, it controls the visible light source 211 and the camera module 24 to light up and expose respectively during the black insertion period. Control the visible light source 211 and the camera module 24 to turn off and stop exposure, respectively, during the display period. That is to say, since the camera module 24 is not exposed when the display screen 23 is in the display period, it does not need to take a photo of the user's eyeball, and therefore does not need to light the visible light source 211 , which effectively saves energy.
  • the controller 25 when the controller 25 responds to the eye tracking instruction, it controls the near-infrared light source 212 and the camera module 24 to light up and expose during the black-insertion period, respectively.
  • the near-infrared light source 212 and the camera module 24 are controlled to turn off and stop exposure respectively during the display period. That is to say, since the camera module 24 is not exposed when the display screen 23 is in the display period, there is no need to take a photo of the user's eyeball, and therefore the near-infrared light source 212 does not need to be turned on, which effectively saves energy.
  • the lighting period may cover multiple adjacent black-insertion periods, and may also cover multiple spaced black-insertion periods.
  • the lighting period may also cover the black insertion period and a part of the display period before the black insertion period.
  • the camera module 24 controls the camera module 24 to expose during the black-insertion period of the display screen 23, and to stop exposure during the display period of the display screen 23, the second light of the display screen 23 will not be blocked by the second light of the display screen 23 during the process of eye tracking and identification.
  • the camera module 24 senses, effectively preventing stray light from the display screen 23 from entering the camera module 24, and effectively improving the accuracy of eye tracking or identification.
  • S130 Form an eyeball image according to the first light ray.
  • the camera module 24 when the controller 25 responds to the identification instruction, the camera module 24 is exposed to obtain a first eyeball image, and the first eyeball image is used for sclera identification; when the controller 25 responds to the eye tracking instruction, the The camera module 24 is exposed to obtain a second eyeball image, and the second eyeball image is used for eye tracking.
  • the eyeball image (the first eyeball image or the second eyeball image) may be the image obtained by the camera module 24 exposed during the black insertion period in one frame, or may be the image obtained by the camera module 24 inserted in multiple frames.
  • the black insertion period in the multiple frames may be a plurality of adjacent black insertion periods, or may be a plurality of spaced black insertion periods.
  • the camera module 24 is exposed during the black insertion period of the first frame, then stops exposure during the display period of the first frame, and then continues to expose during the black insertion period of the second frame to obtain an eyeball image, which is then the camera image.
  • the module 24 accumulates images formed by exposure in two adjacent black insertion periods.
  • the camera module 24 can also be exposed during the black insertion period of the first frame, stop exposure during the display period of the first frame, not expose in the second frame, and then expose during the black insertion period of the third frame to obtain an eyeball image , then the eyeball image is the image formed by the cumulative exposure of the camera module 24 in two intervals of black insertion periods.
  • the control method further includes comparing the first eyeball image with the pre-stored image, and if the two match, calling the pre-stored eye movement calibration data in the database, and the pre-stored eye movement calibration data corresponds to the pre-stored image.
  • the eye movement calibration data is used for the eye tracking algorithm, and the eye gaze point position is output according to the eye tracking algorithm, and the eye gaze point position is used to assist in processing the display data, that is, gaze point rendering. If the first eyeball image does not match the pre-stored image, the user's eye movement is calibrated to obtain eye movement calibration data.
  • the eye movement calibration data is used for the eye tracking algorithm, and the eye gaze point position is output according to the eye tracking algorithm.
  • the eye gaze point position is used to assist in the processing of display data, that is, gaze point rendering, and the eye movement calibration data is stored in the eye movement calibration database. , for next use.
  • control method can be applied to compare the first eyeball image with the pre-stored image when the electronic device is turned on and in the boot interface, and if the two match, unlock the electronic device to allow the electronic device to enter Main interface. If the first eyeball image does not match the pre-stored image, the unlock password will pop up. The control method unlocks the electronic device quickly and improves user experience.
  • the control method provided by the present application can register the sclera information when the user uses the electronic device 100 for the first time by identifying the user, and save the parameters of the eye movement calibration in the personal file.
  • the user uses the electronic device 100 again, After the system confirms the user's identity through scleral identification, it directly calls the user's eye movement calibration parameters, and the user does not need to perform eye movement calibration again, which saves the user's time and greatly improves the user experience.
  • identity recognition can also be used in scenarios in which the electronic device 100 is in use, for example, scleral recognition payment is used instead of password payment.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

本申请实施例提供一种电子设备的控制方法及电子设备。所述电子设备包括光源、透镜组、显示屏和摄像模组，所述光源用于向用户的眼球发射第一光线，所述第一光线经所述眼球反射，透过所述透镜组进入摄像模组；所述显示屏发射的第二光线部分经所述透镜组反射或透射进入摄像模组，所述控制方法包括：控制所述显示屏按照显示周期进行显示，所述显示周期包括交替排布的显示时段和插黑时段；控制所述摄像模组在所述插黑时段曝光，以获得所述第一光线，且在所述显示时段停止曝光；依据所述第一光线形成眼球图像。该控制方法能够有效提高用户体验。

Description

电子设备及其控制方法
本申请要求于2021年03月26日提交中国专利局、申请号为202110328563.3、申请名称为“电子设备及其控制方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及电子设备技术领域，特别涉及一种电子设备及其控制方法。
背景技术
用户在使用例如头戴显示设备等类似的电子设备时，每次使用头戴显示设备前均需要进行眼动校准，以校准得到眼球的视轴。但是眼动校准时间较长，在校准时间内用户无法使用设备，用户体验较差。
发明内容
本申请实施例提供一种电子设备的控制方法及电子设备。该控制方法能够有效提高用户体验。
第一方面，提供了一种电子设备的控制方法。所述电子设备包括光源、透镜组、显示屏和摄像模组，所述光源用于向用户的眼球发射第一光线，所述第一光线经所述眼球反射，透过所述透镜组进入摄像模组；所述显示屏发射的第二光线部分经所述透镜组反射或透射进入摄像模组，所述控制方法包括：
控制所述显示屏按照显示周期进行显示，所述显示周期包括交替排布的显示时段和插黑时段；
控制所述摄像模组在所述插黑时段曝光，以获得所述第一光线，且在所述显示时段停止曝光；
依据所述第一光线形成眼球图像。
可以理解的是，本申请的光源、透镜组和摄像模组构成身份识别装置，以对用户进行身份识别，可以在用户首次使用电子设备时注册其眼球图像信息，并将其眼动校准的参数保存在个人档案中，当用户再次使用电子设备时，系统通过巩膜识别确认用户的身份后，直接调用用户的眼动校准参数，用户可以不用再次进行眼动校准，节约了用户的时间，大大提升用户体验。同时本申请通过控制摄像模组在显示屏处于插黑时段曝光，而在显示屏处于显示时段停止曝光，从而在身份识别过程中，显示屏的第二光线不会被摄像模组感测，防止来自显示屏的杂散光进入摄像模组，有效提高身份识别的精度。
一种可能的实现方式中，在控制所述摄像模组曝光之前，所述控制方法还包括点亮光源，从而摄像模组在曝光时就能接收到第一光线。
一种可能的实现方式中，所述点亮光源包括在点亮时段点亮光源，所述点亮时段覆盖至少一个所述显示时段和至少一个所述插黑时段；或者所述点亮时段覆盖至少一个所述插黑时段。也就是说，光源可以处于长亮状态，也可以仅在插黑时段显示，即光源在摄像模组曝光时点亮，在摄像模组不曝光时关闭，能够有效节约能源。
一种可能的实现方式中，所述光源为可见光源，所述眼球图像用于巩膜识别。由于眼球的眼纹在可见光照明下的对比度较高，因此巩膜识别采用可见光照明能提高身份识别的精度。
一种可能的实现方式中，所述光源包括可见光源和近红外光源，所述摄像模组为近红外 光和可见光双通摄像模组，所述控制方法还包括：
响应于用户识别指令，控制所述可见光源发出所述第一光线，所述摄像模组曝光，以获得第一眼球图像，所述第一眼球图像用于巩膜识别；
响应于眼动追踪指令，控制所述近红外光源发出所述第一光线，所述摄像模组曝光，以获得第二眼球图像，所述第二眼球图像用于眼动追踪。
本申请的控制方法可以摄像模组为近红外光和可见光双通摄像模组，眼动追踪和用户识别均可使用同一个摄像模组，有效节省成本及电子设备的空间。
一种可能的实现方式中，所述控制方法还包括：将所述第一眼球图像与预存图像进行对比；若两者匹配，则调用预存眼动校准数据，所述预存眼动校准数据与所述预存图像对应。
本申请提供的控制方法通过对用户进行身份识别，可以在用户首次使用电子设备时注册其巩膜信息，并将其眼动校准的参数保存在个人档案中，当用户再次使用电子设备时，系统通过巩膜识别确认用户的身份后，直接调用用户的眼动校准参数，用户可以不用再次进行眼动校准，节约了用户的时间，大大提升用户体验。当然，身份识别还能用于电子设备在使用过程中的场景，例如用巩膜识别支付代替密码支付。
一种可能的实现方式中，所述控制方法应用于电子设备开启且处于开机界面时，所述控制方法还包括：将所述第一眼球图像与预存图像进行对比；若两者匹配，则解锁所述电子设备，以使所述电子设备进入主界面。本申请的控制方法还可以同于电子设备的解锁，通过巩膜识别解锁速度快，提高用户体验。
一种可能的实现方式中，所述控制方法还包括：若所述第一眼球图像与所述预存图像不匹配，则对用户的眼动进行校准，获取眼动校准数据，并将眼动校准数据存入眼动校准数据库中，以供下次使用。
第二方面，提供了一种电子设备。所述电子设备包括光源、透镜组、摄像模组、显示屏和控制器，所述显示屏设于所述透镜组的出光侧，所述光源设于所述透镜组远离所述显示屏的一侧，所述摄像模组的入光侧朝向所述透镜组；所述显示屏的显示周期包括交替排布的显示时段和插黑时段；所述光源用于向用户的眼球发射第一光线，所述第一光线经所述眼球反射，透过所述透镜组进入摄像模组；所述显示屏发射的第二光线部分经所述透镜组反射或透射进入摄像模组；所述光源、所述摄像模组和所述显示屏均与所述控制器电连接，所述控制器用于控制所述摄像模组在所述插黑时段曝光，以获得所述第一光线，且在所述显示时段停止曝光，并依据所述第一光线形成眼球图像。
可以理解的是，电子设备具有身份识别功能，本申请通过控制器控制摄像模组在插黑时段曝光，而在显示屏处于显示时段停止曝光，从而在身份识别过程中，显示屏的第二光线不会被摄像模组感测，有效防止来自显示屏的杂散光进入摄像模组，以使得到的眼球图像更加清晰，有效提高身份识别的精度。
一种可能的实现方式中，所述光源为可见光源。由于眼球的眼纹在可见光照明下的对比度较高，因此巩膜识别采用可见光照明能提高身份识别的精度。
一种可能的实现方式中，所述光源包括可见光源和近红外光源，所述摄像模组为近红外光和可见光双通摄像模组，从而眼动追踪和用户识别均可使用同一个摄像模组，有效节省成本及电子设备的空间。
一种可能的实现方式中，所述透镜组包括透镜和半反半透膜，所述半反半透膜设于所述透镜的出光侧。来自显示屏的第二光线一半通过半反半透膜反射，一半透过半反半透膜，在透镜组中产生折叠，增长了第二光线的光路但是不会增加电子设备的厚度，有利于电子设备 的小型化。
一种可能的实现方式中，所述半反半透膜在第一波段的透过率大于95％，反射率小于1％，其中，所述第一波段为近红外光所在的频段。第一光线为近红外光，该方案一方面能有效减少经半反半透膜反射且最终进入摄像模组的光线，减少形成的鬼像，另一方面能减小第一光线强度的损耗，以使最终被摄像模组接收的第一光线的强度足够大，摄像模组能够识别清晰的第一光线，有利于眼动追踪装置进行算法技术，提高了眼动追踪的精度，进而提高用户体验。
一种可能的实现方式中，所述透镜组还包括反射偏振片和第一波片，所述第一波片设于所述透镜背向所述半反半透膜一侧，所述反射偏振片设于所述第一波片背向所述透镜一侧；所述摄像模组还包括第一线偏振片和第二波片，所述第一线偏振片位于所述第二波片与所述摄像模组的镜头之间，所述反射偏振片的偏振方向与所述第一线偏振片的偏振方向相同或相互垂直，所述第一波片与所述第二波片均为四分之一波片。反射偏振片、第一波片及半反半透膜可以理解为Pancake镜组中的多层膜结构，目的在于使得光线在膜层之间折叠。摄像模组包括第一线偏振片和第二波片，可以有效地消除经透镜组折叠后出射的光线，大大降低鬼像亮度。
一种可能的实现方式中，所述透镜组还包括反射偏振片和第一波片，所述第一波片设于所述透镜背向所述半反半透膜一侧，所述反射偏振片设于所述第一波片背向所述透镜一侧；所述摄像模组还包括第一圆偏振片，所述第一圆偏振片与所述第一波片的旋向相同，所述第一圆偏振片与所述反射偏振片的偏振方向相同或相互垂直。反射偏振片、第一波片及半反半透膜可以理解为Pancake镜组中的多层膜结构，目的在于使得光线在膜层之间折叠。摄像模组包括第一圆偏振片，可以有效地消除经透镜组折叠后出射的光线，大大降低鬼像亮度。
一种可能的实现方式中，所述电子设备还包括第二线偏振片和第三波片，所述第二线偏振片设于所述显示屏的出光侧，所述第三波片设于所述第二线偏振片和所述透镜组之间。本申请通过在显示屏和透镜组之间设置第二线偏振片和第三波片，能够拦截从显示屏发出的第二光线经透镜组反射后回到显示屏的部分，减少电子设备中由于反射光线产生的鬼影现象，使用户能够观看到更清晰的画面。
附图说明
为了更清楚地说明本申请实施例或背景技术中的技术方案，下面将对本申请实施例或背景技术中所需要使用的附图进行说明。
图1是相关技术中的头戴显示设备的校准示意图；
图2是本申请实施例提供的一种电子设备的结构示意图；
图3是图2所示结构的增强现实组件的结构示意图；
图4是眼球示意图；
图5是图3所示的增强现实组件的第一光线的传播路径示意图；
图6是图3所示的增强现实组件的另一光线的传播路径示意图；
图7是图3所示结构的第二光线的一种传播路径示意图；
图8是图3所示结构的第二光线的另一种传播路径示意图；
图9是本申请实施例提供的一种电子设备的控制方法的流程示意图；
图10是图3所示的增强现实组件的显示屏的显示示意图；
图11是图9所示的控制方法的控制示意图；
图12是图9所示的控制方法的另一种控制示意图；
图13是图9所示的控制方法的另一种控制示意图；
图14是图9所示的控制方法的另一种控制示意图；
图15是图9所示的控制方法的部分操作流程示意图。
具体实施方式
下面结合本申请实施例中的附图对本申请实施例进行描述。
在本申请实施例的描述中，需要说明的是，除非另有明确的规定和限定，术语“安装”、“连接”应做广义理解，例如，“连接”可以是可拆卸地连接，也可以是不可拆卸地连接；可以是直接连接，也可以通过中间媒介间接连接。其中，“固定连接”是指彼此连接且连接后的相对位置关系不变。“转动连接”是指彼此连接且连接后能够相对转动。本申请实施例中所提到的方位用语，例如，“左”、“右”、“内”、“外”等，仅是参考附图的方向，因此，使用的方位用语是为了更好、更清楚地说明及理解本申请实施例，而不是指示或暗指所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本申请实施例的限制。“多个”是指至少两个。
可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与发明相关的部分。
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
请参阅图1，图1是相关技术中的头戴显示设备的校准示意图。
相关技术中的头戴显示设备能对用户的进行眼动追踪，在使用头戴显示设备之前需要对用户的进行眼动校准。一般校准过程是，在显示屏上依次显示5～9个点，并提示用户依次看这些点。具体地，如图1所示，在显示屏上显示5个点，并用不同的颜色来标识引导用户看的点，例如图1中的正中间的点变成红色，然后依次沿箭头将下一个点换成红色，以提示用户去看。每个点变成红色的时长约2秒，加上校准前的说明教程，整个眼动校准的时长约20～40秒。由于用户每次佩戴电子设备之前都需要进行眼动校准，因此用户佩戴电子设备后半分钟内无法使用电子设备，用户体验较差。
本申请实施例提供一种电子设备，用于佩戴在用户的头部。电子设备可以为头戴显示设备，例如电子设备可以为增强现实(augmented reality，AR)眼镜、AR头盔、混合现实(mixrtual reality，MR)眼镜或MR头盔等将数字内容和现实场景结合在一起的电子产品。电子设备还可以不佩戴在头部。本申请的电子设备解决了相关技术中的电子设备用户体验差的问题。本申请以电子设备是AR眼镜为例进行具体说明。
请参阅图2，图2是本申请实施例提供的一种电子设备的结构示意图。
本实施例中，电子设备100包括镜架10以及安装于镜架10的增强现实组件20。其中，增强现实组件20有两个，两个增强现实组件20间隔安装于镜架10。
镜架10包括镜身11以及与镜身11连接的镜腿12。其中，镜腿12有两个，两个镜腿12连接于镜身11的相对两端。需要说明的是，在其他实施例中，镜架10也可以包括镜身11和与镜身11连接的固定带，本申请对此不作具体限定。
镜身11可以包括安装面110和眼罩111，眼罩111设于安装面110并与安装面110形成收容腔112。本实施例中，收容腔112的数量为两个，两个收容腔112分别用于收容电子设备100的电子元器件。示例的，眼罩111位于两个收容腔112之间的位置形成鼻托，以便于佩戴。镜身11可以为一体结构，以保证镜身11的整体强度。其中，镜身11的材料包括且不限于金 属、塑料、树脂或天然材料等。应当理解的是，镜身11不仅限于图2所示的结构，也可以为半框型或无框型的眼镜式框架，只要能够收容电子设备100的电子元器件即可。
两个镜腿12转动连接于镜身11的相对两端。在电子设备100处于展开状态(如图2所示)时，两个镜腿12通过相对镜身11转动至彼此相对，此时电子设备100的两个镜腿12可分别架设于用户的两个耳朵上，鼻托架设于用户的鼻梁上，以穿戴于用户的头部。佩戴时，用户的两个眼球分别与其对应的收容腔112相对，以便于观看位于收容腔112中的电子元器件呈现的画面。在电子设备100处于折叠状态时，两个镜腿12通过相对镜身11转动，至彼此至少部分地重叠且收容于镜身11的内侧，此时电子设备100可收纳起来。
示例的，眼罩111可以具有一定柔性，当用户佩戴电子设备100时，眼罩111与用户的皮肤接触，防止外界的光从用户的皮肤和眼罩111之间的缝隙透过，影响用户的观赏体验。同时，眼罩111为柔性，能够缓冲电子设备100的重力，给用户带来很好的佩戴体验。当然，在其他实施例中，眼罩111仅接触用户的部分为柔性。或者眼罩111也可以不为柔性。
可以理解的是，在其他实施例中，两个镜腿12可以分别固定连接于镜身11的相对两端。或者，两个镜腿12可以与镜身11一体成型，即电子设备100始终处于展开状态，本申请对此不作具体限定。需要说明的是，镜腿12的内部也可以设有收容腔，镜腿12的收容腔也可以收容电子设备100的电子元器件。
需要说明的是，本申请提及电子设备100时所采用“内侧”、“外侧”等方位用词主要依据电子设备100被用户佩戴于头部时的方位进行阐述。电子设备100被用户佩戴时，以靠近用户头部为内侧，以远离用户头部为外侧，其并不形成对电子设备100于其他场景中的方位的限定。
接下来，为了便于描述，如图2所示，定义电子设备100的长度方向为x轴方向，电子设备100的宽度方向为y轴方向，电子设备100的厚度方向为z轴方向，且x轴方向、y轴方向和z轴方向彼此两两垂直。其中，x轴方向即为收容腔112中一个收容腔112至另一个收容腔112的方向，z轴方向即为镜身11朝向镜腿12的方向。
本实施例中，两个增强现实组件20的结构相同。具体的，两个增强现实组件20分别安装于与其对应的镜身11的收容腔112内。即两个增强现实组件20沿x轴方向并排设置。电子设备100穿戴于用户头部时，一个增强现实组件20对应于用户的左眼，另一个增强现实组件20对应于用户的右眼，此时用户的双眼可以通过两个增强现实组件20观看虚拟场景和真实场景。需要说明的是，在其他实施例中，两个增强现实组件20的结构也可以不同，本申请对此不作具体限定。
接下来，为了便于理解，以与用户的右眼相对应的增强现实组件20为例对增强现实组件20的结构进行具体描述。请参阅图3，图3是图2所示结构的增强现实组件20的结构示意图。
增强现实组件20包括光源21、透镜组22、显示屏23、摄像模组24和控制器25。摄像模组24位于透镜组22和显示屏23之间，摄像模组24的入光侧朝向透镜组22，光源21设于透镜组22远离摄像模组24的一侧，光源21用于向用户的眼球发射第一光线，第一光线经眼球反射，透过透镜组22进入摄像模组24。显示屏23发射的第二光线部分经透镜组22反射进入摄像模组24。光源21、摄像模组24和显示屏23均与控制器25电连接，控制器25用于根据显示屏23的显示周期控制光源21和摄像模组24。
可以理解的是，两个增强现实组件20各自均有一个显示屏23，或者两个增强现实组件20共用一个显示屏23。
其中，光源21、透镜组22、摄像模组24和控制器25构成眼动追踪装置或者身份识别装 置，显示屏23和透镜组22构成显示装置。眼动追踪装置/身份识别装置、显示装置共用透镜组22。由于增强现实组件20与用户的眼球正对，可以通过巩膜识别对用户进行身份识别。巩膜识别又称眼纹识别，是通过用户眼球巩膜区域内的毛细血管分布来进行身份识别。图4是眼球示意图，其中灰色的弯曲短线即为巩膜区域内的毛细血管，又称为眼纹。
可以理解的是，本实施例中的增强现实组件20既能通过显示装置用于模拟虚拟环境。例如，可以在显示屏23上显示立体图像以模拟深度的错觉，即制造虚拟环境，使用户产生置身于该虚拟环境的错觉。增强现实组件20又能通过眼动追踪装置来估计用户正在查看虚拟环境的哪一部分。也就是说，增强现实组件20的显示装置能够为用户呈现虚拟环境的同时，眼动追踪装置能够检测用户目光注视的位置，即用户目光注视的虚拟环境的具体位置，并根据用户目光注视的位置更好的调整呈现给用户的虚拟环境，提高用户体验。增强现实组件20还能通过身份识别装置对用户进行身份识别，以便于减少相关校准步骤及提高用户体验。
具体的，光源21包括可见光源211和近红外光源212，可见光源211、透镜组22、摄像模组24和控制器25构成身份识别装置。近红外光源212、透镜组22、摄像模组24和控制器25构成眼动追踪装置。由于眼动追踪和巩膜识别均需要用户的眼球图像，眼动追踪和巩膜识别可以共用一个摄像模组24，有效节省成本及电子设备100的空间。
当电子设备100需要对用户进行眼动追踪时，控制器25控制近红外光源212点亮，第一光线为近红外光，被摄像模组24感测形成眼球图像。由于眼动追踪可能伴随大部分电子设备100的体验过程，眼动追踪时使用近红外光源212，用户察觉不到，提高用户体验。当电子设备100需要对用户进行身份识别时，控制器25控制可见光源211点亮，第一光线为可见光，被摄像模组24感测形成眼球图像。由于眼球的眼纹在可见光照明下的对比度较高，因此巩膜识别采用可见光照明能提高身份识别的精度。
当然，在其他实施例中，眼动追踪装置的光源还可以是可见光源，身份识别装置的光源可以是近红外光源。或者，眼动追踪装置和身份识别装置的光源均为可见光源。或者，眼动追踪装置和身份识别装置的光源均为近红外光源。
可以理解的是，可见光源211和近红外光源212可以是两个单独的光源。可见光源211和近红外光源212可以设于透镜组22背向显示屏23的一侧，便于对用户的眼球发射光线，以对用户进行眼动追踪或身份识别。具体的，光源21可以固定于镜身11或者镜腿12，或通过中间连接件固定于镜身11或者镜腿12(图2)。可见光源211和近红外光源212的数量都可以为一个或多个。
当然，在其他实施例中，可见光源和近红外光源还可以为一体的光源21，也就是说，光源既可以发射可见光，也可以近红外光，可通过控制器控制光源发射可见光或近红外光。或者，光源仅为可见光源。
图3所示的实施例中增强现实组件20仅示出一个可见光源211和一个近红外光源212，当然，在其他实施例中，可见光源211和近红外光源212的数量还可以是多个。当可见光源211和近红外光源212的数量为多个时，相比于可见光源211和近红外光源212的数量为一个的方案，多个可见光源211和近红外光源212中的单个光源21的发光强度可以设置得更弱，能够减小用户眼球受到过强光线照射而损伤的风险。可见光源211和近红外光源212还可以设置在增强现实组件20的其他位置，本申请不对可见光源211和近红外光源212的数量和设置位置进行限定。
本申请的电子设备100通过增加身份识别装置，可以在用户首次使用电子设备100时注册其巩膜信息，并将其眼动校准的参数保存在个人档案中，当用户再次使用电子设备100时， 系统通过巩膜识别确认用户的身份后，直接调用用户的眼动校准参数，用户可以不用再次进行眼动校准，节约了用户的时间，大大提升用户体验。当然，身份识别装置还能用于电子设备100在使用过程中的场景，例如用巩膜识别支付代替密码支付。
本实施例中，透镜组22为薄饼(Pancake)镜组，透镜组22包括反射偏振片221、第一波片222、透镜223和半反半透膜(Beam Splitter，BS)224，第一波片222设于透镜223的入光侧，可以理解的是，透镜223的入光侧为光源21的光线进入透镜223的一侧，反射偏振片221设于第一波片222背向透镜223的一侧，半反半透膜224设于透镜223的出光侧，也就是透镜223背向第一波片222的一侧，并位于透镜223和显示屏23之间。也就是说，在透镜223的光轴上，反射偏振片221、第一波片222、透镜223及半反半透膜224依次设置。示例的，透镜223的数量为一片，当然，透镜223的数量还可以是多片，且反射偏振片221、第一波片222和半反半透膜224的数量也可以不限于一片。透镜组22可以直接固定于镜架10的镜身11(图2)，也可以间接固定于镜架10的镜身11。在其他实施例中，透镜组22还可以是除Pancake镜组以外的其他类型的透镜组22。
本实施例中，第一波片222贴合于透镜223位于入光侧的表面，半反半透膜224贴合于透镜223位于出光侧的表面，反射偏振片221贴合于第一波片222背向透镜223的表面。也就是说，反射偏振片221、第一波片222及半反半透膜224均直接或间接贴合固定于透镜223，透镜223固定于镜身11，避免了使用其他支架固定反射偏振片221、第一波片222及半反半透膜224，有利于透镜组22的小型化。反射偏振片221、第一波片222及半反半透膜224可以理解为Pancake镜组中的多层膜结构，目的在于使得光线在膜层之间折叠。当然，在其他实施例中，第一波片222、反射偏振片221和半反半透膜224还可以通过其他固定结构固定于镜身11。
示例的，反射偏振片221为反射偏振膜(Reflective Polarizer，RP)，第一波片222是四分之一波片(Quarter Wave-plate，QWP)。其中，偏振反射膜的功能是透射偏振方向与x轴方向垂直的偏振光，反射偏振方向与x轴方向平行的偏振光，即反射偏振片221可以反射x轴方向的偏振光，透射y轴方向的偏振光。也就是说，反射偏振片221的偏振方向沿y轴方向。四分之一波片是具有一定厚度的双折射单晶波片。第一波片222的快轴方向与y轴成45度。
本实施例中，半反半透膜224在第一波段的透过率大于95％，反射率小于1％，其中，第一波段为近红外光所在的频段，即第一波段的频段为850nm±20nm。也就是说，当近红外光源212发射光线时，也就是第一光线为近红外光时，第一光线通过半反半透膜224时，大部分都能从半反半透膜224射出，光线的极少部分会被反射。该方案一方面能有效减少经半反半透膜224反射且最终进入摄像模组24的光线，减少形成的鬼像，另一方面能减小第一光线强度的损耗，以使最终被摄像模组24接收的第一光线的强度足够大，摄像模组24能够识别清晰的第一光线，有利于眼动追踪装置进行算法技术，提高了眼动追踪的精度，进而提高用户体验。
半反半透膜224在第二波段半反半透，即半反半透膜224在第二波段的透过率为50％，反射率为50％。其中，第二波段为可见光(或者说是显示屏23发出的光线)所在的波段，即第二波段的频段为400nm～720nm。来自可见光源211的第一光线一半通过半反半透膜224透射，进入摄像模组24。来自显示屏23的第二光线一半通过半反半透膜224反射，一半透过半反半透膜，在透镜组22中产生折叠，增长了第二光线的光路但是不会增加增强现实组件20的厚度，有利于增强现实组件20的小型化。
本实施例中，摄像模组24为近红外光和可见光双通摄像模组24，即摄像模组24能够拍 摄近红外光也能拍摄可见光。摄像模组24可以包括镜头241、感测第一光线的图像传感器242和光学元件243。光学元件243设于镜头241的入光侧。本实施例中，图像传感器242为感光芯片，摄像模组24固定于镜身11。摄像模组24可以直接固定于镜身11，也可以间接固定于镜身11。
如图3，光学元件243包括第一线偏振片2431和第二波片2432，第一线偏振片2431设于镜头241的入光侧，第二波片2432设于第一线偏振片2431背向镜头241的一侧。即第一线偏振片2431位于镜头241和第二波片2432之间。本实施例中，反射偏振片221的偏振方向与第一线偏振片2431的偏振方向垂直。第一线偏振片2431为偏振膜。第二波片2432为四分之一波片。示例的，第一线偏振片2431可以通过粘接固定于镜头241的入光侧所在的表面，第二波片2432可以通过粘接固定于第一线偏振片2431背向镜头241的表面。
当然，在其他实施场景中，第一线偏振片2431和第二波片2432还可以通过卡环固定于镜头241的入光侧。或者第一线偏振片2431和第二波片2432还可通过中间件螺接固定于镜头241。或者第一线偏振片2431和第二波片2432还可以固定于镜架10的镜身11。
当然，在其他实施例的一种实施场景中，反射偏振片221的偏振方向与第一线偏振片2431的偏振方向相同。在其他实施例的另一种实施场景中，光学元件243包括第一圆偏振片，第一圆偏振片与第一波片222的旋向相同，第一圆偏振片和反射偏振片221的偏振方向相同或相互垂直。也就是说，第一圆偏振片能够代替第一线偏振片2431和第二波片2432。
为了便于描述，如图5所示，定义摄像模组24的光轴方向为z’轴，定义x轴的反方向为x’轴，定义垂直于摄像模组24的光轴方向的方向为y’轴，且x’轴方向、y’轴方向和z’轴方向彼此两两垂直。第一线偏振片2431的光轴沿y’轴方向，第二波片2432的快轴方向与第一线偏振片2431的偏振方向夹角为45度，即，第二波片2432的快轴方向与x’轴成45度。
为了清晰地描述摄像模组24拍摄用户眼球的过程，该过程可以是眼动追踪拍摄的眼球图像，也可以是身份识别拍摄的眼球图像。图5是眼球反射的第一光线210a到摄像模组24的图像传感器242的过程中经过不同元件后的偏振状态。第一光线210a可以是可见光源211发出的可见光，也可以是近红外光源212发出的近红外光。
可见光源或近红外光源出射的光线为自然光，自然光包括各个方向振动的偏振光，自然光经过用户眼球200反射后形成第一光线210a，仍为自然光。经过反射偏振片221后，第一光线210a被调制为偏振方向与x轴垂直的线偏光。第一波片222的快轴方向与x轴成45度，经过第一波片222后第一光线210a的偏振态被调制为左旋偏振光。由于半反半透膜224不改变第一光线210a的偏振态，因此第一光线210a穿过半反半透膜224后仍为左旋偏振光。左旋偏振光经过第二波片2432调制后，变成偏振方向沿x’轴方向的线偏光。第一线偏振片2431的透光轴沿x’轴方向，因此线偏光会穿过第一线偏振片2431，并经由摄像模组24的镜头2412聚焦在图像传感器242上，即摄像模组24拍摄到了用户眼球200反射的光线，形成图像。
本实施例中的半反半透膜224在第一波段(近红外光850nm±20nm)高透低反，因此第一光线210a为近红外光时，半反半透膜224相当于一片平行平板，由半反半透膜224表面反射造成的杂散光或鬼像较为轻微。半反半透膜224在第二波段(可见光400nm～720nm)半反半透，当第一光线210a为可见光时，第一光线210a一半透过半反半透膜224进入摄像模组24。
与此同时，如图6所示，本实施例还可以有效地消除经透镜组22折叠后出射的光线，大大降低鬼像亮度。在本实施例中，可见光源或近红外光源出射的光线为自然光，经过用户眼球200反射后，形成光线210b，仍为自然光。经过反射偏振片221后，光线210b被调制为 偏振方向与x轴垂直的线偏光。第一波片222的快轴方向与x轴成45度，经过第一波片222后光线210b的偏振态被调制为左旋偏振光。由于半反半透膜224不改变光线210b的偏振态，光线210b部分经半反半透膜224反射形成右旋偏振光，此后，光线210b穿过第二波片2432，并被其调制为偏振方向与x’轴方向呈90°的线偏光。由于该偏振方向与第一线偏振片2431的透光轴方向垂直，因此该光线210b被第一线偏振片2431吸收，即经过折叠的光线210b会被第一线偏振片2431吸收掉，不会打在摄像模组24的图像传感器242上，无法与第一光线210a形成的像叠加在一起，能有效消除鬼像，提高眼动追踪算法或身份识别的精确度。
当然，在其他实施例中，若第一线偏振片2431的透光轴与反射偏振片221的透光轴相同，则第一光线210a会被第一线偏振片2431阻挡，不会被摄像模组24感测，而光线210b会透过第一线偏振片2431，被摄像模组24感测。从而眼球反射的光线210b在反射偏振片221和半反半透膜224之间折叠了一次，因此相比于直接透过透镜组22拍摄用户眼球的情况，摄像模组24可以以更小的倾斜角度拍摄到用户的眼球，便于摄像模组24拍摄光线210b，减小摄像模组24的相差和畸变，提高眼动追踪或身份识别的精度。
本实施例中，显示屏23包括但不限于快速液晶显示屏(Fast liquid crystal display,Fast LCD)，Micro-OLED、基于硅基液晶(Liquid Crystal on Silicon,LCOS)、数字光处理器(Digital Light Processing,DLP)、Micro-LED等。
可以理解的是，为了消除用户头动时画面的拖影现象，会对显示屏23进行插黑处理，即在显示周期内仅约20％的时间内显示屏23处于点亮状态，该时段为显示时段，其他的时间显示屏23处在关闭的状态，该时段为插黑时段。控制器25通过与显示屏23电连接，以获取显示屏23处于插黑时段还是处于显示时段。电子设备100需要对用户进行眼球追踪或者需要对用户进行身份识别时，控制器25可以控制可见光源211或近红外光源212点亮，当显示屏23处于插黑时段时，控制摄像模组24进行曝光，以获得第一光线，当显示屏23处于显示时段，控制器25控制摄像模组24停止曝光，并依据第一光线形成眼球图像。
由于摄像模组24在显示屏23处于插黑时段时曝光，处于显示时段时停止曝光，因此第一光线会被摄像模组24感测，第二光线不会被摄像模组24感测，有效防止来自显示屏23的杂散光进入摄像模组24，以使得到的眼球图像更加清晰，提高眼动追踪或身份识别的精度。
如图3，增强现实组件20还包括第二线偏振片26和第三波片27，第二线偏振片26设于显示屏23的出光侧，第三波片27设于第二线偏振片26和半反半透膜224之间。本实施例中，第二线偏振片26为偏振膜，第三波片27为四分之一波片。第二线偏振片26可以粘接固定于显示屏23的显示面，第三波片27可以粘接固定于第二线偏振片26背向显示屏23的表面。
当然，在其他实施例中，第二线偏振片26和第三波片27还可以通过中间件依次固定于显示屏23的显示面朝向的一侧。或者，增强现实组件20还包括第二圆偏振片，第二圆偏振片设于半反半透膜224和显示屏23之间，也就是说，第二圆偏振片可以替换第二线偏振片26和第三波片27。
为了清晰地描述用户观看显示屏23显示画面的过程，请参阅图7，图7是图3所示结构的一种第二光线路径示意图。图7给出了显示屏23发射的第二光线到用户的眼球的过程中经过不同元件后的偏振状态。如图7所示，来自显示屏23的第二光线在透镜组22中进行折叠，最终出射到用户的眼球中。
具体的，显示屏23出射的第二光线230先经过第二线偏振片26后被调制为线偏光，不失一般性地，可设偏振方向沿y轴方向。经过第三波片27后第二光线230变为右旋偏振光，第三波片27的快轴方向与y轴成45度。此后，第二光线230到达半反半透膜224，第二光 线230的一部分反射，第二光线230的另一部分透射，并穿过第一波片222到达反射偏振片221。第一波片222的快轴方向与第三波片27相同，此时，第二光线230重新被调制为线偏光，偏振方向沿x轴方向。反射偏振片221可以反射x轴方向的偏振光，透射y轴方向的偏振光。因此，第二光线230被反射，并透过第一波片222到达半反半透膜224，此时，第二光线230为右旋偏振光。与此前相同，第二光线230的一部分透射，第二光线230的另一部分反射。第二光线230的偏振态变为左旋偏振光，再经过第一波片222后，第二光线230再次被调制为线偏光，偏振方向沿y轴方向。根据反射偏振片221的特性，第二光线230将透过反射偏振片221出射，最终进入眼球。从上述第二光线230路径描述可知，显示屏23发射的第二光线230在透镜组22中经过折叠，最终进入到眼球，通过透镜组22以使显示屏23发射的第二光线230发生折叠，增长了第二光线230的光路但是不会增加增强现实组件20的厚度，有利于增强现实组件20的小型化。
同时，在第二光线230到达半反半透膜224，第二光线230的一部分透射，最终进入眼球，第二光线230的另一部分经半反半透膜224反射形成左旋偏振光，部分反射至第三波片27，形成线偏光，偏振方向沿x轴方向，根据第二线偏振片26的特性，第二光线230被第二线偏振片26拦截(图7中虚线箭头所代表的光路部分)。
本申请通过在显示屏23和透镜组22之间设置第二线偏振片26和第三波片27，能够拦截从显示屏23发出的第二光线230经透镜组22反射后回到显示屏23的部分，减少增强现实组件20中由于反射光线产生的鬼影现象，使用户能够观看到更清晰的画面。
如图8，经半反半透膜224反射形成的第二光线230还有部分反射至第二波片2432，经第二波片2432调制变为偏振方向沿x’轴方向的线偏光，第一线偏振片2431的透光轴沿x’轴方向，因此线偏光会穿过第一线偏振片2431进入摄像模组24。由于本申请通过控制器25控制摄像模组24在显示屏23处于插黑时段时曝光，在显示屏23处于显示时段时停止曝光，从而在眼动追踪和身份识别过程中，显示屏23的第二光线230不会被摄像模组24感测，有效防止来自显示屏23的杂散光进入摄像模组24，提高眼动追踪或身份识别的精度。
当然，在其他实施例中，摄像模组还可以设于透镜组的入光侧并朝向透镜组设置，此时，第二光线部分经过透镜组透射进入摄像模组。或者，摄像模组还可以嵌设于显示屏内部并朝向透镜组设置，在此种情况下，显示屏处于显示时段时，显示屏发射的光线也不会进入摄像模组，增强现实设备可以不包括控制器，也就是说，控制器可以不用控制摄像模组在插黑时段曝光，在显示时段停止曝光可以防止来之显示屏的杂光进入到摄像模组。
请参阅图9，图9是本申请实施例提供的一种电子设备100的控制方法的流程示意图。该控制方法应用于如图3所示的电子设备100。该如图9所示，电子设备100的控制方法包括如下的S110～S130。
S110：控制显示屏23按显示周期进行显示，显示周期包括交替排布的显示时段和插黑时段。
具体的，在电子设备100中，为了消除头动时画面的拖影现象，会对显示屏23进行插黑处理，即在显示周期内仅约20％的时间内显示屏23处于点亮状态，该时段为显示时段，其他的时间显示屏23处在关闭的状态，该时段为插黑时段。如图10所示，本实施例以帧率为90fps的显示屏为例，两个Vsync信号之间的间隔为显示的帧间隔，约为11.1ms，Vsync信号之后的8.9ms时间内显示屏23处于关闭状态，在帧间隔的最后约2.2ms内显示屏23处于点亮状态，然后进入下一帧周期。也就是说，显示屏23在一帧内依次为插黑时段和点亮时段，接着下一帧也依次为插黑时段和点亮时段，如此周期重复。
S120：控制摄像模组24在插黑时段曝光，以获得第一光线，且在显示时段停止曝光。
具体的，在控制摄像模组24曝光之前，点亮光源。具体的，首先控制器25响应于用户识别指令或者响应于眼动追踪指令。用户识别指令和眼动追踪指令可以通过电子设备100的中央处理器下达给控制器25。当然，控制器25也可以获取电子设备100的状态，根据电子设备100的状态开启用户识别指令或者眼动追踪指令。
请参阅图11，当控制器25响应于用户识别指令时，控制可见光源211发出第一光线，摄像模组24在插黑时段曝光，且在显示时段停止曝光。
请参阅图12，当控制器25响应于眼动追踪指令，控制近红外光源212发出第一光线，摄像模组24在插黑时段曝光，且在显示时段停止曝光。
示例的，光源在点亮时段点亮，图11和图12的实施例中，可见光源211和近红外光源212在点亮时段点亮，点亮时段覆盖显示时段和插黑时段，也就是说，可见光源211在响应于身份识别指令时，处于长亮状态，近红外光源212在响应于眼动追踪指令时，处于长亮状态。在其他实施例中，点亮时段覆盖至少一个显示时段和至少一个插黑时段。
当然，在其他实施例中，光源在点亮时段点亮，点亮时段覆盖至少一个插黑时段。可以理解的是，光源还可以在摄像模组24曝光时点亮，也就是说，光源的点亮和摄像模组曝光同时进行。示例的，如图13，当控制器25响应于用户识别指令时，控制可见光源211和摄像模组24在插黑时段分别点亮和曝光。控制可见光源211和摄像模组24在显示时段分别关闭和停止曝光。也就是说，由于摄像模组24在显示屏23处于显示时段时不曝光，不需要拍摄用户眼球的照片，因此也不需要点亮可见光源211，有效节约能源。
如图14，当控制器25响应于眼动追踪指令时，控制近红外光源212和摄像模组24在插黑时段分别点亮和曝光。控制近红外光源212和摄像模组24在显示时段分别关闭和停止曝光。也就是说，由于摄像模组24在显示屏23处于显示时段时不曝光，不需要拍摄用户眼球的照片，因此也不需要点亮近红外光源212，有效节约能源。
当然，点亮时段可以覆盖多个相邻的插黑时段，也可以覆盖多个间隔的插黑时段。点亮时段还可以覆盖插黑时段和插黑时段前的部分显示时段。
本申请通过控制摄像模组24在显示屏23处于插黑时段曝光，而在显示屏23处于显示时段停止曝光，从而在眼动追踪和身份识别过程中，显示屏23的第二光线不会被摄像模组24感测，有效防止来自显示屏23的杂散光进入摄像模组24，有效提高眼动追踪或身份识别的精度。
S130：依据第一光线形成眼球图像。
具体的，当控制器25响应于身份识别指令时，则摄像模组24曝光，以获得第一眼球图像，第一眼球图像用于巩膜识别；当控制器25响应于眼动追踪指令时，则摄像模组24曝光，以获得第二眼球图像，第二眼球图像用于眼动追踪。
可以理解的是，眼球图像(第一眼球图像或第二眼球图像)可以是摄像模组24在一帧中的插黑时段曝光得到的图像，也可以是摄像模组24在多帧中的插黑时段曝光得到的图像，其中多帧中的插黑时段可以是多个相邻的插黑时段，也可以是多个间隔的插黑时段。示例的，摄像模组24在第一帧的插黑时段曝光，接着在第一帧的显示时段停止曝光，然后在第二帧的插黑时段继续曝光，得到眼球图像，则该眼球图像就是摄像模组24在两个两邻的插黑时段累加曝光形成的图像。同样，摄像模组24还可以在第一帧的插黑时段曝光，在第一帧的显示时段停止曝光，在第二帧均不曝光，然后在第三帧的插黑时段曝光，得到眼球图像，则该眼球图像就是摄像模组24在两个间隔的插黑时段累加曝光形成的图像。
如图15，控制方法还包括将第一眼球图像与预存图像进行对比，若两者匹配，则调用数据库中的预存眼动校准数据，预存眼动校准数据与预存图像对应。眼动校准数据用于眼动追踪算法，根据眼动追踪算法输出眼球注视点位置，眼球注视点位置用于协助处理显示数据，即注视点渲染。若第一眼球图像与预存图像不匹配，则对用户的眼动进行校准，获取眼动校准数据。眼动校准数据用于眼动追踪算法，根据眼动追踪算法输出眼球注视点位置，眼球注视点位置用于协助处理显示数据，即注视点渲染，并将眼动校准数据存入眼动校准数据库中，以供下次使用。
当然，在其他实施例中，上述的控制方法可以应用于电子设备开启且处于开机界面时，将第一眼球图像与预存图像进行对比，若两者匹配，则解锁电子设备，以使电子设备进入主界面。若第一眼球图像与预存图像不匹配，则弹出解锁密码。该控制方法解锁电子设备的速度快，提高用户体验。
本申请提供的控制方法通过对用户进行身份识别，可以在用户首次使用电子设备100时注册其巩膜信息，并将其眼动校准的参数保存在个人档案中，当用户再次使用电子设备100时，系统通过巩膜识别确认用户的身份后，直接调用用户的眼动校准参数，用户可以不用再次进行眼动校准，节约了用户的时间，大大提升用户体验。当然，身份识别还能用于电子设备100在使用过程中的场景，例如用巩膜识别支付代替密码支付。
以上，仅为本申请的部分实施例和实施方式，本申请的保护范围不局限于此，任何熟知本领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (16)

  1. 一种电子设备的控制方法，其特征在于，所述电子设备包括光源、透镜组、显示屏和摄像模组，所述光源用于向用户的眼球发射第一光线，所述第一光线经所述眼球反射，透过所述透镜组进入摄像模组；所述显示屏发射的第二光线部分经所述透镜组反射或透射进入摄像模组，所述控制方法包括：
    控制所述显示屏按照显示周期进行显示，所述显示周期包括交替排布的显示时段和插黑时段；
    控制所述摄像模组在所述插黑时段曝光，以获得所述第一光线，且在所述显示时段停止曝光；
    依据所述第一光线形成眼球图像。
  2. 根据权利要求1所述的控制方法，其特征在于，在控制所述摄像模组曝光之前，所述控制方法还包括点亮光源。
  3. 根据权利要求2所述的控制方法，其特征在于，所述点亮光源包括在点亮时段点亮光源，所述点亮时段覆盖至少一个所述显示时段和至少一个所述插黑时段；或者所述点亮时段覆盖至少一个所述插黑时段。
  4. 根据权利要求1-3中任一项所述的控制方法，其特征在于，所述光源为可见光源，所述眼球图像用于巩膜识别。
  5. 根据权利要求1-3中任一项所述的控制方法，其特征在于，所述光源包括可见光源和近红外光源，所述摄像模组为近红外光和可见光双通摄像模组，所述控制方法还包括：
    响应于用户识别指令，控制所述可见光源发出所述第一光线，所述摄像模组曝光，以获得第一眼球图像，所述第一眼球图像用于巩膜识别；
    响应于眼动追踪指令，控制所述近红外光源发出所述第一光线，所述摄像模组曝光，以获得第二眼球图像，所述第二眼球图像用于眼动追踪。
  6. 根据权利要求5所述的控制方法，其特征在于，所述控制方法还包括：
    将所述第一眼球图像与预存图像进行对比；
    若两者匹配，则调用预存眼动校准数据，所述预存眼动校准数据与所述预存图像对应。
  7. 根据权利要求5所述的控制方法，其特征在于，所述控制方法应用于电子设备开启且处于开机界面时，所述控制方法还包括：
    将所述第一眼球图像与预存图像进行对比；
    若两者匹配，则解锁所述电子设备，以使所述电子设备进入主界面。
  8. 根据权利要求6所述的控制方法，其特征在于，所述控制方法还包括：若所述第一眼球图像与所述预存图像不匹配，则对用户的眼动进行校准，获取眼动校准数据。
  9. 一种电子设备，其特征在于，所述电子设备包括光源、透镜组、摄像模组、显示屏和控制器，所述显示屏设于所述透镜组的出光侧，所述光源设于所述透镜组远离所述显示屏的一侧，所述摄像模组的入光侧朝向所述透镜组；所述显示屏的显示周期包括交替排布的显示时段和插黑时段；所述光源用于向用户的眼球发射第一光线，所述第一光线经所述眼球反射，透过所述透镜组进入摄像模组；所述显示屏发射的第二光线部分经所述透镜组反射或透射进入摄像模组；所述光源、所述摄像模组和所述显示屏均与所述控制器电连接，所述控制器用于控制所述摄像模组在所述插黑时段曝光，以获得所述第一光线，且在所述显示时段停止曝光，并依据所述第一光线形成眼球图像。
  10. 根据权利要求9所述的电子设备，其特征在于，所述光源为可见光源。
  11. 根据权利要求9所述的电子设备，其特征在于，所述光源包括可见光源和近红外光源，所述摄像模组为近红外光和可见光双通摄像模组。
  12. 根据权利要求9至11中任一项所述的电子设备，其特征在于，所述透镜组包括透镜和半反半透膜，所述半反半透膜设于所述透镜的出光侧。
  13. 根据权利要求12所述的电子设备，其特征在于，所述半反半透膜在第一波段的透过率大于95％，反射率小于1％，其中，所述第一波段为近红外光所在的频段。
  14. 根据权利要求12或13所述的电子设备，其特征在于，所述透镜组还包括反射偏振片和第一波片，所述第一波片设于所述透镜背向所述半反半透膜一侧，所述反射偏振片设于所述第一波片背向所述透镜一侧；所述摄像模组还包括第一线偏振片和第二波片，所述第一线偏振片位于所述第二波片与所述摄像模组的镜头之间，所述反射偏振片的偏振方向与所述第一线偏振片的偏振方向相同或相互垂直，所述第一波片与所述第二波片均为四分之一波片。
  15. 根据权利要求12或13所述的电子设备，其特征在于，所述透镜组还包括反射偏振片和第一波片，所述第一波片设于所述透镜背向所述半反半透膜一侧，所述反射偏振片设于所述第一波片背向所述透镜一侧；所述摄像模组还包括第一圆偏振片，所述第一圆偏振片与所述第一波片的旋向相同，所述第一圆偏振片与所述反射偏振片的偏振方向相同或相互垂直。
  16. 根据权利要求9至15中任一项所述的电子设备，其特征在于，所述电子设备还包括第二线偏振片和第三波片，所述第二线偏振片设于所述显示屏的出光侧，所述第三波片设于所述第二线偏振片和所述透镜组之间。
PCT/CN2022/082282 2021-03-26 2022-03-22 电子设备及其控制方法 WO2022199580A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22774236.8A EP4293412A4 (en) 2021-03-26 2022-03-22 ELECTRONIC DEVICE AND ITS CONTROL METHOD
US18/552,553 US20240171846A1 (en) 2021-03-26 2022-03-22 Electronic Device and Control Method Thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110328563.3 2021-03-26
CN202110328563.3A CN115128803B (zh) 2021-03-26 2021-03-26 电子设备及其控制方法

Publications (1)

Publication Number Publication Date
WO2022199580A1 true WO2022199580A1 (zh) 2022-09-29

Family

ID=83374339

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/082282 WO2022199580A1 (zh) 2021-03-26 2022-03-22 电子设备及其控制方法

Country Status (4)

Country Link
US (1) US20240171846A1 (zh)
EP (1) EP4293412A4 (zh)
CN (1) CN115128803B (zh)
WO (1) WO2022199580A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116679458A (zh) * 2023-08-03 2023-09-01 歌尔光学科技有限公司 光学模组以及头戴显示设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118151370A (zh) * 2022-12-06 2024-06-07 北京字跳网络技术有限公司 眼动追踪装置和虚拟现实装置
CN116009304A (zh) * 2022-12-12 2023-04-25 苏州苏大维格科技集团股份有限公司 圆偏振分束膜、制备方法及vr显示系统
CN117311004B (zh) * 2023-11-03 2024-07-05 广州纳立多科技有限公司 短距离光学放大模组、头戴显示器及虚拟现实系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109143581A (zh) * 2018-08-28 2019-01-04 北京七鑫易维信息技术有限公司 一种头戴式显示设备及其眼球追踪方法
EP3514606A1 (en) * 2018-01-23 2019-07-24 BAE SYSTEMS plc Eye tracking for head-worn display
US10429656B1 (en) * 2018-01-18 2019-10-01 Facebook Technologies, Llc Eye tracking for a head mounted display including a pancake lens block
US10474229B1 (en) * 2017-11-01 2019-11-12 Facebook Technologies, Llc Folded viewing optics with high eye tracking contrast ratio
US20200073123A1 (en) * 2018-08-31 2020-03-05 Google Llc Near-eye display system with polarization-based optical path folding and variable focus catadioptric lens assembly

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000194490A (ja) * 1998-12-24 2000-07-14 Canon Inc 視線検出装置
JP5046788B2 (ja) * 2007-08-10 2012-10-10 キヤノン株式会社 撮像装置及びその制御方法
JP6137006B2 (ja) * 2014-03-19 2017-05-31 株式会社Ｊｖｃケンウッド 画像表示装置および画像表示方法
JP6311823B2 (ja) * 2017-04-25 2018-04-18 株式会社Ｊｖｃケンウッド 画像表示装置および画像表示方法
US10642048B2 (en) * 2018-08-07 2020-05-05 Facebook Technologies, Llc Reflective circular polarizer for head-mounted display
CN108986752B (zh) * 2018-08-31 2020-12-01 厦门天马微电子有限公司 显示装置及其控制方法
CN113614783A (zh) * 2019-01-25 2021-11-05 奇跃公司 使用具有不同曝光时间的图像的眼睛跟踪
CN110824699B (zh) * 2019-12-25 2020-12-04 歌尔光学科技有限公司 一种近眼显示设备的眼球追踪系统及近眼显示设备
CN111766702A (zh) * 2020-06-29 2020-10-13 京东方科技集团股份有限公司 眼球追踪的近眼显示光学系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10474229B1 (en) * 2017-11-01 2019-11-12 Facebook Technologies, Llc Folded viewing optics with high eye tracking contrast ratio
US10429656B1 (en) * 2018-01-18 2019-10-01 Facebook Technologies, Llc Eye tracking for a head mounted display including a pancake lens block
EP3514606A1 (en) * 2018-01-23 2019-07-24 BAE SYSTEMS plc Eye tracking for head-worn display
CN109143581A (zh) * 2018-08-28 2019-01-04 北京七鑫易维信息技术有限公司 一种头戴式显示设备及其眼球追踪方法
US20200073123A1 (en) * 2018-08-31 2020-03-05 Google Llc Near-eye display system with polarization-based optical path folding and variable focus catadioptric lens assembly

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4293412A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116679458A (zh) * 2023-08-03 2023-09-01 歌尔光学科技有限公司 光学模组以及头戴显示设备
CN116679458B (zh) * 2023-08-03 2023-11-07 歌尔光学科技有限公司 光学模组以及头戴显示设备

Also Published As

Publication number Publication date
EP4293412A4 (en) 2024-08-28
CN115128803A (zh) 2022-09-30
EP4293412A1 (en) 2023-12-20
CN115128803B (zh) 2023-04-18
US20240171846A1 (en) 2024-05-23

Similar Documents

Publication Publication Date Title
WO2022199580A1 (zh) 电子设备及其控制方法
JP7285579B2 (ja) 拡張現実装置、およびそのための光学系
US10600352B1 (en) Display device with a switchable window and see-through pancake lens assembly
US9087471B2 (en) Adaptive brightness control of head mounted display
US10983341B2 (en) Eye tracking based on polarization volume grating
US8867139B2 (en) Dual axis internal optical beam tilt for eyepiece of an HMD
US9151984B2 (en) Active reflective surfaces
US9927613B2 (en) Image display apparatus and head mounted display
US8503085B2 (en) Head-mounted display
KR101598480B1 (ko) 시스루형 헤드 마운트 표시장치
US11480800B1 (en) Compact catadioptric projector
CN113448089A (zh) 增强现实设备及其显示方法
CN216285981U (zh) 光学模组及系统、显示装置、头戴式显示设备及其系统
WO2022130372A1 (en) Optical systems and methods for eye tracking based on eye imaging via collimating element and light-guide optical element
WO2022111601A1 (zh) 眼动追踪装置及电子设备
CN115145028A (zh) 增强现实设备及其显示方法
TWI677711B (zh) 虛實影像整合裝置
CN110967828A (zh) 显示系统以及头戴显示装置
WO2021232967A1 (zh) 光学显示组件和智能穿戴设备
US12085717B2 (en) Hybrid waveguide to maximize coverage in field of view (FOV)
EP4414769A1 (en) Folded beam two-dimensional (2d) beam scanner
CN113885206A (zh) 光学模组及系统、显示装置、头戴式显示设备及其系统
KR20210083777A (ko) 증강현실 영상의 노출을 최소화한 증강현실 광학장치 및 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22774236

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022774236

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022774236

Country of ref document: EP

Effective date: 20230912

WWE Wipo information: entry into national phase

Ref document number: 18552553

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE