CN115393947A

CN115393947A - Iris image acquisition method based on head-mounted display and related product

Info

Publication number: CN115393947A
Application number: CN202211056340.7A
Authority: CN
Inventors: 韦燕华
Original assignee: Shanghai Wingtech Electronic Technology Co Ltd
Current assignee: Shanghai Wingtech Electronic Technology Co Ltd
Priority date: 2022-08-30
Filing date: 2022-08-30
Publication date: 2022-11-25
Also published as: WO2024045446A1

Abstract

The embodiment of the application discloses iris image acquisition method and relevant product based on head mounted display, head mounted display includes light source and camera device, and camera device includes: the camera shooting device comprises a camera shooting hole, an imaging sensor and a position adjusting module of the imaging sensor; the camera shooting hole is arranged on one side of the head-mounted display, which is contacted with the head of a wearer; the light emitted by the light source is shot by the camera Kong Sheru; the method comprises the following steps: acquiring a first iris image generated by an imaging sensor based on incident light; controlling a position adjusting module to execute moving operation and/or rotating operation according to the first iris image; the moving operation is used for adjusting the distance from the imaging sensor to the camera hole so as to maximize the imaging of the iris in the image; the rotating operation is used for rotating the imaging sensor to change the angle of the light rays entering the imaging sensor; a second iris image generated by the adjusted imaging sensor based on the incident light is acquired. By implementing the embodiment of the application, the image quality of the acquired iris image can be improved.

Description

Iris image acquisition method based on head-mounted display and related product

Technical Field

The application relates to the technical field of image processing, in particular to an iris image acquisition method based on a head-mounted display and a related product.

Background

The iris is an internal tissue of a human body, and an area between the sclera and the pupil of the eyeball is the iris. The iris has an inner concentric circle and an outer concentric circle and has obvious texture characteristics. The texture characteristics of the iris are determined by heredity, so that the identification by utilizing the iris has higher accuracy.

In a Virtual Reality (VR) scene, a camera may be built in VR equipment such as a head mounted display for acquiring an iris image of a user, and the acquired iris image is used for performing identity verification on the user to verify whether the identity of the user using the VR equipment is legal. However, in practice, it is found that the accuracy of user identification is closely related to the quality of the iris image, and the low quality iris image will greatly reduce the accuracy of identification.

Disclosure of Invention

The embodiment of the application discloses an iris image acquisition method based on a head-mounted display and a related product, which can improve the image quality of the acquired iris image.

The embodiment of the application discloses iris image acquisition method based on head mounted display, head mounted display includes light source and camera device, camera device includes: the device comprises a camera hole, an imaging sensor and a position adjusting module of the imaging sensor; the camera hole is arranged towards a wearer of the head-mounted display; the light emitted by the light source is captured by the camera Kong Sheru; and, the method comprises:

acquiring a first iris image generated by the imaging sensor based on incident light;

controlling the position adjusting module to execute a moving operation and/or a rotating operation according to the first iris image; the moving operation is used for adjusting the distance from the imaging sensor to the camera hole so as to maximize the imaging of the iris in the image; the rotating operation is used for rotating the imaging sensor to change the angle of the light rays entering the imaging sensor;

and acquiring a second iris image generated by the adjusted imaging sensor based on the incident light.

In one embodiment, controlling the position adjustment module to perform a moving operation according to the first iris image includes:

determining an iris region in the first iris image and comparing a region area of the iris region to a region area of a reference region; the reference region is an image region corresponding to the complete iris when imaging maximization is carried out in the image;

determining the moving direction of the position adjusting module to execute the moving operation according to the area difference between the iris area and the reference area;

and controlling the position adjusting module to execute the moving operation according to the moving direction.

In one embodiment, the determining the moving direction of the position adjusting module to perform the moving operation according to the area difference between the iris region and the reference region includes:

when the area of the iris area is smaller than that of the reference area, determining the moving direction of the position adjusting module to perform the moving operation as the direction of increasing the distance; alternatively, the first and second electrodes may be,

when the area of the iris area is larger than that of the reference area, determining the moving direction of the position adjusting module to perform the moving operation as a direction to shorten the distance.

In one embodiment, controlling the position adjustment module to perform a rotation operation according to the first iris image includes:

calculating the incidence angle of the light ray entering the imaging sensor according to the first iris image; the incidence angle is an included angle between the light ray and the plane where the imaging sensor is located in the target direction; the target direction is a direction from an interior of the head mounted display to an exterior of the head mounted display;

if the incident angle is a right angle, determining the rotating direction of the rotating operation according to the incident angle;

and controlling the position adjusting module to execute rotation operation according to the rotation direction.

In one embodiment, the determining the rotation direction of the rotation operation according to the injection angle includes:

if the injection angle is an obtuse angle, determining that the rotating direction of the rotating operation is anticlockwise rotation; or;

and if the incidence angle is an acute angle, determining that the rotation direction of the rotation operation is clockwise rotation. In one embodiment, the position adjustment module comprises: a telescopic link; one end of the telescopic connecting rod is connected with the imaging sensor, and the other end of the telescopic connecting rod is connected with the shell of the head-mounted display; the telescopic connecting rod is used for executing the moving operation; and/or the presence of a gas in the gas,

the position adjustment module may include: a rotatable platform; the imaging sensor is placed on the rotatable platform; the rotatable platform is used for executing the rotating operation.

In one embodiment, the image pickup apparatus further includes: correcting the chip; the head mounted display further comprises: an eye tracking sensor;

the correction chip stores a distortion correction model, the distortion correction model is obtained by training sample data, the sample data comprises a plurality of sample iris images, the iris images are respectively collected when a user looks at different positions of a display screen of the head-mounted display, and the sample data further comprises non-distortion supervision iris images respectively corresponding to the iris images; the method further comprises the following steps:

determining a gaze location of a user's gaze on the display screen detected by the eye tracking sensor;

transmitting the second iris image generated by the imaging sensor and the determined gaze position to the correction chip;

and carrying out distortion correction on the second iris image according to the gaze position through a distortion correction model stored by the correction chip to obtain a third iris image output by the distortion correction model.

The embodiment of the application discloses iris image acquisition device based on head mounted display, head mounted display includes light source and camera device, camera device includes: the device comprises a camera hole, an imaging sensor and a position adjusting module of the imaging sensor; the camera hole is arranged towards a wearer of the head-mounted display; the light emitted by the light source is shot Kong Sheru; and, the apparatus comprises:

the imaging module is used for acquiring a first iris image generated by the imaging sensor based on the incident light;

the motion control module is used for controlling the position adjusting module to execute a moving operation and/or a rotating operation according to the first iris image; the moving operation is used for adjusting the distance from the imaging sensor to the camera hole so as to maximize the imaging of the iris in the image; the rotating operation is used for rotating the imaging sensor to change the angle of the light rays entering the imaging sensor;

the imaging module is further configured to acquire a second iris image generated by the adjusted imaging sensor based on the incident light.

The embodiment of the application discloses an electronic device, which comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is enabled to realize the steps of any one of the iris image acquisition methods based on the head-mounted display disclosed by the embodiment of the application.

The embodiment of the application discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the computer program realizes the steps of any one of the iris image acquisition methods based on the head-mounted display disclosed by the embodiment of the application.

Compared with the related art, the embodiment of the application has the following beneficial effects:

the head-mounted display includes a light source and a camera device, the light source is used for providing illumination light, the camera device includes: the camera shooting device comprises a camera shooting hole, an imaging sensor and a position adjusting module of the imaging sensor. The method comprises the steps of firstly generating a first iris image by using an imaging sensor of a head-mounted display, controlling a position adjusting module to execute moving operation and/or rotating operation by using the first iris image so as to adjust the distance between the imaging sensor and a camera hole and/or the angle of light rays entering the imaging sensor, and generating a second iris image by using the adjusted imaging sensor.

The distance between the adjusted imaging sensor and the camera hole is changed, compared with the first iris image, the imaging size of the iris in the second iris image is closer to the imaging size when the imaging is maximized, and the second iris image can comprise more iris information. And/or the adjusted imaging sensor has a larger area for receiving light, and the second iris image has higher brightness compared with the first iris image. Therefore, the image quality of the second iris image is higher than that of the first iris image, and the second iris image is used for identity authentication, so that a more accurate identity authentication result can be obtained.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of an application scenario of a head-mounted display based iris image acquisition method according to an embodiment;

FIG. 2A is a schematic diagram illustrating an exemplary embodiment of a head-mounted display;

FIG. 2B is a schematic diagram of another embodiment of a head mounted display;

FIG. 2C is a schematic diagram of another embodiment of a head mounted display;

FIG. 3 is a flowchart illustrating a method for a head-mounted display based iris image acquisition method according to an exemplary embodiment;

FIG. 4 is a flowchart illustrating a method for a head-mounted display based iris image capture method according to an exemplary embodiment;

FIG. 5 is a flowchart illustrating a method for acquiring an iris image based on a head-mounted display according to an exemplary embodiment;

FIG. 6 is a flowchart illustrating a method for head-mounted display based iris image acquisition according to an exemplary embodiment;

FIG. 7 is a schematic structural diagram of an iris image capturing device based on a head-mounted display according to an embodiment;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It is to be noted that the terms "comprises" and "comprising" and any variations thereof in the examples and figures of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

The embodiment of the application discloses an iris image acquisition method based on a head-mounted display and a related product, which can improve the image quality of the acquired iris image. The following are detailed below.

Referring to fig. 1, fig. 1 is a schematic view of an application scenario of an iris image capturing method based on a head-mounted display according to an embodiment. As shown in fig. 1, a first operating environment is provided, which may include a head mounted display 101, a terminal device 102, and a server 103.

The user may wear the head mounted display 101 to cause the head mounted display 101 to acquire data. Here, the head-mounted display 101 does not have data processing capability, and after acquiring data, the head-mounted display can transmit data to and from the terminal 102 by using the near field communication technology.

The terminal device 102 may include an electronic device such as a smart television, a three-dimensional visual display device, a large projection system, a multimedia playing device, a mobile phone, a tablet Computer, a game console, and a PC (Personal Computer). The terminal device 102 may receive data transmitted by the head mounted display 101 and process the data.

The server 103 is configured to provide a background service for the terminal 102, so that the terminal 102 processes the received data transmitted by the head mounted display 101, thereby completing the iris image acquisition method provided by the present application. Optionally, the server 103 may further generate a corresponding control instruction according to the data processing result, and the control instruction may be sent to the terminal 102 and/or the head mounted display 101, respectively, so as to control the terminal 102 and/or the head mounted display 103. For example, server 103 may be a backend server. The server 103 may be one server, a server cluster composed of a plurality of servers, or a cloud computing service center. Optionally, the server 103 provides background services for a plurality of terminals 102 simultaneously.

A second operating environment is given, which may include a head mounted display 101 and a terminal device 102.

Here, the head mounted display 101 may include various types of devices as stated above, and the head mounted display 101 has no data processing capability, and can transmit data with the terminal 102 by the near field communication technology after acquiring the data.

The terminal device 102 may include various types of electronic devices as set forth above. The terminal device 102 may receive data transmitted by the head mounted display 101 and process the data to complete the iris image capturing method provided by the present application. Optionally, the terminal 102 may further generate corresponding control instructions according to the data processing result, and the control instructions may respectively send the head mounted display 101 to control the head mounted display 103.

A third operating environment is given which includes only the head mounted display 101. Here, the head-mounted display 101 not only has data acquisition capability, but also has data processing capability, that is, the functions implemented by the iris image acquisition method provided in the present application can be implemented by calling program codes through a processor in the head-mounted display 101, and of course, the program codes can be stored in a computer storage medium, and thus, the head-mounted display at least comprises a processor and a storage medium.

Referring to fig. 2A, fig. 2A is a schematic structural diagram of a head-mounted display according to an embodiment. As shown in fig. 2A, the head-mounted display 20 may include a display screen 22, a light source 23, and a camera 24.

The Display screen 22 may be a Light-emitting Diode (LED) screen or a Liquid Crystal Display (LCD) screen for outputting image data.

The light source 23 may be a visible light source or an infrared light source, and is not limited in particular.

Referring to fig. 2B, fig. 2B is a schematic structural diagram of another head-mounted display according to an embodiment. The head mounted display shown in fig. 2B may be a side view of the head mounted display as shown in fig. 2A. As shown in fig. 2B, the head-mounted display 20 may include a lens module 21, a display screen 22, a light source 23, and a camera 24.

The lens module 21 can be arranged on the display screen 22 and can be used for refracting light rays and drawing the image on the display screen 22 to the retina position, so that the eyes of people can easily see the display screen 22 almost stuck in front of the eyes; the lens module 21 also has a light condensing function, and condenses light inside the head-mounted display, so that more light can enter the camera device 24.

In the embodiment of the present application, the lens module 21 may be a Panckae optical module, which is formed by combining 2 or more lenses; alternatively, the lens module 21 may be a fresnel optical module, and the optical module may be formed by one lens.

It should be noted that, the number of the lens modules 21, the display screen 22 and the light sources 23 in the head-mounted display 20 disclosed in the embodiment of the present application is not limited.

For example, the head-mounted display 20 may include a display screen 22 and a lens module 21, and a plurality of light sources 23 may be disposed around the lens module 21.

For example, the head-mounted display 20 may also include two display screens 22 corresponding to the left eye and the right eye, respectively, and a lens module 21 corresponding to each display screen 22. A plurality of light sources 23 may be disposed around each lens module 21.

The camera device 24 may be configured to collect image data, and may at least include: the camera includes a camera hole 241, an imaging sensor 242, a position adjusting module 243 of the imaging sensor, and a base 244.

The camera hole 241 may be disposed inside the head-mounted display 20, i.e., toward a wearer of the head-mounted display 20. Light generated by the light source 23 can be incident from the camera hole 241.

For example, as shown in fig. 2B, the display screen 22 may be a hole digging screen, and the hole digging position of the display screen 22 may be the setting position of the camera 241. For example, if the head-mounted display 20 includes two lens modules 21, the camera hole 241 may be disposed above or below the middle position of the two lens modules 21.

The imaging sensor 242 may be any photosensitive Device, such as a Charge Coupled Device (CCD) or a complementary Metal-Oxide Semiconductor (CMOS), for example.

The position adjustment module 243 may include: a telescoping linkage 243a and/or a rotatable platform 243b.

The telescopic link 243a has one end connected to the imaging sensor 242 and the other end connected to the housing of the head-mounted display 20. When the length of the telescopic link 243a is lengthened, the distance between the imaging sensor 242 and the camera hole 241 is shortened, and the optical zoom factor is reduced; when the length of the telescopic link 243a is shortened, the distance between the imaging sensor 242 and the imaging hole 241 is increased, and the optical zoom factor is increased.

The rotatable platform 243b may be used to place the imaging sensor 242, and the rotatable platform 243b may be rotated in a clockwise direction or a counterclockwise direction, thereby rotating the imaging sensor 242 placed on the rotatable platform 243b. Rotation of imaging sensor 242 may change the angle at which light is incident on imaging sensor 242, thereby changing the area of imaging sensor 242 that receives light. When light is incident perpendicularly to the imaging sensor 242, the area of the imaging sensor 242 that receives the light is maximized.

It should be noted that, if the position adjusting module 243 includes a retractable link 243a and a rotatable platform 243B, as shown in fig. 2B, one end of the retractable link 243a may be connected to the rotatable platform 243B. The telescopic link 243a expands and contracts to change the position of the rotatable platform 243b, so as to change the distance between the imaging sensor 242 placed on the rotatable platform 243b and the camera hole 241.

Furthermore, the telescopic link 243a may be an alternative embodiment for adjusting the distance between the imaging sensor 242 and the camera hole 241, and in other possible embodiments, the distance between the imaging sensor 242 and the camera hole 241 may be adjusted by other elements. The rotatable platform 243 may be an alternative embodiment of the rotating imaging sensor 242, and in other possible embodiments, the imaging sensor 242 may be rotated by other elements as well.

And a base 244 for carrying the position adjusting module 243 and the imaging sensor 242.

For example, please refer to fig. 2C, fig. 2C is a schematic structural diagram of another head-mounted display according to an embodiment. As shown in fig. 2C, the image pickup device 24 may further include a reflecting mirror 245, and the reflecting mirror 245 may be disposed between the image pickup hole 241 and the imaging sensor 242.

And a reflector 245 for changing the optical path of the light incident from the image capturing hole 241, and reflecting the light to the image sensor 242. The arrangement of the reflecting mirror 245 enables the arrangement positions of the camera hole 241 and the imaging sensor 242 not to be on the same straight line, which is beneficial to the design of the internal space of the head-mounted display 20.

Referring to fig. 3, fig. 3 is a flowchart illustrating a method for acquiring an iris image based on a head-mounted display according to an embodiment of the present disclosure, where the method is applicable to any electronic device such as a terminal device, a service device, or a head-mounted display body shown in the foregoing operating environment. Referring to fig. 3, the method shown in fig. 3 may include the following steps:

310. a first iris image generated by an imaging sensor based on incident light is acquired.

At this application embodiment, after the user wore head mounted display, the light that the light source launched can be reflected by the iris of user's eyeball, and the light that the iris reflection was passed through the camera hole and is got into inside the casing of head mounted display, reachs imaging sensor. The imaging sensor can convert light reflected by the iris into a digital signal to obtain a first iris image.

In some embodiments, the head mounted display may also include an eye tracking sensor that may be used to detect the user's gaze. Specific embodiments of step 310 may include: when it is detected that the sight line of the user falls on a display screen of the head-mounted display, a first iris image generated by the imaging sensor based on the incident light is acquired.

320. And controlling the position adjusting module to execute the moving operation and/or the rotating operation according to the first iris image.

In some embodiments, the iris area occupied by the iris in the first iris image may be compared with a standard reference area, and the position adjusting module is controlled to perform a moving operation according to the comparison result to adjust the distance from the imaging sensor to the camera hole, and adjust the size of the iris image in the image, wherein the adjusting direction is such that the iris image in the image is consistent with the reference area. The reference region may be an image region corresponding to the image maximization of the complete iris in the image, and may compare the iris region with a region parameter such as a region area or a region position of the reference region.

In other embodiments, the angle of the light beam incident on the imaging sensor may be calculated according to the first iris image, and the position adjusting module may be controlled to perform a rotation operation according to the calculated incident angle, so as to change the angle of the light beam incident on the imaging sensor, and adjust the area of the imaging sensor receiving the light beam, so that the imaging sensor is perpendicular to the incident light beam. The larger the area of the imaging sensor receiving the light, the higher the brightness of the image obtained after imaging.

In still other embodiments, the moving operation and the rotating operation may be performed simultaneously according to the comparison result of the iris region in the first iris image and the reference region and the first iris image control position adjustment module to simultaneously adjust the imaging size of the iris in the image and the image brightness.

330. A second iris image generated by the adjusted imaging sensor based on the incident light is acquired.

In the embodiment of the application, the first iris image can be shot firstly, and the distance between the imaging sensor and the camera hole and/or the angle of the light entering the imaging sensor can be adjusted by utilizing the first iris image. The distance between the adjusted imaging sensor and the camera hole is changed, compared with the first iris image, the imaging size of the iris in the second iris image is closer to the imaging size when the imaging is maximized, and the second iris image can comprise more iris information. And/or the adjusted imaging sensor has a larger area for receiving light, and the second iris image has higher brightness compared with the first iris image. Therefore, the image quality of the second iris image is higher than that of the first iris image, and the second iris image is used for identity authentication, so that a more accurate identity authentication result can be obtained.

In some embodiments, the position adjustment module may perform the move operation. Referring to fig. 4, fig. 4 is a flowchart illustrating a method for acquiring an iris image based on a head-mounted display according to an embodiment of the present disclosure, where the method is applicable to any electronic device such as a terminal device, a service device, or a head-mounted display body shown in the foregoing operating environment. Referring to fig. 4, the method shown in fig. 4 may include the following steps:

410. a first iris image generated by an imaging sensor based on the incident light is acquired.

420. An iris region in the first iris image is determined, and a region area of the iris region is compared with a region area of a reference region.

In an embodiment of the application, the iris region may be located in the first iris image by an iris feature based target detection method. After comparing the area of the iris region with the area of the reference region, if the area of the iris region is not equal to the area of the reference region, the step 430 may be continued.

430. And determining the moving direction of the position adjusting module to execute the moving operation according to the area difference between the iris area and the reference area.

In the embodiment of the present application, the area difference between the iris region and the reference region can be at least used for determining the moving direction when the position adjustment module performs the moving operation.

When the area of the iris area is smaller than that of the reference area, the moving direction of the position adjusting module for executing the moving operation is determined to be the direction of increasing the distance between the imaging sensor and the camera shooting hole. Alternatively, the first and second electrodes may be,

when the area of the iris area is larger than that of the reference area, the moving direction of the position adjusting module for executing the moving operation is determined to be the direction of shortening the distance between the imaging sensor and the camera hole.

440. And controlling the position adjusting module to execute the moving operation according to the moving direction.

In an embodiment of the present application, the position adjustment module may include a telescopic link. When the moving direction is the direction of shortening the distance, the telescopic connecting rod is controlled to be stretched; when the moving direction is increased to the direction of the distance, the telescopic connecting rod is controlled to be shortened.

In some embodiments, in the step 440, when the position adjustment module is controlled to perform the moving operation according to the moving direction, the moving distance of each time the moving operation is performed may be a preset distance value. For example, a distance change amount corresponding to 1 mm, 2 mm per one movement operation may be set. Accordingly, after step 450 is executed, the second iris image generated by the imaging sensor can be used as a new first iris image, and the aforementioned step 420 is further executed until the area of the iris region in the first iris image is determined to be equal to the area of the reference region. Namely, the position adjusting module can be controlled to move a preset distance value every time, and the second iris image which meets the requirements is acquired through successive adjustment.

In other embodiments, when step 430 is performed, the area difference between the iris region and the reference region can also be used to determine the moving distance when the position adjustment module performs the moving operation. Wherein the area difference between the iris region and the reference region can be calculated, and the moving distance can be calculated by using the area difference. Accordingly, in step 440, the position adjustment module is controlled to move the calculated moving distance according to the moving direction. Namely, the position adjusting module can be controlled to move the calculated moving distance at one time, and the second iris image which meets the requirement is directly acquired through one-time adjustment.

450. A second iris image generated by the adjusted imaging sensor based on the incident light is acquired.

In the embodiment of the present application, if the area of the iris region in the first iris image is smaller than the area of the reference region, which indicates that the image formation of the iris in the first iris image is too small, the optical zoom factor of the imaging device needs to be increased, and the distance between the imaging sensor and the imaging hole needs to be increased. Therefore, when the distance between the imaging sensor and the camera hole is increased, the image of the iris in the second iris image generated by the imaging sensor becomes large.

If the area of the iris region in the first iris image is larger than that of the reference region, it is indicated that the iris image in the first iris image is too large and may be incomplete, and it is necessary to reduce the optical zoom factor of the camera device and shorten the distance between the imaging sensor and the camera hole. Therefore, after the distance between the imaging sensor and the camera hole is shortened, the imaging of the iris in the second iris image generated by the imaging sensor becomes small.

It can be seen that, in the foregoing embodiment, the position adjusting module may be controlled to perform a moving operation by a difference between the area of the iris region in the first iris image and the area of the reference region, so as to adjust the optical zoom factor of the camera device, and adjust the imaging size of the iris, so as to acquire the second iris image with maximized iris imaging.

In some embodiments, the position adjustment module may perform a rotation operation. Referring to fig. 5, fig. 5 is a flowchart illustrating a method for acquiring an iris image based on a head-mounted display according to an embodiment of the present disclosure, where the method is applicable to any electronic device such as a terminal device, a service device, or a head-mounted display body shown in the foregoing operating environment. Referring to fig. 5, the method shown in fig. 5 may include the following steps:

510. a first iris image generated by an imaging sensor based on the incident light is acquired.

520. And calculating the incidence angle of the light rays entering the imaging sensor according to the first iris image.

In the embodiment of the application, the first iris image can be converted into a gray image, and the gray mean value of each pixel point in the gray image is calculated. The mean value of the gray levels of the gray level map may be used to indicate the image brightness of the first iris image, the higher the mean value of the gray levels, the higher the image brightness. The incidence angle of the light can be calculated by using the image brightness of the first iris image.

In the embodiment of the present application, the incidence angle may be in the range of [0 ° -180 ° ], and the incidence angle may refer to an angle between a light ray and a plane in which the imaging sensor is located in the target direction. Wherein the target direction may be a direction from inside the head mounted display to outside the head mounted display. For example, in the head mounted display shown in fig. 2B, the target direction may be a direction perpendicular to the horizontal plane and upward; as with the head mounted display shown in fig. 2C, the target direction may be a direction parallel to the horizontal plane and to the right.

530. If the incident angle of the light is not a right angle, the rotating direction of the rotating operation is determined according to the incident angle.

In the embodiment of the present application, if the incident angle of the light is an obtuse angle, the rotation direction of the rotation operation can be determined to be counterclockwise rotation; or;

if the light incidence angle is an acute angle, the rotation direction of the rotation operation is determined to be clockwise rotation.

540. And controlling the position adjusting module to execute rotation operation according to the rotation direction.

In an embodiment of the present application, the position adjustment module may include a rotatable platform. When the rotating direction is clockwise rotation, the rotatable platform is controlled to rotate clockwise; when the rotation direction is counterclockwise, the rotatable platform is controlled to rotate counterclockwise.

550. A second iris image generated by the adjusted imaging sensor based on the incident light is acquired.

If the incident angle of the light beam incident into the imaging sensor is not a right angle, the imaging sensor cannot receive the light beam in the maximum area, so that the image brightness of the imaged first iris image is reduced, and the iris information is not easy to be accurately extracted from the first iris image. Therefore, in the embodiment of the application, the imaging sensor can be rotated through the position adjusting module, so that the angle of the light rays entering the imaging sensor is changed, and the image brightness of the second iris image generated after adjustment is improved.

For example, if the lens module of the head-mounted display is a Pancake optical module, due to the characteristics of the Pancake optical module, the light passes through the Pancake optical module and then has only one quarter of the brightness. Therefore, the imaging sensor can be rotated by the method shown in fig. 5 to adjust the area of the imaging sensor receiving the light, thereby improving the brightness of the generated image.

In this embodiment of the application, when the step 540 is executed to control the position adjusting module to execute the rotation operation according to the rotation direction, the rotation angle of each time of executing the rotation operation may be a preset angle value. For example, an angular change amount corresponding to 3 °, 5 °, 10 ° of the rotatable platform per rotation operation may be set. Accordingly, after step 550 is performed, the second iris image generated by the imaging sensor can be used as a new first iris image, and the aforementioned step 520 is further performed until it is determined that the incident angle is vertical. Namely, the position adjusting module can be controlled to rotate the preset angle value every time, and the second iris image which meets the requirement is acquired through successive adjustment.

Generally, the camera hole of the head-mounted display camera device is often disposed above, below, or on one side of the display screen, and is not directly disposed in the center of the display screen. Therefore, when the user looks at the display screen, the camera hole may not be located right in front of the eyes, which may cause distortion of the captured iris image, i.e. the inner and outer circles of the iris in the iris image are not concentric.

In some embodiments, the camera 24 of the head mounted display as shown in fig. 2A or fig. 2B may further include: and the correcting chip can be another integrated circuit module independent from the imaging sensor. Alternatively, the calibration chip may also be a part of the imaging sensor, and is not limited in particular. The correction chip can store a distortion correction model, and the distortion correction model is obtained by training sample data.

The sample data comprises a plurality of sample iris images which are respectively collected when a user looks at different positions of a display screen of the head-mounted display, and the sample data also comprises undistorted supervision iris images which respectively correspond to the sample iris images. That is, the sample data may include a plurality of iris image pairs, each iris image pair corresponding to a location of a user's gaze at the display screen, each iris image pair may include a sample iris image, and an undistorted supervised iris image corresponding to the iris image.

The training process of the aberration correction model may include: and inputting the sample iris image corresponding to each position of the display screen watched by the user and the position information watched by the user on the display screen into the distortion correction model to obtain an output result of the distortion correction model. Determining an output result and a loss parameter of a corresponding supervision iris image based on a preset loss function; and carrying out feedback updating on the distortion correction model according to the loss parameters until the training is finished so as to obtain the trained distortion correction model.

In addition, the head mounted display may further include an eye-tracking sensor for detecting a user's gaze.

Referring to fig. 6, fig. 6 is a flowchart illustrating a method for acquiring an iris image based on a head-mounted display according to an embodiment, where the method is applicable to any electronic device shown in the foregoing operating environment, such as a terminal device, a service device, or a body of the head-mounted display. Referring to fig. 6, the method shown in fig. 6 may include the following steps:

610. the user's gaze is detected by an eye tracking sensor.

620. When it is detected that the sight line of the user falls on a display screen of the head-mounted display, a first iris image generated by the imaging sensor based on the incident light is acquired.

630. And controlling the position adjusting module to execute the moving operation and/or the rotating operation according to the first iris image.

640. A second iris image generated by the adjusted imaging sensor based on the incident light is acquired.

The implementation of the foregoing steps 610-640 can be referred to the foregoing embodiments, and the following description is omitted.

650. A gaze location of a user's gaze on a display screen detected by an eye tracking sensor is determined.

In the embodiment of the application, the sight line of the user can be detected through the eye movement tracking sensor, and the watching position of the sight line of the user on the display screen is further calculated.

660. And transmitting the second iris image generated by the imaging sensor and the determined gaze position to the correction chip.

670. And carrying out distortion correction on the second iris image according to the fixation position through a distortion correction model stored in the correction chip to obtain a third iris image output by the distortion correction model.

In the embodiment of the application, the trained distortion correction model can be stored in the correction chip, and the distortion correction model can perform distortion correction on the second iris image by using the distortion correction capability learned in the training stage, so that a third iris image without distortion is output.

In the foregoing embodiment, the position of the imaging sensor according to the first iris image may be adjusted to adjust the imaging size of the iris in the second iris image and the image brightness of the second iris image. And further, carrying out distortion correction on the second iris image by using the trained distortion correction model according to the fixation position of the user watching the display screen, and further obtaining a third iris image without distortion. The third iris image with moderate imaging size, high image brightness and no distortion is beneficial to extracting more accurate iris information from the third iris image.

In addition, in some embodiments, the sample iris images used in training the aberration correction model may be acquired by a head-mounted display using Pancake optics as lens modules. Here, the image brightness of the supervised iris image in the sample data and the corresponding sample iris image may be different, and the image brightness of the supervised iris image may be higher than the image brightness of the sample iris image. Therefore, the trained distortion correction model can adjust the image distortion and improve the image brightness.

Accordingly, when the method for acquiring an iris image based on a head-mounted display is performed, if the head-mounted display also employs a Pancake optical module, the distortion correction model may further perform a brightness enhancement operation on the second iris image when performing step 670, and an image brightness of a third iris image output by the distortion correction model may be higher than an image brightness of the second iris image.

It should be understood that although the various steps in the flowcharts of fig. 3-6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 3-6 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an iris image capturing device based on a head-mounted display according to an embodiment. The iris image acquisition device can be applied to any electronic equipment such as terminal equipment, service equipment or a head-mounted display body shown in the running environment. As shown in fig. 7, the iris image collecting apparatus 700 may include: an imaging module 710 and a motion control module 720.

An imaging module 710 for acquiring a first iris image generated by the imaging sensor based on the incident light;

the motion control module 720 is used for controlling the position adjusting module to execute a moving operation and/or a rotating operation according to the first iris image; the moving operation is used for adjusting the distance from the imaging sensor to the camera hole so as to maximize the imaging of the iris in the image; the rotating operation is used for rotating the imaging sensor to change the angle of the light rays entering the imaging sensor;

the imaging module 710 is further configured to acquire a second iris image generated by the adjusted imaging sensor based on the incident light.

In one embodiment, the motion control module 720 may include: an iris recognition unit, a movement determination unit, and a movement execution unit.

An iris recognition unit operable to determine an iris region in the first iris image;

a movement determination unit operable to compare a region area of the iris region with a region area of a reference region; the reference region is an image region corresponding to the complete iris when imaging maximization is carried out in the image; determining the moving direction of the position adjusting module to execute the moving operation according to the area difference between the iris area and the reference area;

and the movement execution unit can be used for controlling the position adjustment module to execute movement operation according to the movement direction.

Optionally, the position adjustment module may include: a telescopic link; and the movement executing unit can be used for controlling the telescopic connecting rod to execute the movement operation according to the movement direction.

In an embodiment, the movement determining unit may be further configured to determine, when the area of the iris region is smaller than the area of the reference region, a moving direction in which the position adjusting module performs the moving operation as a direction of the increasing distance; or, the position adjusting module can be further used for determining the moving direction of the position adjusting module to perform the moving operation as the direction of the shortened distance when the area of the iris area is larger than that of the reference area.

In one embodiment, the motion control module 720 may include: an incident angle calculating unit, a rotation determining unit, and a rotation executing unit.

The incidence angle calculation unit can be used for calculating the incidence angle of the light rays entering the imaging sensor according to the first iris image; the incident angle is an included angle between the light and the plane where the imaging sensor is located in the target direction; the target direction is a direction from an interior of the head mounted display to an exterior of the head mounted display;

the rotation determining unit is used for determining the rotation direction of the rotation operation according to the incident angle after judging that the incident angle is a right angle;

and the rotation executing unit can be used for controlling the position adjusting module to execute rotation operation according to the rotation direction.

Optionally, the position adjustment module may include: the platform can be rotated. And the rotation execution unit can be used for controlling the rotatable platform to execute the rotation operation according to the rotation direction.

In one embodiment, the rotation determining unit may be further configured to determine that the rotation direction of the rotation operation is counterclockwise rotation when the incident angle is an obtuse angle; or; and the rotating direction of the rotating operation is determined to be clockwise rotation when the injection angle is acute.

In one embodiment, the head-mounted display based iris image capture device 700 may further include: the device comprises a sight line determining module, a data transmission module and a correcting module.

The sight line determining module can be used for determining the watching position of the sight line of the user on the display screen, which is detected by the eye movement tracking sensor;

the data transmission module can be used for transmitting the second iris image generated by the imaging sensor and the determined gaze position to the correction chip;

and the correction module can be used for carrying out distortion correction on the second iris image according to the gaze position through the trained distortion correction model stored in the correction chip to obtain a third iris image output by the distortion correction model.

It can be seen that, the iris image acquisition device based on the head-mounted display disclosed in the foregoing embodiment can capture the first iris image first, and adjust the distance between the imaging sensor and the camera hole and/or the angle at which light enters the imaging sensor by using the first iris image. And the second iris image generated by the adjusted imaging sensor has the imaging size of the iris in the second iris image closer to the maximized imaging and higher image brightness. Therefore, the image quality of the second iris image is higher than that of the first iris image, and the second iris image is used for identity authentication, so that a more accurate identity authentication result can be obtained.

For specific limitations of the iris image capturing device based on the head-mounted display, reference may be made to the above limitations of the iris image capturing method based on the head-mounted display, and details thereof are not repeated herein. The modules in the head-mounted display-based iris image acquisition device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent of a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment, where the electronic device may be any one of a terminal device, a service device, or a head-mounted display body shown in the foregoing operating environment. As shown in fig. 8, the electronic device 800 may include:

a memory 810 storing executable program code;

a processor 820 coupled to the memory 810;

the processor 820 calls the executable program code stored in the memory 810 to execute any one of the methods for acquiring an iris image based on a head-mounted display disclosed in the embodiments of the present application.

Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the head-mounted display-based iris image capture device provided by the present application can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in fig. 8. Various program modules constituting the head-mounted display-based iris image capture apparatus, such as the imaging module 710 and the motion control module 720 shown in fig. 7, may be stored in the memory of the computer device. The program modules constitute computer programs to make the processor execute the steps of the method for acquiring iris images based on the head-mounted display according to the embodiments of the present application described in the specification.

For example, the computer device shown in fig. 8 can perform the step of acquiring a first iris image generated by the imaging sensor based on the incident light through the imaging module 710 in the head-mounted display based iris image acquisition device shown in fig. 7. The computer device may control the position adjustment module to perform the moving operation and/or the rotating operation according to the first iris image through the motion control module 720. The computer device may perform the step of acquiring a second iris image generated by the adjusted imaging sensor based on the incident light via the imaging module 710.

In one embodiment, an electronic device is provided, comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program:

acquiring a first iris image generated by an imaging sensor based on incident light;

controlling a position adjusting module to execute moving operation and/or rotating operation according to the first iris image; the moving operation is used for adjusting the distance from the imaging sensor to the camera hole so as to maximize the imaging of the iris in the image; the rotating operation is used for rotating the imaging sensor to change the angle of the light rays entering the imaging sensor;

a second iris image generated by the adjusted imaging sensor based on the incident light is acquired.

In one embodiment, the processor when executing the computer program further performs the steps of:

when the area of the iris area is smaller than that of the reference area, determining the moving direction of the position adjusting module for executing the moving operation as the direction of the increasing distance; alternatively, the first and second electrodes may be,

and when the area of the iris area is larger than that of the reference area, determining the moving direction of the position adjusting module for executing the moving operation as the direction of shortening the distance.

In one embodiment, the processor, when executing the computer program, further performs the steps of:

calculating the incidence angle of the light rays entering the imaging sensor according to the first iris image; the incidence angle is an included angle between the light and the plane where the imaging sensor is located in the target direction; the target direction is a direction from an interior of the head mounted display to an exterior of the head mounted display;

if the incident angle is a right angle, determining the rotation direction of the rotation operation according to the incident angle;

and controlling the position adjusting module to execute the rotating operation according to the rotating direction.

if the incident angle is an acute angle, the rotation direction of the rotation operation is determined to be clockwise rotation.

determining a gaze location of a user's gaze on a display screen detected by an eye tracking sensor;

transmitting the second iris image generated by the imaging sensor and the determined gaze position to a correction chip;

and carrying out distortion correction on the second iris image according to the gaze position through the trained distortion correction model stored in the correction chip to obtain a third iris image output by the distortion correction model.

The processor executes the computer program disclosed in the foregoing embodiment, and can first capture a first iris image by using the imaging sensor, and adjust the distance between the imaging sensor and the camera hole and/or the angle of the light entering the imaging sensor by using the first iris image. And the second iris image generated by the adjusted imaging sensor has the imaging size of the iris in the second iris image closer to the maximized imaging and higher image brightness. Therefore, the image quality of the second iris image is higher than that of the first iris image, and the second iris image is used for identity authentication, so that a more accurate identity authentication result can be obtained.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor implements the steps of:

determining an iris region in the first iris image and comparing a region area of the iris region to a region area of a reference region; the reference region is an image region corresponding to the imaging maximization of the complete iris in the image;

calculating the incidence angle of the light rays entering the imaging sensor according to the first iris image;

if the incident angle is a right angle, determining the rotation direction of the rotation operation according to the incident angle; the incident angle is an included angle between the light and the plane where the imaging sensor is located in the target direction; the target direction is a direction from an interior of the head mounted display to an exterior of the head mounted display;

and controlling the position adjusting module to execute the rotation operation according to the rotation direction.

When the computer program is executed by the processor, the first iris image can be firstly shot by the imaging sensor, and the distance between the imaging sensor and the camera hole and/or the angle of the light rays entering the imaging sensor can be adjusted by the first iris image. And the second iris image generated by the adjusted imaging sensor has the imaging size of the iris in the second iris image closer to the maximized imaging and higher image brightness. Therefore, the image quality of the second iris image is higher than that of the first iris image, and the second iris image is used for identity authentication, so that a more accurate identity authentication result can be obtained.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also appreciate that the embodiments described in this specification are all alternative embodiments and that the acts and modules involved are not necessarily required for this application.

In various embodiments of the present application, it should be understood that the size of the serial number of each process described above does not mean that the execution sequence is necessarily sequential, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated units, if implemented as software functional units and sold or used as a stand-alone product, may be stored in a computer accessible memory. Based on such understanding, the technical solution of the present application, which is a part of or contributes to the prior art in essence, or all or part of the technical solution, may be embodied in the form of a software product, stored in a memory, including several requests for causing a computer device (which may be a personal computer, a server, a network device, or the like, and may specifically be a processor in the computer device) to execute part or all of the steps of the above-described method of the embodiments of the present application.

It will be understood by those skilled in the art that all or part of the steps of the methods of the embodiments described above may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, including Read-Only Memory (ROM), random Access Memory (RAM), programmable Read-Only Memory (PROM), erasable Programmable Read-Only Memory (EPROM), one-time Programmable Read-Only Memory (OTPROM), electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc-Read-Only Memory (CD-ROM) or other Memory capable of storing data, a magnetic tape, or any other computer-readable medium capable of storing data.

The embodiments of the present application disclose a method for capturing iris images based on a head-mounted display and related products, and specific examples are applied herein to explain the principles and embodiments of the present application, and the above description of the embodiments is only provided to help understand the method and its core idea of the present application. Meanwhile, for a person skilled in the art, according to the idea of the present application, the specific implementation manner and the application scope may be changed, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. An iris image acquisition method based on a head-mounted display is characterized in that the head-mounted display comprises a light source and a camera device, and the camera device comprises: the device comprises a camera hole, an imaging sensor and a position adjusting module of the imaging sensor; the camera hole is arranged towards a wearer of the head-mounted display; the light emitted by the light source is shot Kong Sheru; and, the method comprises:

acquiring a first iris image generated by the imaging sensor based on the incident light;

2. The method of claim 1, wherein controlling the position adjustment module to perform a movement operation according to the first iris image comprises:

3. The method of claim 2, wherein determining a moving direction of the position adjustment module to perform the moving operation according to the area difference between the iris region and the reference region comprises:

4. The method of claim 1, wherein controlling the position adjustment module to perform a rotation operation according to the first iris image comprises:

if the incidence angle is not a right angle, determining the rotation direction of the rotation operation according to the incidence angle;

5. The method of claim 4, wherein said determining a rotation direction of the rotation operation based on the angle of incidence comprises:

if the incidence angle is an obtuse angle, determining that the rotating direction of the rotating operation is anticlockwise rotation; or;

and if the injection angle is an acute angle, determining that the rotating direction of the rotating operation is clockwise rotation.

6. The method of any of claims 1-5, wherein the position adjustment module comprises: a telescopic link; one end of the telescopic connecting rod is connected with the imaging sensor, and the other end of the telescopic connecting rod is connected with the shell of the head-mounted display; the telescopic connecting rod is used for executing the moving operation; and/or the presence of a gas in the gas,

7. The method according to any one of claims 1-5, wherein the imaging device further comprises: correcting the chip; the head mounted display further comprises: an eye tracking sensor;

the correction chip stores a trained distortion correction model; the method further comprises the following steps:

and carrying out distortion correction on the second iris image according to the fixation position through the distortion correction model stored in the correction chip to obtain a third iris image output by the distortion correction model.

8. The utility model provides an iris image acquisition device based on head mounted display, its characterized in that, head mounted display includes light source and camera device, camera device includes: the device comprises a camera hole, an imaging sensor and a position adjusting module of the imaging sensor; the camera hole is arranged towards a wearer of the head-mounted display; the light emitted by the light source is shot Kong Sheru; and, the apparatus comprises:

9. An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and wherein the computer program, when executed by the processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.