WO2021197466A1

WO2021197466A1 - Eyeball detection method, apparatus and device, and storage medium

Info

Publication number: WO2021197466A1
Application number: PCT/CN2021/085237
Authority: WO
Inventors: 张小伟; 项伟; 刘更代
Original assignee: 百果园技术(新加坡)有限公司; 张小伟
Priority date: 2020-04-03
Filing date: 2021-04-02
Publication date: 2021-10-07
Also published as: CN111476151B; CN111476151A

Abstract

Disclosed are an eyeball detection method, apparatus and device, and a storage medium. The method comprises: acquiring a target eye image to be subjected to detection; inputting the target eye image into a pretrained eyeball detection model, wherein the eyeball detection model is a convolutional neural network model including a reversible residual network; and determining position information of an eyeball key point in the target eye image according to an output result from the eyeball detection model.

Description

Eyeball detection method, device, equipment and storage medium

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office with the application number 202010261001.7 on April 3, 2020, and the entire content of the application is incorporated into this application by reference.

Technical field

The embodiments of the present application relate to the field of image recognition, such as eyeball detection methods, devices, equipment, and storage media.

Background technique

Eyeball detection technology generally includes eyeball key point positioning technology, which is an important technology in the field of image processing and computer vision. The eyeball detection technology is to accurately locate the iris and pupil in the input face image or video. The eyeball detection technology mainly includes the detection of the iris boundary or the key points on the boundary and the detection of the pupil center point. Eyeball detection technology plays an important role in the fields of live entertainment, short video special effects, virtual dolls, and security.

Eyeball detection methods can be roughly divided into two categories, one is based on manual feature extraction methods in the traditional computer vision field, and the other is based on neural network technology. The manual feature extraction method based on the traditional computer vision field mainly uses the gradient of the image to extract features, such as scale-invariant feature transform (SIFT) features, and combines traditional algorithms (such as Hough transform and support vector machine). Etc.) Do iris edge detection or key point detection. This kind of scheme needs to set different parameters for different scenarios, and the accuracy is generally low. The method based on neural network technology mainly uses the multi-layer convolutional neural network to extract the features of the image, and then return to the position of the key point. This kind of scheme is more accurate than the former, but the computational complexity of the model is high, which has a great impact on computing resources. High demands. Therefore, the eyeball detection scheme in the related technology is still not perfect and needs to be improved.

Summary of the invention

The embodiments of the present application provide eyeball detection methods, devices, equipment, and storage media, which can optimize eyeball detection solutions in related technologies.

The embodiment of the present application provides an eyeball detection method, which includes: acquiring a target eye image to be detected; inputting the target eye image into a pre-trained eyeball detection model, wherein the eyeball detection model is A convolutional neural network model including a reversible residual network; the position information of key eyeball points in the target eye image is determined according to the output result of the eyeball detection model.

An embodiment of the present application provides an eyeball detection device, which includes: a target eye image acquisition module configured to acquire a target eye image to be detected; an image input module configured to input the target eye image to a preset In the trained eyeball detection model, the eyeball detection model is a convolutional neural network model including a reversible residual network; a position information determination module is configured to determine the target eye image according to the output result of the eyeball detection model The position information of the key points of the eyeballs in.

The embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, Eyeball detection method.

The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the eyeball detection method as provided in the embodiment of the present application is implemented.

Description of the drawings

FIG. 1 is a schematic flowchart of an eyeball detection method provided by an embodiment of this application;

FIG. 2 is a schematic diagram of the distribution of key eyeball points according to an embodiment of the application;

FIG. 3 is a schematic flowchart of another eyeball detection method provided by an embodiment of the application;

4 is a schematic flowchart of another eyeball detection method provided by an embodiment of the application;

FIG. 5 is a schematic diagram of a flow of eyeball detection provided by an embodiment of this application;

FIG. 6 is a schematic diagram of a network structure of an eyeball detection model provided by an embodiment of this application;

FIG. 7 is a schematic structural diagram of a reversible residual network provided by an embodiment of this application;

FIG. 8 is a structural block diagram of an eyeball detection device provided by an embodiment of the application;

FIG. 9 is a structural block diagram of a computer device provided by an embodiment of this application.

Detailed ways

The application will be described below with reference to the drawings and embodiments. It can be understood that the embodiments described here are only used to explain the application, but not to limit the application. FIG. 1 is a schematic flow chart of an eyeball detection method provided by an embodiment of the application. The method can be executed by an eyeball detection device, which can be implemented by software and/or hardware, and generally can be integrated in a computer device. As shown in Figure 1, the method includes the following steps.

Step 101: Acquire an eye image of a target to be detected.

Exemplarily, the computer device may include mobile terminal devices such as mobile phones, tablet computers, notebook computers, and personal digital assistants, and may also include other devices such as desktop computers. In addition, the embodiment of the present application can effectively improve the calculation efficiency of eyeball detection while ensuring the accuracy of eyeball detection, and the calculation complexity is also effectively controlled. Therefore, the method provided in this embodiment can be widely applied to mobile computing platforms and other computing resources. Restricted platforms, that is, computer equipment can be equipment with limited computing resources, such as low-end (such as low hardware configuration) mobile phones and security equipment. Tests have shown that computer equipment can reach millisecond-level operating speeds.

The solutions provided by the embodiments of the present application can be applied to a variety of application scenarios, such as tracking the direction of the user's line of sight, eye tracking, and other applications that require the use of eyeball position related information. Optionally, it can be used in special effects, stickers, virtual dolls and 3-dimensional (3D) expressions in live video or short video applications, and can also be used in security equipment to assist iris faces Recognition and live detection, etc.

Exemplarily, the target eye image may be an image containing human eyes. The proportion of the human eye area in the entire target eye image is not limited, and the target eye image may include other parts of the facial features of the human face, or may only include human eyes, which is not limited in the embodiment of the present application.

Optionally, for some application scenarios, the original image collected by an image capture device such as a camera generally contains the entire face, and may also contain other image information such as the background of the person. Therefore, the original image can be cropped, etc. Operation to obtain the target eye image to reduce the amount of calculation.

Step 102: Input the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network model including a reversible residual network.

The eyeball detection model used for eyeball detection used in the embodiments of the present application may be a convolutional neural network model including a reversible residual network. Eyeball detection models in related technologies generally use more layers of convolutional networks. The computational complexity is very high and cannot be used on devices with limited computing resources. Moreover, due to the high computational complexity, the calculation speed is also greatly affected. The calculation efficiency is low, which affects the real-time performance of eyeball detection. In order to reduce the computational complexity, the embodiment of the present application applies the reversible residual network to the eyeball detection model. One or more modules based on the reversible residual network can be set in the model, which can improve the calculation efficiency while ensuring accuracy. The position of the reversible residual network in the eyeball detection model, the number of reversible residual networks, and the parameters in the reversible residual network can be set according to actual applications and scenarios, which are not limited in the embodiment of the present application. In addition, the eyeball detection model may also include a convolutional layer, a pooling layer, and a fully connected layer, etc. The structure of the eyeball detection model is not limited in this embodiment. The convolutional layer can be recombined and designed to balance the neural network. Accuracy and complexity, reduce the complexity of the network while maintaining accuracy.

Exemplarily, the network structure corresponding to the eye detection model can be determined according to actual needs, and the eye detection training model can be obtained, and the training data can be used to train the eye detection training model to optimize the selection of multiple parameters in the eye detection training model. Value to obtain a trained eyeball detection model, that is, the pre-trained eyeball detection model in the embodiment of the present application.

Step 103: Determine position information of key eyeball points in the target eye image according to the output result of the eyeball detection model.

Exemplarily, the key points of the eyeball in the target eye image may include, for example, points around the iris, and may also include the center point of the pupil. The number of key points of the eyeball is not limited, for example, 20, which may include 19 points on the periphery of the iris and the center point of the pupil.

The position information of the eyeball key points in the target eye image may include information related to the position of the eyeball key points, such as coordinate information of the eyeball key points, and visibility information of the eyeball key points. The coordinate information may include the plane coordinate values of the eyeball key points in the target eye image, and the visibility information may include whether the eyeball key points are occluded by the eyelids. Figure 2 is a schematic diagram of a distribution of key eyeball points provided by an embodiment of the application. As shown in the figure, a total of 20 key points are marked, among which the points numbered 11 to 17 are hidden by the eyelids, and the visibility information is invisible point.

Exemplarily, the training data used for model training may be marked according to the content contained in the location information. For example, a preset number of eye images can be selected, the key point coordinates and the visibility of the key points in the eye image can be marked to obtain the training eye image, and the training eye image can be used for model training, where, The preset number can be set according to actual requirements such as model accuracy and accuracy, and is generally tens of thousands of levels, such as 60,000.

The eyeball detection method provided in the embodiments of the present application obtains a target eye image to be detected, and inputs the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network including a reversible residual network The network model determines the position information of the key points of the eyeball in the target eye image according to the output result of the eyeball detection model. By adopting the above technical solution, since the pre-trained eyeball detection model is a convolutional neural network model that includes a reversible residual network, it can effectively improve the calculation efficiency of eyeball detection while ensuring the accuracy of eyeball detection, quickly obtain eyeball detection results, and improve eyeballs Detect the response speed of related applications.

FIG. 3 is a schematic flow diagram of another eyeball detection method provided by an embodiment of the application. Based on the above-mentioned multiple optional embodiments, the acquisition of the target eye image to be detected is described.

Exemplarily, the acquiring the target eye image to be detected may include: using a preset face detection method to detect the image to be detected to determine the position information of the corners of the eyes; intercepting the binocular images according to the position information of the corners of the eyes; The binocular image determines the target eye image. The advantage of this setting is that it can further reduce the amount of calculation and improve the detection efficiency. The binocular image may be an image containing both the left eye and the right eye, and the binocular image may also be two images containing the left eye image and the right eye image respectively. Optionally, the capturing binocular images according to the position information of the corners of the eyes includes: respectively capturing a left-eye image and a right-eye image according to the position information of the corners of the eyes. The advantage of this setting is that it facilitates targeted detection for the left eye and the right eye, and can effectively control the scale of the eyeball detection model.

Optionally, determining the target eye image according to the binocular image includes: reducing and adjusting the binocular image to a preset size to obtain the target eye image. The advantage of this setting is that the amount of calculation can be further controlled. The input picture, that is, the image to be detected, may be of a large size, such as some high-definition images. If the intercepted binocular images are directly used as the target eye image and input into the eye detection model, it will bring a greater computational burden and is more accurate The degree increase has a limited effect, so the size can be reduced while ensuring the accuracy, and the preset size can be obtained. The preset size can be set according to actual needs. Different types of binocular images have different corresponding preset sizes. Taking a binocular image as an image that contains both left and right eyes as an example, the preset size can be 30 pixels * 90 pixels; for a binocular image that contains both left and right eye images, the preset size can be 30 pixels * 30 Pixels.

Optionally, the method includes the following steps.

Step 301: Use a preset face detection method to detect the image to be detected to determine the position information of the corner of the eye.

Exemplarily, the image to be detected may be an image containing a human face, for example, it may be derived from a live video image, an image in a surveillance video, etc., and the source of the image to be detected is not limited. The preset face detection method can be selected according to the actual situation, such as the SIFT method. The corner position information may include position information of the two inner corners of the left eye and the two inner corners of the right eye in the image to be detected, such as coordinate information.

Step 302: Separately intercept the left-eye image and the right-eye image according to the position information of the corner of the eye.

Exemplarily, taking the left-eye image as an example, a rectangular cropping frame can be constructed with the distance between two points corresponding to the two inner corners of the left eye being one side length. Optionally, in order to ensure that this cropping frame can contain the entire eye, the rectangular frame can be expanded outward by a preset ratio, and the preset ratio can be set according to actual needs. For example, if the distance between the two points corresponding to the two inner corners of the eye is L, the preset ratio is k, and the rectangle is a square, then the side length of the square is kL, and L and k are all greater than zero. The square cropping frame can be centered on the midpoint of the line connecting the two inner corners of the eye.

Exemplarily, capturing the left-eye image and the right-eye image separately according to the eye corner position information may include: determining the two inner corner points of each eye according to the eye corner position information corresponding to each of the left eye and the right eye Relative position; rotate the image to be detected according to the relative position, so that the two inner corner points of each eye are on the same horizontal line; intercept the image of each eye. The advantage of this setting is that due to the different head postures and different shooting angles, the connection between the two inner corner points may not be on the same horizontal line. After the image to be detected is rotated, the two inner corners can be rotated. The corners of the eyes are adjusted to be on the same horizontal line, so that the captured left-eye and right-eye images are more standard, ensuring that the images input to the network have less changes and have roughly the same layout, which facilitates the eye detection model to quickly and accurately locate the key points.

The rotating the image to be detected according to the relative position so that the two inner corner points of each eye are on the same horizontal line includes: calculating the two inner corner points of each eye according to the relative position. The center point of the line connecting the corners of the eyes; calculating the included angle between the horizontal line passing through the center point and the line connecting the two inner corners of each eye; determining the rotation matrix according to the included angle; based on the rotation matrix Rotate the image to be detected so that the two inner corner points of each eye are on the same horizontal line. The advantage of this setting is that the image to be inspected can be rotated more accurately.

Optionally, the training data corresponding to the eyeball detection model includes training eye images that have undergone random perturbation processing and random rotation processing. The advantage of this setting is that it can improve the robustness of the model. Random rotation processing can be performed for the crop frame. For example, the crop frame is rotated at a random angle with a certain probability, and the range of the random angle can be preset, such as 1 degree to 5 degrees.

Step 303: The left-eye image and the right-eye image are respectively reduced and adjusted to a preset size to obtain a target eye image.

Exemplarily, the left-eye image and the right-eye image are respectively reduced and adjusted to a size of 30*30 to obtain the target left-eye image and the target right-eye image.

Step 304: Input the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network model including a reversible residual network.

Step 305: Determine coordinate information and visibility information of key eyeball points in the target eye image according to the output result of the eyeball detection model.

If the image to be detected is rotated before the eye image is intercepted, optionally, the position information of the key eye points in the target eye image includes the coordinates of the key eye points in the image to be detected Information; the determining the position information of the key eyeball points in the target eye image according to the output result of the eyeball detection model includes: determining the eyeball in the target eye image according to the output result of the eyeball detection model Relative position information of key points; performing reverse rotation processing on the relative position information based on the rotation matrix to obtain coordinate information of key eye points in the target eye image in the image to be detected. The advantage of this setting is that it can accurately calculate the coordinate information of the key points of the eyeball in the image to be detected, and provide a basis for subsequent related applications such as special effects and stickers. Performing the reverse rotation processing on the relative position information based on the rotation matrix, for example, may include: calculating the reverse rotation matrix according to the rotation matrix, calculating the product of the reverse rotation matrix and the coordinate information contained in the relative position information, to obtain the target eye The coordinate information of the key eyeball points in the image in the image to be detected, and the inverse rotation matrix is the inverse matrix of the rotation matrix.

The relative position information of the key eyeball points is the position information of the key eyeball points in the to-be-detected image after the rotation.

The eyeball detection method provided by the embodiment of the present application uses a preset face detection method to detect the image to be detected to determine the position information of the corner of the eye, intercept the left eye image and the right eye image according to the corner position information, and perform size reduction processing to determine the corresponding The target left-eye image and target right-eye image can effectively reduce the amount of calculation, effectively control the scale of the eyeball detection model, and improve the detection efficiency.

FIG. 4 is a schematic flowchart of another eyeball detection method provided by an embodiment of the application, which is described on the basis of the foregoing multiple optional embodiments.

Exemplarily, the inputting the target eye image into a pre-trained eyeball detection model, and determining the position information of key eyeball points in the target eye image according to the output result of the eyeball detection model includes: Input the first target eye image into a pre-trained eyeball detection model, and determine the position information of key eyeball points in the first target eye image according to the first output result of the eyeball detection model; The eye image is flipped horizontally, and the second eye image after the horizontal flip is input into the eyeball detection model, and the second target eye after the horizontal flip is determined according to the second output result of the eyeball detection model The position information of the eyeball key points in the image is horizontally flipped to obtain the position information of the eyeball key points in the second target eye image; wherein the first target eye image is the target For a left eye image, the second target eye image is a target right eye image; or, the first target eye image is a target right eye image, and the second target eye image is a target left eye image. The advantage of this setting is that using the symmetrical relationship between the left and right eyes, only one eye detection model needs to be trained for one of the eyes, and then this model can also be used on the other eye, that is, a model is dual-purpose, without separate training Two models to improve the training efficiency and scope of application of the model.

FIG. 5 is a schematic diagram of a flow of eyeball detection provided by an embodiment of this application, and the embodiment of this application can be described with reference to FIG. 5.

Optionally, the method includes the following steps.

Step 401: Use a preset face detection method to detect the image to be detected to determine the position information of the corner of the eye.

Step 402: Separate the left-eye image and the right-eye image according to the position information of the corner of the eye.

Step 403: The left-eye image and the right-eye image are respectively reduced and adjusted to a preset size to obtain the target left-eye image and the target right-eye image.

Exemplarily, the left-eye image and the right-eye image are respectively reduced and adjusted to a size of 30*30 pixels to obtain the target left-eye image and the target right-eye image.

Step 404: Input the target right-eye image into the pre-trained eyeball detection model, and determine the position information of the key eyeball points in the target right-eye image according to the first output result of the eyeball detection model.

Exemplarily, the eyeball detection model includes multiple reversible residual networks, and also includes a convolutional layer, a pooling layer, and a fully connected layer. From input to output, the eye detection model includes a convolutional layer, a pooling layer, a reversible residual network, a pooling layer, a reversible residual network, a pooling layer, a reversible residual network, and a fully connected layer. The eye detection model may include at least two fully connected layers, the coordinate information of the key eyeball points in the target eye image is determined according to the output of the first fully connected layer, and the output of the preset activation function under the second fully connected layer is determined Determine visibility information of key eyeball points in the target eye image. The preset activation function may be, for example, a sigmoid function.

FIG. 6 is a schematic diagram of a network structure of an eyeball detection model provided by an embodiment of the application. As shown in FIG. 6, an optional network structure of the eyeball detection model may include sequentially connected convolutional layers and first maximum pooling. Layer, the first reversible residual module (reversible residual network), the second maximum pooling layer, the second reversible residual module, the third maximum pooling layer, the third reversible residual module, the fourth reversible residual module, and The third fully connected layer (C64), the third fully connected layer connects the first fully connected layer (C40) and the second fully connected layer (C20).

In the embodiment of the present application, the input image size of the model can be reduced to 30 pixels, that is, the size of 30*30 pixels. The C in the convolutional layer and the fully connected layer in Figure 6 represent the number of output channels in the convolutional layer and the fully connected layer respectively. For example, 3x3 convolution C8 means that the current layer is a 3x3 convolutional layer and outputs 8 features picture. Maximum pooling uses 2x2 pooling. The embodiment of the application uses the structure of the reversible residual module to improve the accuracy of the model. Figure 7 is a schematic structural diagram of a reversible residual network provided by an embodiment of the application. As shown in Figure 7, the figure shows that the number of input feature channels is m, the expansion parameter is k, and the number of output feature channels is n The reversible residual module in Figure 6, the values in the reversible residual module in Figure 6 represent m, k, and n, such as (8,8,1) indicates that the number of input feature channels of the first reversible residual module is 8, expansion The parameter is 8, the number of output characteristic channels is 1, and the value in each reversible residual module can be set according to actual needs. In the network structure shown in Figure 6, after each convolutional layer, a batch normalization (BN) normalization layer and a linear rectification function (Rectified Linear Unit, ReLU) activation layer can also be set. The setting of the BN normalization layer can make the training objective function better converge; the setting of the ReLU activation layer can increase the nonlinearity of the network. The fully connected layer C20 in Figure 6 will get 20 values. The sigmoid activation function that follows independently operates on the 20 values, and outputs 20 numbers between 0 and 1, which can be used as the probability of whether the corresponding key point is visible. 0 means completely invisible, 1 means completely visible.

Step 405: Perform a horizontal flip of the target left-eye image, and input the horizontally flipped target left-eye image into the eyeball detection model, and determine the eyeball in the horizontally flipped target left-eye image according to the second output result of the eyeball detection model For the position information of the key point, the position information is horizontally flipped to obtain the position information of the key point of the eyeball in the left-eye image of the target.

Optionally, in the embodiment of the present application, the target right-eye image and the target left-eye image may be input into the network separately, or the target right-eye image and the horizontally flipped target left-eye image may be combined and input into the network. Make a limit.

Step 406: Summarize the position information of the key eyeball points in the right-eye image of the target and the position information of the key eyeball points in the left-eye image of the target to obtain a detection result of the key eyeball points.

The eyeball detection method provided in the embodiment of the application trains the eyeball detection model for one eye, and when the eyeball is detected, the other eye is horizontally flipped to achieve the purpose of detecting two eyes by the same model, and the model is improved The scope of application. In addition, by optimizing the network structure of the eyeball detection model in the embodiments of the present application, it is possible to effectively improve the calculation efficiency of eyeball detection while ensuring the accuracy of eyeball detection, quickly obtain eyeball detection results, and improve the response speed of eyeball detection related applications.

FIG. 8 is a structural block diagram of an eyeball detection device provided by an embodiment of the application. The device can be implemented by software and/or hardware, generally can be integrated in a computer device, and eyeball detection can be performed by executing an eyeball detection method. As shown in FIG. 8, the device includes: a target eye image acquisition module 801, configured to acquire a target eye image to be detected; an image input module 802, configured to input the target eye image to a pre-trained eyeball detection In the model, the eyeball detection model is a convolutional neural network model including a reversible residual network; the position information determination module 803 is configured to determine the eyeball in the target eye image according to the output result of the eyeball detection model Location information of key points.

The eyeball detection device provided in the embodiments of the present application acquires a target eye image to be detected, and inputs the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network including a reversible residual network The network model determines the position information of the key points of the eyeball in the target eye image according to the output result of the eyeball detection model. By adopting the above technical solution, since the pre-trained eyeball detection model is a convolutional neural network model that includes a reversible residual network, it can effectively improve the calculation efficiency of eyeball detection while ensuring the accuracy of eyeball detection, quickly obtain eyeball detection results, and improve eyeballs Detect the response speed of related applications.

The embodiment of the present application provides a computer device, and the eyeball detection device provided in the embodiment of the present application can be integrated in the computer device. FIG. 9 is a structural block diagram of a computer device provided by an embodiment of this application. The computer device 900 includes a memory 901, a processor 902, and a computer program that is stored on the memory 901 and can run on the processor 902. The processor 902 implements the eye detection method provided in the embodiment of the present application when the computer program is executed.

The embodiment of the present application also provides a storage medium containing computer-executable instructions, which are used to execute the eyeball detection method provided by the embodiments of the present application when the computer-executable instructions are executed by a computer processor.

The eyeball detection device, device, and storage medium provided in the above embodiments can execute the eyeball detection method provided by any embodiment of the present application, and have corresponding functional modules for executing the method. For technical details not described in the above embodiments, please refer to the eyeball detection method provided in any embodiment of the present application.

Claims

An eyeball detection method, including:

Acquiring an image of the target eye to be detected;

Inputting the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network model including a reversible residual network;

The position information of key eyeball points in the target eye image is determined according to the output result of the eyeball detection model.
The method according to claim 1, wherein the position information of the eyeball key point includes at least one of the coordinate information of the eyeball key point and the visibility information of the eyeball key point.
The method according to claim 1, wherein said acquiring an image of the target eye to be detected comprises:

Detect the image to be detected by using a preset face detection method to determine the position information of the corner of the eye;

Intercepting binocular images according to the position information of the corners of the eyes;

The target eye image is determined based on the binocular images.
The method according to claim 3, wherein the capturing binocular images according to the position information of the corners of the eyes comprises:

The left-eye image and the right-eye image are respectively intercepted according to the position information of the corner of the eye.
The method according to claim 4, wherein the corner of the eye position information includes corner position information of the left eye and corner position information of the right eye;

Said capturing the left-eye image and the right-eye image separately according to the position information of the corner of the eye includes:

Determining the relative positions of the two inner corner points of each eye according to the corner position information corresponding to each of the left eye and the right eye;

Rotating the image to be detected according to the relative positions of the two inner corner points of each eye, so that the two inner corner points of each eye are on the same horizontal line;

Capture the image of each eye.
The method according to claim 5, wherein the image to be detected is rotated according to the relative positions of the two inner corner points of each eye, so that the two inner corner points of each eye are at The same horizontal line, including:

Calculating the center point of a line connecting the two inner corner points of each eye according to the relative positions of the two inner corner points of each eye;

Calculating the angle between the horizontal line passing through the center point and the line connecting the two inner corner points;

Determine the rotation matrix according to the included angle;

Rotate the image to be detected based on the rotation matrix, so that the two inner corner points of each eye are on the same horizontal line.
The method according to claim 6, wherein the position information of the key eyeball points in the target eye image includes coordinate information of the key eyeball points in the image to be detected;

The determining position information of key eyeball points in the target eye image according to the output result of the eyeball detection model includes:

The relative position information of the key eyeball points in the target eye image is determined according to the output result of the eyeball detection model; wherein the relative position information of the key eyeball points is the to-be-detected image of the key eyeball points after rotation Location information in;

Perform reverse rotation processing on the relative position information based on the rotation matrix to obtain coordinate information of key eyeball points in the target eye image in the image to be detected.
The method according to claim 3, wherein the determining the target eye image according to the binocular image comprises:

The binocular image is reduced and adjusted to a preset size to obtain the target eye image.
The method according to claim 4, wherein said inputting said target eye image into a pre-trained eyeball detection model, and determining the eyeball key in said target eye image according to the output result of said eyeball detection model Point location information, including:

Inputting a first target eye image into a pre-trained eyeball detection model, and determining position information of key eyeball points in the first target eye image according to a first output result of the eyeball detection model;

Perform horizontal flipping on the second target eye image, and input the horizontally flipped second eye image into the eyeball detection model, and determine the horizontally flipped second eye image according to the second output result of the eyeball detection model 2. The position information of the key eyeball points in the target eye image, and horizontally flipping the position information to obtain the position information of the key eyeball points in the second target eye image;

Wherein, the first target eye image is a target left eye image, and the second target eye image is a target right eye image; or, the first target eye image is a target right eye image, and the second target eye image is a target right eye image. The target eye image is the target left eye image.
The method according to claim 1, wherein the eyeball detection model includes a first fully connected layer and a second fully connected layer, and the eyeball in the target eye image is determined according to the output of the first fully connected layer The coordinate information of the key point determines the visibility information of the eyeball key point in the target eye image according to the output of the preset activation function that inherits the second fully connected layer.
An eyeball detection device, including:

The target eye image acquisition module is configured to acquire the target eye image to be detected;

An image input module, configured to input the target eye image into a pre-trained eyeball detection model, where the eyeball detection model is a convolutional neural network model including a reversible residual network;

The position information determining module is configured to determine the position information of key eyeball points in the target eye image according to the output result of the eyeball detection model.
A computer device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program when the computer program is executed as described in any one of claims 1-10 Methods.
A computer-readable storage medium that stores a computer program, and when the program is executed by a processor, the method according to any one of claims 1-10 is realized.