CN110705468B - Eye movement range identification method and system based on image analysis - Google Patents

Abstract

The invention provides an eye movement range identification method and system based on image analysis and processing, and relates to the field of image analysis and processing. The method comprises the following steps: carrying out primary positioning on a human eye image on an acquired video frame, preprocessing the human eye image, removing illumination influence, and determining an eye crack outline and an eye crack central point on the human eye image after illumination correction; further determining the iris outline and the iris center in the human eye image; the eye movement Range is determined based on the determined Iris Center point Center _ Iris and eye crack Center point Center _ eye. The eye movement range identification method and system based on image analysis can automatically analyze and detect based on the collected human eye movement video frames, and extract effective eyeball movement range characteristics by using an image processing method.

Description

Eye movement range recognition method and system based on image analysis

technical field

The invention relates to the field of image analysis and processing, in particular to an eye movement range recognition method and system based on image analysis.

Background technique

In clinic, the doctor's judgment of certain symptoms or diseases needs to be based on some behavioral indicators or parameters combined with other diagnostic theories and clinical experience to draw conclusions. For example, for Parkinson's disease (PD), the second most common neurodegenerative disease, it is clinically found that the eye movement range of PD patients is much smaller than that of normal people, so clinicians can combine the eye movement range and other patient behavior parameters and clinical experience. For the detection of eye movement range, eye tracking is mainly realized by eye tracker. However, the existing eye trackers are very expensive, and it is difficult to store and analyze a large amount of data; secondly, during the experiment, the freedom of the subjects is also limited. In addition, due to the fact that the gaze tracking technology is not yet fully mature and the characteristics of eye movement itself, the data may be interrupted during the experiment, and there are many problems such as interference signals, which eventually lead to the reduction of the accuracy of the gaze tracking. If the human eye movement range can be analyzed without using an eye tracker, the cost of eye movement analysis will be greatly reduced and the analysis accuracy will be improved.

SUMMARY OF THE INVENTION

Aiming at the above technical problems existing in the prior art, the present invention provides an eye movement range recognition method and system based on image analysis. Calculate the relative distance between the center of the iris and the center of the eye split, thereby realizing the analysis and evaluation of the range of motion of the human eye.

The technical scheme adopted in the present invention is as follows:

An eye movement range recognition method based on image analysis, comprising the following steps:

(1) carry out the initial positioning of the human eye image to the collected video frame I_frame, that is, carry out the preliminary positioning of the human eye range to all the collected images including the human eye, and obtain the human eye image I_eye after the preliminary positioning;

(2) preprocess the human eye image I_eye, and remove the influence of illumination to obtain the human eye image I_eye_color after illumination correction;

(3) determine the contour of the eye cleft Contour_eye and the center point of the eye cleft Center_eye to the human eye image I_eye_color after illumination correction;

(4) further determine the iris contour Contour_Iris and the iris center Center_Iris in the human eye image I_eye_color after illumination correction;

(5) Determine the eye movement range Range based on the determined iris center point Center_Iris and the eye split center point Center_eye.

Further, the step (1) specifically includes:

(1.1) Initial positioning of the human eye image for the first video frame:

The first video frame is set as I_frame(1), and the initial positioning of I_frame(1) includes calibrating the face with the training model of deep learning, and finding the position of the human eye crack from the calibrated feature points, using The rectangle fits the calibration point of the human eye, thereby determining the human eye image I_eye(1) containing the initial positioning of the human eye;

(1.2) Perform the initial positioning of the human eye image for the remaining video frames:

(1.2.1) Determine the range_eye of the human eye activity range of the current video frame;

Assuming that the current video frame is the nth frame, the image frame is denoted as I_frame(n), and the human eye activity range Range_eye of the current frame is determined by using the human eye image I_eye(n-1) after the initial positioning of the previous frame;

The horizontal coordinates of the left and right boundary points of the range_eye are Range_left_x and Range_right_x, and the vertical coordinates of the upper and lower boundary points are Range_top_y and Range_bottom_y, respectively:

Range_left_x=eye_left_x-W

Range_right_x=eye_right_x+W

Range_top_y=eye_top_y-H

Range_bottom_y=eye_bottom_y+H;

Among them, W and H are the width and height of the human eye image I_eye(n-1) respectively;

(1.2.2) Preliminary eye positioning is performed on the current video frame based on the range_eye of the human eye movement range of the current video frame, and the preliminary human eye positioning for all video frames is completed in turn;

Use the sliding window method to divide the range_eye of the human eye into multiple windows. The vertex corresponds to the upper left vertex of the range_eye of the current frame of the human eye;

Calculate the similarity between each window Window and the human eye image I_eye(n-1) of the previous frame, find the window Window with the highest similarity, and use the window Window with the highest similarity as the human eye image I_eye(n-1) of the current frame. ).

Further, the specific method for preprocessing the human eye image in the step (2) is: for the human eye image I_eye after the initial positioning, the multi-scale Retinex algorithm with color recovery is used to eliminate the uneven illumination of the extracted human eye image I_eye. The resulting effect, the preprocessed image I_eye_color is obtained.

Further, the step (3) determines the contour of the eye split Contour_eye and the center point of the eye split Center_eye to the human eye image I_eye_color after the illumination correction specifically includes:

(3.1) Obtain the Scharr gradient component image corresponding to the human eye image I_eye_color after illumination correction:

(3.1.1) Extract the Scharr gradient of the image I_eye_color in the RGB space;

First, use the Scharr operator to calculate the gradient of the image I_eye_color, and the Scharr operator includes the Scharr operator Gx in the horizontal direction and the Scharr operator Gy in the vertical direction;

If the gradient value of the (i,j)th pixel of the human eye image I_eye_color after illumination correction is Gx(i,j) in the horizontal direction and Gy(i,j) in the vertical direction, then the The total gradient value G(i,j) at the pixel point (i,j) is:

(3.1.2) Convert the initially extracted Scharr gradient image I_Scharr to HSV space, and extract the corresponding V component to obtain the corresponding component image I_ScharrV;

(3.2) Use the edge detection algorithm to perform edge detection on the human eye image I_eye_color to obtain the corresponding gradient image I_HED;

(3.3) Determine the position of the eye cleft contour Contour_eye based on the gradient component image I_ScharrV and the gradient image I_HED;

Further, in the described step (3.3), determining the position of the eye split contour Contour_eye specifically includes:

(3.3.1) The gradient component image I_ScharrV and the gradient image I_HED are combined to extract the corresponding binary image;

Let the maximum value of the pixel values of all pixels in the gradient component image I_ScharrV be V_Vmax, and the binarized image I_binary1 obtained when the threshold is set to V_Vmax/4 is:

Let the maximum value of the pixel values of all pixels in the gradient image I_HED be V_HEDmax, and the binarized image I_binary2 obtained when the threshold is set to V_HEDmax/3 is:

Add the binarized image I_binary1 and the binarized image I_binary2, and take the intersection of the two to obtain the processed binary image I_binary, that is:

I_binary(i,j)=I_binary1(i,j)&I_binary2(i,j);

Wherein, I_binary(i,j), I_binary1(i,j) and I_binary2(i,j) represent the pixel value of the (i,j)th pixel in the binary images I_binary, I_binary1 and I_binary2 respectively;

(3.3.2) Morphological processing is performed on the extracted binary image I_binary;

First perform the dilation operation on the binary image I_binary to obtain I_dilate, and then perform the erosion operation to obtain the image I_closing. The calculation formula of the closing operation is as follows:

Among them, A is a square structure element of 2*2, and I_dilate ^c is the complement of I_dilate;

(3.3.3) Further remove the redundant small objects in the image, and extract the largest connected domain in the image;

For the image I_closing obtained after the closing operation, the method of removing small objects in morphology is used to remove the small noise in the image I_closing to obtain the corresponding image I_morphological, and further extract the maximum connected domain from the image I_morphological as the corresponding eye cleft contour Contour_eye.

Further, the specific steps of extracting the largest connected domain in the image in the step (3.3.3) are as follows:

3.3.3.1) First, extract all contours in the image I_morphological, denoted as C_mor_set, namely:

C_mor_set={C_mor ₁ , C_mor ₂ , ..., C_mor _k1 , ..., C_mor _n1 }

Among them, C_mor _k1 (1≤k1≤n1) represents the k1th contour, and n1 is the total number of contours in the image I_morphological;

3.3.3.2) Next, calculate the area of each contour to obtain the area set Area_set, namely:

Area_set={Area ₁ , Area ₂ , ..., Area _k1 , ..., Area _n1 }

Among them, Area _k1 represents the area of the k1th contour;

3.3.3.3) Remove the connected domain whose area is less than 300 to obtain the corresponding image I_RemoveSmallObjects;

3.3.3.4) Find the contour C_mormax with the largest area in the image I_RemoveSmallObjects, and the contour C_mormax is the corresponding eye cleft contour Contour_eye.

Further, the concrete steps of determining the center point Center_eye of the eye split contour Contour_eye in the step (3.4) include:

For the eye cleft contour Contour_eye, find the left and right boundary points of the contour, which are Point_left and Point_right respectively, namely:

Point_left_x=min(Contour_eye_x)

Point_right_x=max(Contour_eye_x)

Among them, Point_left_x and Point_right_x represent the abscissas of Point_left and Point_right, and Contour_eye_x represent the abscissas of the points that make up the contour of the eye cleft;

Using the abscissas Point_left_x and Point_right_x of the left and right boundary points Point_left and Point_right, find the corresponding ordinates Point_left_y and Point_right_y in the eye cleft contour Contour_eye;

Take the mean value of the abscissas Point_left_x and Point_right_x of the left and right boundary points Point_left and Point_right as the abscissa value Center_eye_x of the center point Center_eye of the eye cleft, and take the mean value of the ordinates Point_left_y and Point_right_y of Point_left and Point_right as the ordinate value Center_eye_y of the eye cleft center Center_eye, Obtain the pixel coordinates (Center_eye_x, Center_eye_y) of the center point of the eye split Center_eye, namely:

Center_eye_x=(Point_left_x+Point_right_x)/2

Center_eye_y=(Point_left_y+Point_right_y)/2.

Further, the concrete steps of determining the iris contour Contour_Iris in the image and the iris center Center_Iris in the step (4) include:

(4.1) Image binarization

Use the Otsu threshold method to calculate the Otsu threshold Otsu_thresh for the gradient image I_ScharrV, and use the threshold Otsu_thresh to binarize the gradient image I_Scharr to obtain the corresponding binary image I_binary3:

(4.2) Morphological corrosion treatment

Use the 3*3 square structure element B to perform morphological corrosion on the image I_binary3, remove the burrs on the edge of the iris, disconnect the connection with the noise points, and obtain the corresponding image I_erosion:

(4.3) Hole filling and finding the largest connected domain

There are holes in the iris contour of the image I_erosion caused by the lights in the video recording, and the holes are filled by the morphological hole filling method to obtain the corresponding image I_holefilled; and further extract all the connected domains in the image I_holefilled, and the area is the largest. The connected domain of is where the iris is located;

(4.4) Determine the corresponding iris center point Center_Iris in the iris contour Contour_Iris:

The iris contour Contour_Iris is used to calculate its centroid, and the centroid point is the center point of the iris (Center_Iris_x, Center_Iris_y). The centroid calculation formula is as follows:

Among them, px(k) and py(k) (1≤k≤m) represent the abscissa and ordinate of the kth point on the iris contour Contour_Iris, respectively, and Contour_Iris(i,j) represents the (i,j)th pixel The pixel value of the point.

Further, the specific steps of finding the largest connected domain in the step (4.3) include:

(4.3.1) First, extract all the contours in the image I_holefilled to form the contour set C_hole_set, namely:

C_hole_set={C_hole ₁ , C_hole ₂ , ..., C_hole _k2 , ..., C_hole _n2 }

Among them, C_hole _k2 (1≤k2≤n2) represents the k2th contour, and n2 is the total number of contours in the image I_holefilled;

(4.3.2) Next, calculate the area of each contour to obtain the area set Area_set1, namely:

Area_set1={Area ₁ , Area ₂ , ..., Area _k2 , ..., Area _n2 }

Among them, Area _k2 (1≤k2≤n2) represents the area of the k2th contour;

(4.3.3) Find the contour C_holemax with the largest area, which is the corresponding iris contour Contour_Iris.

Further, the specific steps of determining the eye movement range Range based on the determined iris center point Center_Iris and the eye split center point Center_eye in the step (5) include:

After locating and determining the center point Center_eye of the eye split and the center point Center_Iris of the iris in each frame of image, the motion amplitude Mag of the iris is calculated, where the position values of the motion amplitude Mag in the x-direction and y-direction are Mag_x and Mag_y respectively:

Mag_x=Center_Iris_x-Center_eye_x

Mag_y=Center_Iris_y-Center_eye_y;

Calculate the maximum and minimum values of the iris movement amplitude Mag in the x-direction and y-direction, namely the maximum and minimum values of Mag_x and Mag_y in all image frames, and take the maximum value of Mag_x as the maximum rightward movement of the eyeball in the horizontal direction. Amplitude Mag_right, take the minimum value of Mag_x as the maximum amplitude Mag_left of the leftward movement of the eyeball in the horizontal direction, take the maximum value of Mag_y as the maximum amplitude of the downward movement of the eyeball in the vertical direction Mag_bottom, take the minimum value of Mag_y as the eyeball in the vertical direction. The maximum amplitude Mag_top of the upward movement in the vertical direction, namely:

Mag_right=max(Mag_x)

Mag_left=min(Mag_x)

Mag_bottom=max(Mag_y)

Mag_top=min(Mag_y)

Then the eye movement range corresponding to the image frame is Range: [Mag_left:Mag_right,Mag_top:Mag_bottom].

On the other hand, the present invention also provides an eye movement range recognition system based on image analysis, characterized in that the system is a recognition system composed of module units corresponding to the steps of any of the foregoing eye movement range recognition methods, so as to realize Identify the eye movement range in the captured video frames.

To sum up, due to the adoption of the above-mentioned technical solutions, the beneficial effects of the present invention are:

The eye movement range recognition method and system based on image analysis provided by the present invention can automatically analyze and detect based on the collected human eye movement video frames, and use the image processing method to extract effective eye movement range features. Compared with the prior art , the identification scheme of the present invention achieves a high degree of automatic measurement, overcomes the disadvantage of difficult data collection in the prior art, and the scheme is efficient, practical, objective and accurate.

Description of drawings

FIG. 1 is a schematic diagram of a flowchart of initial positioning of a human eye image provided by an embodiment of the present invention.

FIG. 2 is a schematic diagram of a first frame of a human right eye image provided by an embodiment of the present invention.

FIG. 3 is a schematic diagram of a position corresponding to the range of motion of the human right eye in a second frame in an image according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of a part of a window of a sliding window method provided by an embodiment of the present invention.

FIG. 5 is a schematic flowchart of eliminating the influence of illumination on a human eye image provided by an embodiment of the present invention.

FIG. 6 is a schematic flowchart of determining an eye cleft contour and an eye cleft center point for a human eye image according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of a gradient image provided by an embodiment of the present invention.

FIG. 8 is a schematic diagram of a gradient image and components in an HSV space provided by an embodiment of the present invention.

FIG. 9 is a schematic diagram of a gradient image after edge detection provided by an embodiment of the present invention.

FIG. 10 is a schematic diagram of a binarized image provided by an embodiment of the present invention.

FIG. 11 is a schematic diagram of a binarized image provided by an embodiment of the present invention.

FIG. 12 is a schematic diagram of a binarized image provided by an embodiment of the present invention.

FIG. 13 is a schematic diagram of an image after closing operation processing provided by an embodiment of the present invention.

FIG. 14 is a schematic diagram of a connected domain provided by an embodiment of the present invention.

FIG. 15 is a schematic diagram of an eye cleft profile provided by an embodiment of the present invention.

FIG. 16 is a schematic diagram of an eye cleft outline and an eye cleft center point provided by an embodiment of the present invention.

FIG. 17 is a schematic flowchart of determining an image iris contour and an iris center according to an embodiment of the present invention.

FIG. 18 is a schematic diagram of a binary image provided by an embodiment of the present invention.

FIG. 19 is a schematic diagram after morphological processing provided by an embodiment of the present invention.

FIG. 20 is a schematic diagram of a filled hole provided by an embodiment of the present invention.

FIG. 21 is a schematic diagram of an iris outline provided by an embodiment of the present invention.

FIG. 22 is a schematic diagram of an iris centroid point according to an embodiment of the present invention.

FIG. 23 is a schematic diagram of a flowchart for determining an eye movement range provided by an embodiment of the present invention.

Detailed ways

In order for those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be described clearly and completely below with reference to the accompanying drawings. Other similar embodiments obtained under the premise of no creative work shall fall within the scope of protection of the present application.

It should be noted that the image feature processing method, feature extraction method, signal identification and classification method proposed in the present invention and the corresponding embodiments are only for research and improvement of the image signal processing and identification method itself, although it is aimed at the human eye. The image signal acquired by the moving range and the realized range recognition result can be used as an evaluation reference, but in the clinical or medical field, the evaluation result is only an auxiliary evaluation, and the specific treatment method still needs and mainly depends on the clinical experience of the doctor and the treatment provided by the doctor.

Example 1

The present embodiment is an eye movement range recognition method based on image analysis, including the following steps:

(1) Preliminary positioning of the human eye image is performed on the collected video frames, that is, the preliminary positioning of the human eye range is performed on all the collected images including the human eye, as shown in Figure 1;

(1.1) Initial positioning of the human eye image for the first video frame:

The first frame of video frame is set as I_frame(1), and the captured image is an image captured by an image capture device on the target, and its capture range may be large. Therefore, in order to determine the position of the human eye cleft, the captured image can be determined. Perform initial positioning. Preliminary positioning of I_frame (1) includes calibrating the face with a deep learning training model, finding the position of the human eye crack from the calibrated feature points, and using a rectangle to fit the calibration point of the human eye, so as to determine whether the human eye is included. The human eye image I_eye(1) for the initial positioning of the eye;

In one embodiment, the trained Dlib face 68 mark point model can be used to achieve face calibration, which is marked as Point[1:68], where the calibration point numbers of the left and right eyes of the human being are [43:48] and [43:48] and [ 37:42]. Using the calibration points of the left and right eyes to realize the preliminary positioning of the human eye, the obtained human eye image I_eye(1) contains the human eye images I_eye_left(1) and I_eye_right(1) of the human left eye split and the right eye split respectively. The left eye image I_eye_left(1) and the human right eye image I_eye_right(1) are both W×H three-dimensional color images, where W and H represent the width and height of the human eye image, respectively.

Taking the right eye as an example, the leftmost point number of the right eye cleft is 37, the rightmost point number is 40, the topmost point number is 38 or 39, and the bottommost number is 41 or 42, then the human right eye image I_eye_right( 1) The abscissas eye_left_x, eye_right_x of the left and right boundary points, and the ordinates of the upper and lower boundary points are eye_top_y, eye_bottom_y, respectively:

eye_left_x=Point[37]_x-l_x

eye_right_x=Point[40]_x+l_x

eye_top_y=min(Point[38]_y, Point[39]_y)-l_y

eye_bottom_y=max(Point[41]_y, Point[42]_y)+l_y

Among them, Point[α]_x and Point[α]_y respectively represent the abscissa and ordinate corresponding to the point with serial number α (37≤α≤42), and l_x and l_y respectively represent the width and height of the human right eye image I_eye_right(1) of extension. In this embodiment, l_x=l_y=40 is selected for the parameter.

Then the width W and height H of the human right eye image I_eye_right(1) are:

W=eye_right_x-eye_left_x

H=eye_bottom_y-eye_top_y

The corresponding first frame of the human right eye image I_eye_right(1) is shown in Figure 2.

It should be noted that in the actual image processing process, the processing can be performed in units of I_eye(1), or in units of I_eye_left(1) and I_eye_right(1) respectively. In this embodiment, the subsequent steps of the image processing process will be described using I_eye(1) in order to describe the image processing process more simply and clearly.

(1.2) Perform initial positioning of the human eye image on the remaining video frames;

(1.2.1) Determine the range_eye of the human eye activity range of the current video frame;

Assuming that the current video frame is the nth frame, the image frame is denoted as I_frame(n), and the human eye image I_eye(n-1) after initial positioning of the previous frame is used to determine the human eye activity range Range_eye of the current frame. The abscissas of the left and right boundary points are Range_left_x and Range_right_x, and the ordinates of the upper and lower boundary points are Range_top_y, Range_bottom_y, respectively:

Range_left_x=eye_left_x-W

Range_right_x=eye_right_x+W

Range_top_y=eye_top_y-H

Range_bottom_y=eye_bottom_y+H

Then, the size of the range_eye of the moving range of the human eye in the nth (n≥2) frame is 3W×3H, and the position corresponding to the moving range of the right eye in the second frame image is taken as an example, as shown in FIG. 3 .

(1.2.2) Preliminary positioning of the human eye on the current video frame based on the range_eye of the human eye movement range of the current video frame;

Using the sliding window method (the window size is W×H), the range_eye of the human eye is divided into multiple windows. The method of obtaining the window is as follows:

(1.2.2.1) Set the step size in the horizontal direction as Step_len_x and the step size in the vertical direction as Step_len_y. The upper left corner vertex of the first window corresponds to the upper left corner vertex of the current frame of the human eye range, then:

Window(1)_left_x=Range_left_x

Window(1)_right_x=Range_left_x+W

Window(1)_top_y=Range_top_y

Window(1)_bottom_y=Range_top_y+H

Among them, Window(1)_left_x, Window(1)_right_x, Window(1)_top_y and Window(1)_bottom_y represent the left border, right border, top border and bottom border of the first window Window, respectively.

(1.2.2.2) The boundary point coordinates of the kth window Window corresponding to the kth row and the k_colth column are as follows:

Window(k)_left_x=Range_left_x+(k_col-1)*Step_len_x

Window(k)_right_x=Range_left_x+(k_col-1)*Step_len_x+W

Window(k)_top_y=Range_top_y+(k_row-1)*Step_len_y

Window(k)_bottom_y=Range_top_y+(k_row-1)*Step_len_y+H

Among them, 1≤k_row≤int(2H/(Setp_len_y))+1, 1≤k_col≤int(2W/(Setp_len_x))+1, k= (k_row-1)*(int(2W/(Setp_len_x))+ 1) +k_col, Window(k)_left_x, Window(k)_right_x, Window(k)_top_y and Window(k)_bottom_y respectively represent the left border, right border, upper border and lower border of the kth window Window.

In one embodiment, Step_len_x=Step_len_y=25 is selected as a parameter, and a partial window Window obtained by the sliding window method is shown in FIG. 4 .

(1.2.2.3) Calculate the similarity between each window Window and the human eye image I_eye(n-1) of the previous frame, find the window Window with the highest similarity, and use the window Window with the highest similarity as the person of the current frame Eye image I_eye(n).

In one embodiment, the template matching method—the average difference matching method is used for the similarity algorithm, and the specific calculation steps are as follows:

Traverse the kth window Window and each pixel in the human eye image I_eye(n-1) of the previous frame, and calculate the squared difference Diff(k) between the corresponding pixels. The calculation formula of the squared difference Diff is as follows:

Among them, Window(k)(i,j) and I_eye(n-1)(i,j) respectively represent the kth window Window and the (i,j)th of the previous frame of human eye image I_eye(n-1) pixel value of a pixel.

There is a square difference Diff for all windows Window, and the window Window corresponding to the Diff with the smallest distance value is the human eye image I_eye(n) of the current frame I_frame(n).

(2) Preprocess the human eye image and remove the influence of illumination.

For the human eye image I_eye after initial positioning, since the gray value distribution of the image will be uneven due to uneven illumination during the image acquisition process, it is necessary to preprocess the human eye image I_eye to achieve illumination correction processing. In one embodiment, the multi-scale Retinex (MSRCR) algorithm with color recovery is used to eliminate the influence of uneven illumination on the extracted human eye image I_eye to obtain a preprocessed image I_eye_color. The steps of the algorithm are shown in FIG. 5 .

(2.1) Calculate the logarithm values I_eye_R_log, I_eye_G_log and I_eye_B_log of the pixel values I_eye_R, I_eye_G and I_eye_B of the three channels of R, G and B for each pixel of the human eye image I_eye, and calculate the mean values Mean_R, Mean_G, Mean_B and mean square error Var_R, Var_G, Var_B, namely:

I_eye_R_log(i, j)=log(I_eye_R(i, j))

I_eye_G_iog(i,j)=log(I_eye_G(i,j))

I_eye_B_log(i, j)=log(I_eye_B(i, j))

(2.2) Find the maximum values Max_R, Max_G and Max_B of the pixel mean values of each channel, and the minimum values Min_R, Min_G and Min_B, namely:

Max_R=max(Mean_R)

Max_G=max(Mean_G)

Max_B=max(Mean_B)

Min_R=min(Mean_R)

Min_G=min(Mean_G)

Min_B=min(Mean_B)

(2.3) The pixel logarithm values I_eye_R_log, I_eye_G_log and I_eye_B_log of each channel are linearly mapped to [0, 255] for normalization, and the converted values are R_R, R_G and R_B respectively, namely:

(2.4) The image composed of the values of the three channels R_R, R_G and R_B is the human eye image I_eye_color after illumination correction.

(3) Determine the contour of the eye cleft Contour_eye and the center point of the eye cleft Center_eye for the human eye image I_eye_color after illumination correction. The steps of the algorithm are shown in Figure 6:

(3.1) Obtain the Scharr gradient image corresponding to the human eye image I_eye_color after illumination correction, and the steps are as follows:

(3.1.1) Extract the Scharr gradient of the image I_eye_color in the RGB space; since the iris and the sclera, and the pixel values at the boundaries of the upper and lower eyelids contained in the human eye image I_eye_color after illumination correction are very different, that is, the gradient value Therefore, the gradient information in the image can be well extracted by using the gradient detection operator to realize the extraction of the eye cleft contour.

First, use the Scharr operator to calculate the gradient of the image I_eye_color. The Scharr operator Gx in the horizontal direction and the Scharr operator Gy in the vertical direction are as follows:

If the gradient value of the (i,j)th pixel of the human eye image I_eye_color after illumination correction is Gx(i,j) in the horizontal direction and Gy(i,j) in the vertical direction, then the The total gradient value G(i,j) at the pixel point (i,j) is:

After calculating the gradient G(i,j) for the human eye image I_eye_color, the corresponding gradient image I_Scharr is obtained, as shown in Figure 7. From the image shown in Figure 7, it can be seen that the outline of the eye split is relatively obvious, however, there are still many complex noise textures in the initially extracted gradient image I_Scharr, which needs to be further processed and optimized.

(3.1.2) Convert the initially extracted gradient image I_Scharr to HSV space, and extract the corresponding V component to obtain the corresponding component image I_ScharrV.

In order to highlight the contour of the eye cleft to be extracted, the initially extracted gradient image I_Scharr is converted from RGB space to HSV space, and the gradient image I_ScharrHSV in HSV space is obtained. The corresponding image I_ScharrHSV and its H, S, and V components are shown in Figure 8. (a)-(d).

It can be seen from the graphs in Fig. 8 that the eye cleft contour in the gradient component image I_ScharrV corresponding to the V component of I_ScharrHSV in Fig. (d) is relatively complete and the noise in the image is small, so the gradient component image I_ScharrV is further selected for further analyze.

(3.2) Use the edge detection algorithm to perform edge detection on the human eye image I_eye_color to obtain the corresponding gradient image I_HED;

In one embodiment, the edge detection of the human eye image I_eye_color is performed by using the HED edge detection algorithm based on the convolutional neural network and the deep supervision network, which can realize multi-scale and multi-level feature learning. The corresponding gradient image I_HED is obtained as shown in Figure 9.

(3.3) Determine the position of the eye cleft contour Contour_eye based on the gradient component image I_ScharrV and the gradient image I_HED;

(3.3.1) The gradient component image I_ScharrV and the gradient image I_HED are combined to extract the corresponding binary image;

Assuming that the maximum value of the pixel values of all pixels in the gradient component image I_ScharrV is V_Vmax, in one embodiment, it is found that when the threshold is set to V_Vmax/4, the obtained binarized image I_binary1 is:

The binarized image I_binary1 is shown in Figure 10;

Assuming that the maximum value of the pixel values of all the pixels in the gradient image I_HED is V_HEDmax, in one embodiment, it is found through research that when the threshold is set to V_HEDmax/3, the obtained binarized image I_binary2 is:

The binarized image I_binary2 is shown in FIG. 11 .

Add the binarized image I_binary1 and the binarized image I_binary2, and take the intersection of the two to obtain the processed binary image I_binary, that is:

I_binary(i,j)=I_binary1(i,j)&I_binary2(i,j)

Among them, I_binary(i,j), I_binary1(i,j) and I_binary2(i,j) respectively represent the pixel value of the (i,j)th pixel in the binary images I_binary, I_binary1 and I_binary2. The corresponding binary image I_binary is shown in Figure 12.

(3.3.2) Morphological processing of the extracted binary image I_binary

In the binary image I_binary obtained by the above steps, there are still burrs connected to the contour of the eye cleft, as shown in FIG. 12 . Therefore, it is necessary to further adopt the morphological closing operation to achieve smooth image edges. The closing operation is to first perform the expansion operation on the binary image I_binary to obtain I_dilate, and then perform the erosion operation to obtain the image I_closing. The calculation formula of the closing operation is as follows:

Among them, A is a 2*2 square structure element, and I_dilate ^c is the complement of I_dilate.

The corresponding image I_closing after closing operation processing is shown in FIG. 13 .

(3.3.3) Further remove the redundant small objects in the image, and extract the largest connected domain in the image;

After the closing operation, the image I_closing is obtained as shown in Figure 13. In the figure, it can be seen that there are still some small isolated noise points. The method of removing small objects in morphology is used to remove the small noise points in the image I_closing, and the corresponding image I_morphological is obtained. , and further extract the maximum connected domain from the image I_morphological. The specific extraction steps are as follows:

3.3.3.1) First, extract all contours in the image I_morphological, denoted as C_mor_set, namely:

C_mor_set={C_mor ₁ , C_mor ₂ , ..., C_mor _k1 , ..., C_mor _n1 }

Among them, C_mor _k1 (1≤k1≤n1) represents the k1th contour, and n1 is the total number of contours in the image I_morphological;

3.3.3.2) Next, calculate the area of each contour to obtain the area set Area_set, namely:

Area_set={Area ₁ , Area ₂ , ..., Area _k1 , ..., Area _n1 }

Among them, Area _k1 represents the area of the k1th contour.

3.3.3.3) Remove the connected domain whose area is less than 300 to obtain the corresponding image I_RemoveSmallObjects, as shown in Figure 14.

3.3.3.4) Find the contour C_mormax with the largest area in the image I_RemoveSmallObjects, the contour C_mormax is the corresponding eye cleft contour Contour_eye, and the eye cleft contour is shown in Figure 15.

(3.4) Determine the center point Center_eye of the eye cleft contour Contour_eye

For the eye cleft contour Contour_eye, find the left and right boundary points of the contour, which are Point_left and Point_right respectively, namely:

Point_left_x=min(Contour_eye_x)

Point_right_x=max(Contour_eye_x)

Among them, Point_left_x and Point_right_x represent the abscissas of Point_left and Point_right, and Contour_eye_x represent the abscissas of the points that make up the contour of the eye cleft.

Using the abscissas Point_left_x and Point_right_x of the left and right boundary points Point_left and Point_right, find their corresponding ordinates Point_left_y and Point_right_y in the eye cleft contour Contour_eye.

Take the mean value of the abscissas Point_left_x and Point_right_x of the left and right boundary points Point_left and Point_right as the abscissa value Center_eye_x of the center point Center_eye of the eye cleft, and take the mean value of the ordinates Point_left_y and Point_right_y of Point_left and Point_right as the ordinate value Center_eye_y of the eye cleft center Center_eye, Obtain the pixel coordinates (Center_eye_x, Center_eye_y) of the center point of the eye split Center_eye, namely:

Center_eye_x=(Point_left_x+Point_right_x)/2

Center_eye_y=(Point_left_y+Point_right_y)/2

The contour of the eye cleft Contour_eye and the center point of the eye cleft Center_eye are shown in FIG. 16 .

(4) Determine the iris contour Contour_Iris and the iris center Center_Iris in the image, and the algorithm steps are shown in FIG. 17 .

(4.1) Image binarization

Use the Otsu threshold method to calculate the Otsu threshold Otsu_thresh for the gradient image I_ScharrV, and use the threshold Otsu_thresh to binarize the gradient image I_Scharr to obtain the corresponding binary image I_binary3, namely:

The binary image I_binary3 is shown in FIG. 18 .

(4.2) Morphological corrosion treatment

As shown in Figure 18, there is a lot of noise connected to the iris in the binarized image I_binary3. The 3*3 square structure element B is used to perform a morphological corrosion operation on the image I_binary3 to remove the burrs on the edge of the iris, and disconnect the noise points. The connection between, the corresponding image I_erosion is obtained as shown in Figure 19.

(4.3) Hole filling and finding the largest connected domain

The image I_erosion is shown in Figure 19, in which there are holes in the iris contour due to lights in the video recording, and the holes in the image I_erosion are filled with the morphological hole filling method to obtain the corresponding image I_holefilled, as shown in Figure 20.

Further extract all connected domains in the image I_holefilled, and use the connected domain with the largest area as the location of the iris. The specific steps include:

(4.3.1) First, extract all the contours in the image I_holefilled to form the contour set C_hole_set, namely:

C_hole_set={C_hole ₁ , C_hole ₂ , ..., C_hole _k2 , ..., C_hole _n2 }

Among them, C_hole _k2 (1≤k2≤n2) represents the k2th contour, and n2 is the total number of contours in the image I_holefilled.

(4.3.2) Next, calculate the area of each contour to obtain the area set Area_set1, namely:

Area_set1={Area ₁ , Area ₂ , ..., Area _k2 , ..., Area _n2 }

Among them, Area _k2 (1≤k2≤n2) represents the area of the k2th contour.

(4.3.3) Find the contour C_holemax with the largest area, which is the corresponding iris contour Contour_Iris, and the iris contour is shown in FIG. 21 .

(4.4) Determine the corresponding iris center point Center_Iris in the iris contour Contour_Iris

The iris contour Contour_Iris is used to calculate its centroid, and the centroid point is the center point of the iris (Center_Iris_x, Center_Iris_y). The centroid calculation formula is as follows:

Among them, px(k) and py(k) (1≤k≤m) represent the abscissa and ordinate of the kth point on the iris contour Contour_Iris, respectively, and Contour_Iris(i,j) represents the (i,j)th pixel The pixel value of the point.

The iris center point Contour_Iris is shown in Figure 22.

(5) Determine the eye movement range Range based on the determined iris center point Center_Iris and the eye split center point Center_eye, as shown in FIG. 23 .

After locating and determining the center point Center_eye of the eye split and the center point Center_Iris of the iris in each frame of image, the motion amplitude Mag of the iris is calculated, where the position values of the motion amplitude Mag in the x-direction and y-direction are Mag_x and Mag_y respectively:

Mag_x=Center_Iris_x-Center_eye_x

Mag_y=Center_Iris_y-Center_eye_y

Calculate the maximum and minimum values of the iris movement amplitude Mag in the x-direction and y-direction, namely the maximum and minimum values of Mag_x and Mag_y in all image frames, and take the maximum value of Mag_x as the maximum rightward movement of the eyeball in the horizontal direction. Amplitude Mag_right, take the minimum value of Mag_x as the maximum amplitude Mag_left of the leftward movement of the eyeball in the horizontal direction, take the maximum value of Mag_y as the maximum amplitude of the downward movement of the eyeball in the vertical direction Mag_bottom, take the minimum value of Mag_y as the eyeball in the vertical direction. The maximum amplitude Mag_top of the upward movement in the vertical direction, namely:

Mag_right=max(Mag_x)

Mag_left=min(Mag_x)

Mag_bottom=max(Mag_y)

Mag_top=min(Mag_y)

Then the eye movement range corresponding to the image frame is Range: [Mag_left:Mag_right,Mag_top:Mag_bottom].

Through the above processing steps, the eye movement range recognition method of this embodiment can be completed and a corresponding recognition result can be obtained, and the eyeball corresponding to the eye movement range can be drawn in the horizontal direction (x_axis) and the vertical direction in the collected video image frames. (y_axis) Motion amplitude curve.

Example 2

This embodiment is an eye movement range recognition system based on image analysis. The system is a recognition system composed of module units corresponding to the recognition methods in any of the foregoing embodiments, and is used to realize the recognition of data in captured video image frames. Eye movement range for recognition.

Through the various embodiments provided by the present invention, automatic analysis and detection can be performed based on the collected human eye activity video image frames, and the effective range features of eye movements can be extracted by using the image processing method. Compared with the prior art, the identification scheme of the present invention The detection result is objective and accurate, a high degree of automatic measurement is realized, the disadvantage of difficult data collection in the prior art is overcome, and the scheme is efficient, practical, objective and accurate.

All features disclosed in this specification, or all disclosed steps in a method or process, may be combined in any way except mutually exclusive features and/or steps.

The present invention is not limited to the foregoing specific embodiments. The present invention extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.

Claims (10)

1. An eye movement range identification method based on image analysis is characterized by comprising the following steps:

(1) carrying out primary positioning on human eye images on the acquired video frames I _ frames, namely carrying out primary positioning on all acquired images containing human eyes within the range of human eyes to obtain the human eye images I _ eye after the primary positioning;

(2) preprocessing the human eye image I _ eye, and removing illumination influence to obtain a human eye image I _ eye _ color after illumination correction;

(3) determining an eye crack Contour ContoureEye for the eye image I _ eye _ color after the illumination correction, and determining an eye crack central point Center _ eye from the eye crack Contour ContoureEye;

(4) further determining an Iris outline Contour _ Iris in the human eye image I _ eye _ color after the illumination correction, and determining an Iris Center _ Iris from the Iris outline Contour _ Iris;

(5) determining an eye movement Range based on the determined Iris Center point Center _ Iris and eye fissure Center point Center _ eye;

the step (1) specifically comprises:

(1.1) initially positioning the human eye image of the first frame video frame:

setting a first frame video frame as an I _ frame (1), carrying out primary positioning on the I _ frame (1) comprises calibrating a human face by using a deep learning training model, searching the position of eye cracks of human eyes from calibrated characteristic points, and fitting the calibrated point of the human eyes by using a rectangle so as to determine an eye image I _ eye (1) containing the primary positioning of the human eyes;

(1.2) carrying out initial positioning on the human eye image on the rest video frames:

(1.2.1) determining the human eye activity Range _ eye of the current video frame;

assuming that the current video frame is the nth frame, the image frame is marked as I _ frame (n), and the human eye movement Range _ eye of the current frame is determined by using the human eye image I _ eye (n-1) after the initial positioning of the previous frame;

the abscissa of the left and right boundary points of the Range _ eye of the active Range is Range _ left _ x and Range _ right _ x, and the ordinate of the upper and lower boundary points is Range _ top _ y and Range _ bottom _ y, which are respectively:

Range_left_x＝eye_left_x-W

Range_right_x＝eye_right_x+W

Range_top_y＝eye_top_y-H

Range_bottom_y＝eye_bottom_y+H；

wherein, W and H are the width and height of the human eye image I _ eye (n-1) respectively;

(1.2.2) carrying out primary human eye positioning on the current video frame based on the human eye activity Range _ eye of the current video frame, and sequentially finishing the primary human eye positioning on all the video frames;

dividing the human eye activity Range _ eye into a plurality of windows by using a sliding Window method, wherein the Window size is WxH, the Step length in the horizontal direction is set to be Step _ len _ x, the Step length in the vertical direction is set to be Step _ len _ y, and then the top left corner vertex of the first Window corresponds to the top left corner vertex of the current human eye activity Range _ eye;

calculating the similarity between each Window and the previous human eye image I _ eye (n-1), searching the Window with the highest similarity, and taking the Window with the highest similarity as the human eye image I _ eye (n) of the current frame.

2. The eye movement range recognition method based on image analysis as claimed in claim 1, wherein the specific method for preprocessing the human eye image in the step (2) is as follows: and for the initially positioned human eye image I _ eye, eliminating the influence caused by uneven illumination on the extracted human eye image I _ eye by using a multi-scale Retinex algorithm with color recovery to obtain a preprocessed image I _ eye _ color.

3. The eye movement range recognition method based on image analysis as claimed in claim 2, wherein the step (3) of determining the eye crack Contour _ eye and the eye crack Center point _ eye for the photo-corrected eye image I _ eye _ color specifically comprises:

(3.1) acquiring a Scharr gradient component image corresponding to the human eye image I _ eye _ color after illumination correction:

(3.1.1) extracting the Scharr gradient of the image I _ eye _ color in the RGB space;

firstly, calculating the gradient of an image I _ eye _ color by using a Scharr operator, wherein the Scharr operator comprises a Scharr operator Gx in the horizontal direction and a Scharr operator Gy in the vertical direction;

if the gradient value of the (I, j) th pixel point of the eye image I _ eye _ color after the illumination correction in the horizontal direction is Gx (I, j), and the gradient value in the vertical direction is Gy (I, j), the total gradient value G (I, j) at the pixel point (I, j) is:

(3.1.2) converting the preliminarily extracted Scharr gradient image I _ Scharr into an HSV space, and extracting a corresponding V component to obtain a corresponding component image I _ ScharrV;

(3.2) carrying out edge detection on the human eye image I _ eye _ color by using an edge detection algorithm to obtain a corresponding gradient image I _ HED;

(3.3) determining the position of the eye crack Contour Contour _ eye based on the gradient component image I _ ScharrV and the gradient image I _ HED;

(3.4) determining the Center _ eye of the eye crack Contour Contour _ eye.

4. The eye movement range recognition method based on image analysis as claimed in claim 3, wherein the step (3.3) of determining the position of the eye crack Contour Contour _ eye specifically comprises:

(3.3.1) combining the gradient component image I _ ScharrV and the gradient image I _ HED for extracting a corresponding binary image;

the maximum value of the pixel values of all the pixel points in the gradient component image I _ ScharrV is set as V _ Vmax, and the binary image I _ binary1 obtained when the threshold is set as V _ Vmax/4 is:

the maximum value of the pixel values of all the pixel points in the gradient image I _ HED is set as V _ HEDmax, and the binary image I _ binary2 obtained when the threshold is set as V _ HEDmax/3 is:

and (3) performing phase comparison on the binarized image I _ binary1 and the binarized image I _ binary2, and taking the intersection of the two to obtain a processed binary image I _ binary, namely:

I_binary(i,j)＝I_binary1(i,j)&I_binary2(i,j)；

wherein, I _ binary (I, j), I _ binary1(I, j) and I _ binary2(I, j) respectively represent the pixel values of the (I, j) th pixel points in the binary images I _ binary, I _ binary1 and I _ binary 2;

(3.3.2) performing morphological processing on the extracted binary image I _ binary;

firstly, performing expansion operation on the binary image I _ binary to obtain I _ dilate, and then performing corrosion operation to obtain an image I _ closing, wherein a calculation formula of the closing operation is as follows:

wherein A is 2 x 2 square structural element, I _ die ^c Is the complement of I _ dilate;

(3.3.3) further removing redundant small objects in the image and extracting the maximum connected domain in the image;

and removing small noise points in the image I _ closing obtained after the closed operation by using a method for removing small targets in morphology to obtain a corresponding image I _ morphology, and further extracting a maximum connected domain from the image I _ morphology as a corresponding eye fissure Contour Contour _ eye.

5. The eye movement range recognition method based on image analysis as claimed in claim 4, wherein the specific steps of extracting the maximum connected component in the image in the step (3.3.3) are as follows:

3.3.3.1) first, all contours in the image I _ morphology are extracted, denoted as C _ mor _ set, i.e.:

C_mor_set＝{C_mor ₁ ，C_mor ₂ ，…，C_mor _k1 ，…，C_mor _n1 }

wherein, C _ mor _k1 (1 ≦ k1 ≦ n1) representing the k1 th contour, n1 being the total number of contours in the image I _ morphology;

3.3.3.2) then, the Area of each contour is calculated, resulting in an Area set Area _ set, namely:

Area_set＝{Area ₁ ，Area ₂ ，…，Area _k1 ，…，Area _n1 }

wherein, Area _k1 Area representing the k1 th contour;

3.3.3.3) removing connected domains with the area less than 300 to obtain corresponding images I _ RemoveSmallObjects;

3.3.3.4) find the Contour C _ mormax with the largest area in the image I _ removesmalllobjects, which is the corresponding eye-crack Contour _ eye.

6. The eye movement range recognition method based on image analysis as claimed in claim 5, wherein the step (3.4) of determining the Center point Center _ eye of the eye crack Contour Contour _ eye comprises the following steps:

for the eye crack Contour _ eye, finding the left and right boundary points of the Contour, respectively Point _ left and Point _ right, namely:

Point_left_x＝min(Contour_eye_x)

Point_right_x＝max(Contour_eye_x)

wherein Point _ left _ x and Point _ right _ x represent the abscissa of Point _ left and Point _ right, and content _ eye _ x represents the abscissa of the points constituting the eye crack Contour;

searching the vertical coordinates Point _ left _ y and Point _ right _ y corresponding to the left boundary Point _ left and the right boundary Point _ right by using the horizontal coordinates Point _ left _ x and Point _ right _ x of the left boundary Point and the right boundary Point _ right;

taking the average value of the abscissa points Point _ left _ x and Point _ right _ x of the left and right boundary points Point _ left and Point _ right as the abscissa value Center _ eye _ x of the eye crack Center Point _ eye, taking the average value of the ordinate points Point _ left _ y and Point _ right _ y of Point _ left and Point _ right as the ordinate value Center _ eye _ y of the eye crack Center _ eye, and obtaining the pixel coordinates (Center _ eye _ x, Center _ eye _ y) of the eye crack Center Point _ eye, namely:

Center_eye_x＝(Point_left_x+Point_right_x)/2

Center_eye_y＝(Point_left_y+Point_right_y)/2。

7. the eye movement range recognition method based on image analysis as claimed in claim 2, wherein the step (4) of determining the Iris outline Contour _ Iris and the Iris Center _ Iris in the image comprises the specific steps of:

(4.1) image binarization

Calculating an Otsu threshold value Otsu _ thresh for the gradient image I _ ScharrV by using an Otsu threshold value method, and performing binarization processing on the gradient image I _ Scharr by using the threshold value Otsu _ thresh to obtain a corresponding binary image I _ binary 3:

(4.2) morphological Corrosion treatment

Performing morphological corrosion operation on the image I _ binary3 by using the 3 × 3 square structural element B, removing burrs at the edge of the iris, disconnecting the connection with a noise point, and obtaining a corresponding image I _ evolution:

(4.3) filling holes and finding the maximum connected domain

Filling holes caused by lamplight in video recording in the iris outline of the image I _ exposure by using a morphological hole filling method to obtain a corresponding image I _ holeruled; further extracting all connected domains in the image I _ holeruled, and taking the connected domain with the largest area as the position of the iris;

(4.4) determining the corresponding Iris Center point Center _ Iris in the Iris Contour _ Iris:

calculating the centroid of the Iris outline Contour _ Iris, wherein the centroid point is the central point (Center _ Iris _ x, Center _ Iris _ y) of the Iris, and the centroid calculation formula is as follows:

wherein px (k) and py (k) (k is more than or equal to 1 and less than or equal to m) respectively represent the abscissa and the ordinate of the kth point on the Iris outline Contour _ Iris, and Contour _ Iris (i, j) represents the pixel value of the (i, j) th pixel point.

8. The eye movement range recognition method based on image analysis as claimed in claim 7, wherein the specific step of finding the maximum connected component in step (4.3) comprises:

(4.3.1) first, all contours in the image I _ holeruled are extracted, forming a contour set C _ hole _ set, namely:

C_hole_set＝[C_hole ₁ ，C_hole ₂ ，…，C_hole _k2 ，…，C_hole _n2 }

wherein, C _ hole _k2 (1 ≦ k2 ≦ n2) representing the k2 th contour, n2 being the total number of contours in the image I _ holeruled;

(4.3.2) Next, the Area of each contour is calculated, resulting in the Area set Area _ set1, namely:

Area_set1＝{Area ₁ ，Area ₂ ，…，Area _k2 ，…，Area _n2 }

wherein, Area _k2 (1. ltoreq. k 2. ltoreq. n2) represents the area of the k2 th contour;

(4.3.3) searching the Contour C _ holomax with the largest area, wherein the Contour is the corresponding Iris Contour Contour _ Iris.

9. The eye movement Range recognition method based on image analysis as claimed in claim 2, wherein the step (5) of determining the eye movement Range based on the determined Iris Center point Center _ Iris and eye fissure Center point Center _ eye comprises the specific steps of:

after the eye fissure Center point _ eye and the Iris Center point _ Iris in each frame of image are located and determined, calculating the motion amplitude Mag of the Iris, wherein the position values of the motion amplitude Mag in the x direction and the y direction are Mag _ x and Mag _ y respectively:

Mag_x＝Center_Iris_x-Center_eye_x

Mag_y＝Center_Iris_y-Center_eye_y；

calculating the maximum value and the minimum value of the iris motion amplitude Mag in the x direction and the y direction, namely the maximum value and the minimum value of Mag _ x and Mag _ y, respectively, taking the maximum value of Mag _ x as the maximum amplitude Mag _ right of the eyeball moving to the right in the horizontal direction, taking the minimum value of Mag _ x as the maximum amplitude Mag _ left of the eyeball moving to the left in the horizontal direction, taking the maximum value of Mag _ y as the maximum amplitude Mag _ bottom of the eyeball moving downwards in the vertical direction, and taking the minimum value of Mag _ y as the maximum amplitude Mag _ top of the eyeball moving upwards in the vertical direction, namely:

Mag_right＝max(Mag_x)

Mag_left＝min(Mag_x)

Mag_bottom＝max(Mag_y)

Mag_top＝min(Mag_y)

the eye movement Range corresponding to the image frame is: [ Mag _ left: Mag _ right, Mag _ top: Mag _ bottom ].

10. An eye movement range recognition system based on image analysis, which is characterized in that the system is a recognition system composed of module units corresponding to the steps of the eye movement range recognition method according to any one of claims 1-9, and is used for recognizing the eye movement range in the collected video image frame.

Images