Disclosure of Invention
The invention aims to provide a high-speed accurate searching and positioning method for a binocular pupil and a corneal reflection point, which can quickly find the accurate positions of the boundary and the center of the binocular pupil and the accurate positions of the corneal reflection point of the binocular on each frame of image at a frame rate of 1018-1020fps, and the processing speed is much higher than the eye movement speed.
The invention solves the technical problem, and adopts the technical scheme that: the high-speed accurate searching and positioning method for the reflective points of the pupils and the cornea of the eyes comprises the following steps:
collecting gray level images of two eyes;
selecting a first threshold value set based on finding the pupil boundary of the two eyes according to the gray value of the gray image, and converting the current gray image into a binary image according to the first threshold value;
calculating the boundary of the pupils of the two eyes according to the binary image, and calculating the center point coordinates of the pupils of the two eyes according to the boundary of the pupils of the two eyes;
and meanwhile, selecting a second threshold set based on searching for the corneal reflection points of the eyes according to the gray value of the gray image, searching for a position exceeding the second threshold on the gray image according to the second threshold, and calculating the central coordinates of the corneal reflection points of the eyes according to the position.
As a preferable scheme, before acquiring the grayscale images of both eyes, a light source corresponding to an image acquisition device for acquiring the grayscale images of both eyes needs to be installed, and the light source faces to the direction and the position of the tested both eyes.
Preferably, the image acquisition device is an industrial camera, and the industrial camera can support and output images with the highest frame rate of 1018 and 1020fps through a data acquisition card.
As a preferable scheme, the selecting of the threshold set based on finding the pupil boundary of the two eyes according to the gray value of the gray image is:
extracting and selecting a threshold value adapted to the current gray level image based on the gray level histogram characteristics of the image; or,
the self-defined threshold value is manually set or selected on the parameter setting interface.
As a preferred scheme, the calculating the boundary of the pupils of both eyes according to the binary image, and calculating the coordinates of the central points of the pupils of both eyes according to the boundary of the pupils of both eyes specifically includes the following steps:
positioning the boundaries of the pupils of the two eyes based on a ray algorithm of a horizontal ray and a vertical ray from inside to outside;
fitting each positioned pupil boundary into a circle or an ellipse by adopting a least square fitting circle algorithm;
and finding the position of the circle center by adopting a least square fitting circle algorithm, namely the calculated coordinates of the central points of the pupils of the two eyes.
As a preferable scheme, when the coordinates of the central points of the pupils of both eyes are calculated, the diameters and the areas of the pupils of both eyes are calculated at the same time.
As a preferred scheme, after the coordinates of the central points of the pupils of both eyes are calculated, the central positions of the pupils of both eyes are drawn by using vertical lines with the length being the radius of the pupils of both eyes in a video image window, and the central point of each pupil is in the middle position of the corresponding vertical line;
after the central positions of the pupils of the two eyes are drawn, connecting the central points of the pupils of the two eyes by using a straight line in a video image window;
and a vertical line is drawn at the center of the straight line connecting the pupil center points of the two eyes, and the length of the vertical line is the longitudinal height difference of the pupil centers of the two eyes, so that whether the two eyes are positioned at the horizontal position or not can be observed or judged more easily.
As a preferable scheme, when the threshold value two set based on finding the cornea reflection point of both eyes is selected according to the gray value of the gray image:
extracting and selecting a threshold value adapted to the current gray level image based on the gray level histogram characteristics of the image; or,
the self-defined threshold value is manually set or selected on the parameter setting interface.
As a preferable scheme, the searching for a position exceeding the second threshold on the grayscale image according to the second threshold, and calculating the central coordinate of the cornea reflection point of both eyes according to the position specifically includes:
and calculating the mass center or the gravity center of the cornea reflection points of the eyes according to the positions, wherein the mass center or the gravity center is the central coordinate of the cornea reflection points of the eyes.
As a preferred scheme, after the central coordinates of the corneal reflection points of both eyes are calculated, the central positions of the corneal reflection points of both eyes are drawn by a cross in a video image window.
The method has the advantages that the method can quickly capture the movement change of the eyes by the high-speed accurate searching and positioning method of the reflective points of the pupils and the cornea, is completely faster than the movement speed of the eyes, can lay a good foundation for the follow-up study of the biological characteristics of the eye movements, and simultaneously makes early preparation work for the study of the physiological and cognitive function levels of the brain.
Detailed Description
The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.
Examples
The embodiment provides a high-speed and accurate searching and positioning method for a binocular pupil and a corneal reflection point, a flow chart of which is shown in fig. 1, wherein the method comprises the following steps:
and step S1, installing a light source corresponding to the image acquisition equipment for acquiring the gray scale images of the two eyes, wherein the light source faces to the direction and the position of the tested two eyes.
Here, in order to capture a clear image by the camera in a relatively closed equipment environment, a suitable light source is used, and the light source also forms two cornea reflection spots on the surfaces of the eyes, namely two small bright spots seen on the camera image.
And step S2, installing an industrial-grade professional camera and a data acquisition card, capturing image data of the eyes by the camera, wherein the image data is a gray image in the embodiment, and transmitting the data to a corresponding processing module for calculating the coordinates of the center point of the pupil of the eyes and the center coordinates of the reflective point of the cornea of the eyes by the data acquisition card.
Here, the industrial camera supports outputting an image with the highest frame rate of 1018 + 1020fps through the data acquisition card, and the embodiment can efficiently and accurately calculate the coordinates of the pupil center point of the two eyes and the coordinates of the cornea reflection point of the two eyes in real time.
And step S3, calculating the boundary of the pupils of the two eyes and the coordinates of the central points of the pupils of the two eyes, and simultaneously calculating the central coordinates of the cornea reflection points of the two eyes.
For calculating the boundary of the pupils of the two eyes and the coordinates of the central points of the pupils of the two eyes, the method specifically comprises the following steps:
selecting a first threshold value set based on finding the pupil boundary of the two eyes according to the gray value of the gray image, and converting the current gray image into a binary image according to the first threshold value; and calculating the boundary of the pupils of the two eyes according to the binary image, and calculating the center point coordinates of the pupils of the two eyes according to the boundary of the pupils of the two eyes.
Selecting a threshold set based on finding the pupil boundary of the two eyes according to the gray value of the gray image: a threshold value for current gray level image adaptation can be extracted and selected based on the gray level histogram features of the image; alternatively, a self-defined threshold value is manually set or selected on the parameter setting interface.
Calculating the boundary of the pupil of the two eyes according to the binary image, and calculating the coordinate of the central point of the pupil of the two eyes according to the boundary of the pupil of the two eyes, which comprises the following steps:
positioning the boundaries of the pupils of the two eyes based on a ray algorithm of a horizontal ray and a vertical ray from inside to outside;
fitting each positioned pupil boundary into a circle or an ellipse by adopting a least square fitting circle algorithm;
and finding the position of the circle center by adopting a least square fitting circle algorithm, namely the calculated coordinates of the central points of the pupils of the two eyes.
It should be noted that, when calculating the coordinates of the central points of the pupils of both eyes, the diameters and the areas of the pupils of both eyes can be calculated at the same time.
In addition, after the coordinates of the central points of the pupils of the two eyes are calculated, the central positions of the pupils of the two eyes can be drawn by using vertical lines with the length being the radius of the pupils of the two eyes in a video image window, and the central point of each pupil is positioned in the middle of the corresponding vertical line; after the central positions of the pupils of the two eyes are drawn, the central points of the pupils of the two eyes are connected by a straight line in a video image window, a vertical line is drawn at the center of the straight line connected with the central points of the pupils of the two eyes, the length of the vertical line is the longitudinal height difference of the centers of the pupils of the two eyes, so that a pilot can observe whether the two eyes keep horizontal positions when using the system conveniently, if the connecting line between the pupils of the two eyes in the horizontal position is not horizontal, some distortion can occur, and meanwhile, a straight line segment is seen in the middle of the connecting line between the centers of the pupils of the two eyes (the ideal condition is that the straight line segment cannot be seen.
For calculating the central coordinates of the cornea reflection points of the eyes, the method specifically comprises the following steps:
and selecting a second threshold set based on searching the cornea reflection points of the eyes according to the gray value of the gray image, searching a position exceeding the second threshold on the gray image according to the second threshold, and calculating the central coordinates of the cornea reflection points of the eyes according to the position.
Here, when selecting the threshold value two set based on finding the cornea reflection point of both eyes according to the gray value of the gray image: a threshold value for current gray level image adaptation can be extracted and selected based on the gray level histogram features of the image; alternatively, a self-defined threshold value is manually set or selected on the parameter setting interface.
And searching a position exceeding the second threshold value on the gray level image according to the second threshold value, and calculating the central coordinate of the cornea reflection point of the eyes according to the position, wherein the central coordinate specifically refers to: and calculating the mass center or the gravity center of the cornea reflection points of the eyes according to the position, wherein the mass center or the gravity center is the central coordinate of the cornea reflection points of the eyes.
In this embodiment, after the central coordinates of the corneal reflection points of both eyes are calculated, the central positions of the corneal reflection points of both eyes can be drawn by a cross in a video image window.
In summary, in this embodiment, after receiving the image data of the tested two eyes, the algorithm library for searching and positioning the two-eye pupils and the reflective points of the two-eye cornea may perform analysis according to the image gray-scale values of the image data to find out the positions of the two-eye pupils, gradually find out the boundary points of the two-eye pupils, find out the coordinates of the positions of the centers of the two-eye pupils according to the boundary points of the two-eye pupils, calculate the number of pixels in the range of the two-eye pupils as the areas of the two-eye pupils according to the gray-scale values of the two-eye pupils, and calculate the radii and diameters of the two-eye pupils according to the found centers of the two-eye pupils and the.
And after the tested binocular image data is received by searching and positioning the binocular pupil and the binocular cornea reflection point, analyzing according to the image gray value of the image data to find out the position coordinates of the binocular cornea reflection point. And drawing the centers of the pupils of the two eyes by using a vertical line (the length is the radius of the pupils of the two eyes) in a video image window and drawing the central positions of the cornea reflecting points of the two eyes by using a cross.
Here, if the simultaneous searching and positioning of the pupils of both eyes are selected, after the pupils of both eyes are found, a straight line is required to connect the center positions of the pupils of both eyes in the video image window, a vertical line is drawn at the center of the straight line connecting the center points of the pupils of both eyes, the length of the vertical line is the longitudinal height difference of the centers of the pupils of both eyes, so that a pilot can observe whether both eyes keep horizontal positions when using the system conveniently, if the straight line is not in the horizontal position, some distortion occurs on the connecting line between the pupils of both eyes, and meanwhile, a straight line segment is seen in the middle of the connecting line between the centers of the pupils of both eyes (ideally, the straight line segment cannot be seen, namely, the pupil.
And finally, searching an algorithm library for positioning the binocular pupil and the binocular cornea reflection point, and finally transmitting the coordinate position (horizontal direction X and vertical direction Y) of the center point of the binocular pupil, the diameter of the binocular pupil, the area of the binocular pupil and the coordinate position (horizontal direction X and vertical direction Y) of the binocular cornea reflection point to an upper-layer application, namely, a system for analyzing eye movement data.
Therefore, the invention solves the problem that the time consumption of some pupil positioning algorithms is too long at present, and simultaneously solves the problems that the demands of people are that the pupils and the cornea reflecting points of the eyes are found at the same time, the real-time positions of the pupils of the eyes and the real-time positions of the cornea reflecting points of the eyes are marked in the real-time video image window of the camera, and parameters such as the central position coordinates (X coordinates in the horizontal direction and Y coordinates in the vertical direction), the pupil area and the diameter of the pupils of the eyes, the central coordinates (X coordinates in the horizontal direction and Y coordinates in the vertical direction) of the cornea reflecting points of the eyes, the mark for determining whether the pupils of the eyes are positioned, the mark for determining whether the cornea reflecting points of the eyes are found and the like are.
That is to say, this embodiment can support the highest frame rate (1018 and 1020fps) output by the industrial camera, quickly find the accurate pupil center positions and the accurate positions of the reflective points of the cornea of both eyes on each frame of image, and mark out the boundary points of the pupil of both eyes and the positions of the reflective points of the cornea of both eyes, if both pupils of both eyes are found, it is further necessary to connect the center points of the pupil of both eyes with a straight line, and draw a vertical line at the center of the straight line connecting the center points of the pupil of both eyes, the length of the vertical line being the longitudinal height difference of the center of the pupil of both eyes, so as to observe whether both eyes are in the horizontal position. At present, the accurate positions of the eyes and the accurate positions of the cornea reflecting points of the eyes of each frame image can be found without losing one frame under the condition of no blinking, and the frame rate is always kept between 1018 and 1020 fps.