CN113616196A - Blink data statistical method, device, medium and electronic equipment - Google Patents
Blink data statistical method, device, medium and electronic equipment Download PDFInfo
- Publication number
- CN113616196A CN113616196A CN202110915881.XA CN202110915881A CN113616196A CN 113616196 A CN113616196 A CN 113616196A CN 202110915881 A CN202110915881 A CN 202110915881A CN 113616196 A CN113616196 A CN 113616196A
- Authority
- CN
- China
- Prior art keywords
- target area
- eyelid
- opening
- eye
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007619 statistical method Methods 0.000 title claims abstract description 23
- 210000000744 eyelid Anatomy 0.000 claims abstract description 144
- 230000004424 eye movement Effects 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 210000001508 eye Anatomy 0.000 description 51
- 230000004397 blinking Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 8
- 210000004175 meibomian gland Anatomy 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 150000002632 lipids Chemical class 0.000 description 3
- 210000003205 muscle Anatomy 0.000 description 3
- 210000005252 bulbus oculi Anatomy 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000035807 sensation Effects 0.000 description 2
- 206010006784 Burning sensation Diseases 0.000 description 1
- 208000003556 Dry Eye Syndromes Diseases 0.000 description 1
- 206010013774 Dry eye Diseases 0.000 description 1
- 208000023715 Ocular surface disease Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 208000020538 atrophic muscular disease Diseases 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 208000027993 eye symptom Diseases 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 210000004907 gland Anatomy 0.000 description 1
- 230000007794 irritation Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 230000028327 secretion Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000004488 tear evaporation Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1103—Detecting eye twinkling
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
Abstract
The application discloses a statistical method and device of blink data, a computer readable storage medium and electronic equipment, wherein video data containing eye movement in a preset time period are obtained, then each frame of image of the video data is identified, a target area between an upper eyelid and a lower eyelid is obtained, the eyelid fissure height corresponding to the target area is calculated, and the opening and closing state of the target area is determined according to the eyelid fissure height; finally, according to the opening and closing states of the target areas of all the frame images in the video data, calculating to obtain blink data in a preset time period; the blink data are counted, the difficulty of data counting is reduced, the eyelid fissure height is calculated by automatically identifying the target area, the blink data can be accurately obtained, and an accurate data base is provided for subsequent prejudgment.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a blink data statistical method and device, a computer readable storage medium and electronic equipment.
Background
Blinking, also known as the temporal reflex, is a rapid periodic movement of the eyelids that results from the forces generated during contraction and relaxation of the levator palpebral and orbicularis oculi muscles. The blinking process can be divided into complete blinking and incomplete blinking according to whether the upper and lower eyelids are in full contact over the entire length of the eyelids during blinking.
Muscle action force generated during blinking promotes secretion of meibomian gland meibum, the secreted meibum is extruded to the eyelid margin, the upper eyelid and the lower eyelid are contacted with each other, and the meibum can be uniformly diffused to the tear film along the eyelid margin, so that a lipid layer of the tear film is formed, the eye surface is lubricated, and the tear evaporation is reduced. Abnormal blinking may cause a series of eye symptoms and signs to change accordingly. When incomplete blinking occurs, the eye muscle squeeze force diminishes, the driving force for the meibomian glands to secrete meibum into the lid rim lipid reservoir diminishes, and the meibum stagnates and coagulates in the gland ducts. In addition, as contact between the upper and lower eyelids decreases, lipid depots at the eyelid margin decrease, further uneven tear film distribution occurs. Long term obstruction can cause disuse atrophy of the meibomian glands, resulting in loss of the meibomian glands. Therefore, when incomplete blinking increases, higher Ocular Surface Disease Index (OSDI) scores, more meibomian gland loss and poorer tear film stability occur, as well as a higher proportion of dry eye diagnostic criteria for those who do not complete blinking.
When the eyes of the patient are abnormal, eye burning sensation, irritation sensation or foreign body sensation of different degrees may occur, thereby causing a change in blinking frequency, abnormal blinking amplitude, abnormal complete closure time, and the like. Therefore, the blink data of the patient can be counted to judge whether the performance of the eyes of the patient is normal, however, the conventional counting is mostly realized through manual counting, the counting efficiency is not high, and the counting precision is not high.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. Embodiments of the present application provide a method and an apparatus for blink data statistics, a computer-readable storage medium, and an electronic device, which solve the above problem of low blink data statistics efficiency and accuracy.
According to an aspect of the present application, there is provided a statistical method of blink data, comprising: acquiring video data containing eye movement in a preset time period; identifying each frame of image of the video data to obtain a target area between an upper eyelid and a lower eyelid; calculating the palpebral fissure height corresponding to the target area; determining the opening and closing state of the target area according to the eyelid fissure height; wherein the open-close state comprises an eye opening state, an eye closing state and a half eye opening state; and counting to obtain blink data in the preset time period according to the opening and closing states of the target areas of all the frame images in the video data.
In one embodiment, the identifying each frame of image of the video data to obtain the target area between the upper eyelid and the lower eyelid comprises: inputting each frame of image of the video data into a recognition model to obtain the target area between the upper eyelid and the lower eyelid; wherein the recognition model comprises a neural network model.
In one embodiment, the identifying each frame of image of the video data to obtain the target area between the upper eyelid and the lower eyelid comprises: selecting a seed region enclosed in each frame of image of the video data according to the pixel value of the region between the upper eyelid and the lower eyelid; when the seed region exists, the boundary line of the target region is determined by taking the seed region as a center and expanding the seed region to the periphery; and carrying out binarization processing on each frame of image with the boundary line of the target area determined to obtain a binarized image containing the target area.
In one embodiment, the identifying each frame of image of the video data to obtain the target area between the upper eyelid and the lower eyelid comprises: and when the seed area does not exist, determining that the target area is in a closed-eye state.
In one embodiment, the calculating the palpebral fissure height corresponding to the target region comprises: obtaining a first reference line according to end points of two ends of the upper eyelid; obtaining a second reference line according to end points of two ends of the lower eyelid; determining the height direction of the eyelid fissure according to the first reference line and the second reference line; and selecting the maximum distance between the upper eyelid and the lower eyelid in the direction of the palpebral fissure height as the palpebral fissure height.
In one embodiment, the determining the opening and closing state of the target area according to the palpebral fissure height comprises: selecting the maximum value and the minimum value of all the palpebral fissure heights of the video data; calculating a maximum difference between the palpebral fissure height and the maximum value and a minimum difference between the palpebral fissure height and the minimum value; when the maximum difference is smaller than or equal to a first preset difference, determining that the opening and closing state of the target area is an eye opening state; when the minimum difference is smaller than or equal to a second preset difference, determining that the opening and closing state of the target area is an eye closing state; and when the maximum difference is larger than the first preset difference and the minimum difference is larger than the second preset difference, determining that the opening and closing state of the target area is a half-opening state.
In an embodiment, the obtaining, by statistics, blink data within the preset time period according to the open/close states of the target regions of all frame images in the video data includes: acquiring the complete blink times and the incomplete blink times in the video data according to the opening and closing states of the target areas of all the frame images in the video data; wherein, the single complete blink indicates that the opening and closing state of the target area is changed from the eye opening state to the eye closing state and then changed to the eye opening state again, and the single incomplete blink indicates that the opening and closing state of the target area is changed from the eye opening state to the eye half opening state and then changed to the eye opening state again.
According to another aspect of the present application, there is provided a statistical apparatus of blink data, comprising: the video acquisition module is used for acquiring video data containing eye movement in a preset time period; the image identification module is used for identifying each frame of image of the video data to obtain a target area between an upper eyelid and a lower eyelid; the height calculation module is used for calculating the palpebral fissure height corresponding to the target area; the state determining module is used for determining the opening and closing state of the target area according to the eyelid fissure height; wherein the open-close state comprises an eye opening state, an eye closing state and a half eye opening state; and the data statistics module is used for carrying out statistics to obtain blink data in the preset time period according to the opening and closing states of the target areas of all the frame images in the video data.
According to another aspect of the present application, there is provided a computer-readable storage medium storing a computer program for performing the statistical method of blink data as described in any one of the above.
According to another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory for storing the processor-executable instructions; the processor is configured to perform any of the statistical methods described above for blink data.
According to the statistical method and device for the blink data, the computer readable storage medium and the electronic device, the video data containing the eye movement in the preset time period are obtained, then each frame of image of the video data is identified, the target area between the upper eyelid and the lower eyelid is obtained, the eyelid fissure height corresponding to the target area is calculated, and the opening and closing state of the target area is determined according to the eyelid fissure height; finally, according to the opening and closing states of the target areas of all the frame images in the video data, calculating to obtain blink data in a preset time period; the blink data are counted by automatically acquiring video data within a period of time and identifying each frame of image in the video data to obtain a target area between an upper eyelid and a lower eyelid and determining the opening and closing state according to the height of the eyelid fissure, so that the difficulty of data statistics is reduced, the height of the eyelid fissure is calculated by automatically identifying the target area, the blink data can be accurately obtained, and an accurate data basis is provided for subsequent prejudgment.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a flowchart illustrating a statistical method for blink data according to an exemplary embodiment of the present application.
Fig. 2 is a schematic flowchart of a target area identification method according to an exemplary embodiment of the present application.
Fig. 3 is a schematic flow chart of a method for calculating eyelid fissure height according to an exemplary embodiment of the present application.
Fig. 4 is a flowchart illustrating a method for determining an opening/closing state of a target area according to an exemplary embodiment of the present application.
Fig. 5 is a schematic structural diagram of a statistical apparatus for blink data according to an exemplary embodiment of the present application.
Fig. 6 is a schematic structural diagram of a blink data statistics apparatus according to another exemplary embodiment of the present application.
Fig. 7 is a block diagram of an electronic device provided in an exemplary embodiment of the present application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.
Fig. 1 is a flowchart illustrating a statistical method for blink data according to an exemplary embodiment of the present application. As shown in fig. 1, the statistical method of blink data includes:
step 100: video data containing eye movement within a preset time period is acquired.
By acquiring video data containing eye movement in a preset time period, the blinking times in the time period are obtained through analysis, so that the blinking frequency of a corresponding patient can be counted, and data support is provided for subsequent diagnosis of doctors. The specific acquisition mode may be to acquire video data of the human eye through a camera (e.g., a camera, etc.), where the video data includes multiple frames of image data.
Step 200: each frame of image of the video data is identified, resulting in a target area between the upper eyelid and the lower eyelid.
By recognizing each frame of image in the video data, the region between the upper eyelid and the lower eyelid is found as a target region, i.e., an eyeball region or an eye-opening region.
In an embodiment, the specific implementation manner of step 200 may be: inputting each frame of image of the video data into an identification model to obtain a target area between an upper eyelid and a lower eyelid; wherein the recognition model comprises a neural network model. The recognition model may be trained from standard image data.
Step 300: and calculating the eyelid fissure height corresponding to the target area.
The eyelid fissure height of the target area, i.e., the height at which the eye is open, is calculated to determine whether the eye in the current image is open and the proportion of the eye that is open.
Step 400: determining the opening and closing state of the target area according to the height of the eyelid fissure; the open-close state includes an eye-open state, an eye-close state and a half-open state.
After the eyelid fissure height is calculated, the opening and closing state of the eyes in the current image is determined according to the eyelid fissure height. Namely, the eyes in the current image are judged to be in an eye-open state, an eye-closed state or a half-open state.
Step 500: and counting to obtain blink data in a preset time period according to the opening and closing states of the target areas of all the frame images in the video data.
After the opening and closing state (namely the state of eyes) of the target area in each frame of image is obtained, data such as the blink frequency of the patient in the preset time are obtained through statistics according to the change of the opening and closing state of all continuous target areas.
In an embodiment, the specific implementation manner of step 500 may be: acquiring the complete blink times and the incomplete blink times in the video data according to the opening and closing state of the target area of all the frame images in the video data; the single complete blink indicates that the open-close state of the target region changes from the eye-open state to the eye-closed state and then changes to the eye-open state again, and the single incomplete blink indicates that the open-close state of the target region changes from the eye-open state to the half-open state and then changes to the eye-open state again. That is, the one-time blinking process of the eyes is determined according to the periodic variation of the opening and closing state of the target region in the continuous images, so as to count the number of blinks (including full blinks and incomplete blinks), the frequency of blinks (including full blinks and incomplete blinks), and the like.
According to the statistical method of the blink data, the video data containing the eye movement in the preset time period are obtained, then each frame of image of the video data is identified, the target area between the upper eyelid and the lower eyelid is obtained, the eyelid fissure height corresponding to the target area is calculated, and the opening and closing state of the target area is determined according to the eyelid fissure height; finally, according to the opening and closing states of the target areas of all the frame images in the video data, calculating to obtain blink data in a preset time period; the blink data are counted by automatically acquiring video data within a period of time and identifying each frame of image in the video data to obtain a target area between an upper eyelid and a lower eyelid and determining the opening and closing state according to the height of the eyelid fissure, so that the difficulty of data statistics is reduced, the height of the eyelid fissure is calculated by automatically identifying the target area, the blink data can be accurately obtained, and an accurate data basis is provided for subsequent prejudgment.
Fig. 2 is a schematic flowchart of a target area identification method according to an exemplary embodiment of the present application. As shown in fig. 2, step 200 may include:
step 210: and selecting a seed region enclosed in each frame of image of the video data according to the pixel value of the region between the upper eyelid and the lower eyelid.
Because the pixel values of different positions in the image are different, a closed area is selected as a seed area according to the pixel value corresponding to the eyeball area, namely a small area (which is a part of the target area) is selected as a basis for determining the target area.
Step 220: when the seed region exists, the boundary line of the target region is determined by expanding around the seed region.
When a seed region exists, i.e., a target region exists between the upper eyelid and the lower eyelid (in which case the eyes may be in an open-eye or half-open-eye state), the seed region is centered and expanded to the periphery. Specifically, the difference between the pixel values of the surrounding pixel points and the pixel values of the boundary pixel points of the seed region is detected by taking the seed region as the center, when the difference is smaller than a preset value, the surrounding pixel points are also indicated as a target region, and until the difference is larger than the preset value, the boundary line of the target region is found, so that the range of the target region is determined.
Step 230: and carrying out binarization processing on each frame of image with the boundary line of the target area determined to obtain a binarized image containing the target area.
After the boundary line of the target region is determined, the image is binarized, that is, the pixel value of the target region is adjusted to a fixed gray value (for example, 255) and other non-target regions are uniformly adjusted to the background (the gray value is adjusted to 0), so that the contrast between the target region and other regions can be more clearly realized.
In one embodiment, as shown in fig. 2, step 200 may further include:
step 240: and when the seed area does not exist, determining that the target area is in a closed-eye state.
If the seed region does not exist, that is, the target region exists between the upper eyelid and the lower eyelid, it indicates that the upper eyelid and the lower eyelid are attached to each other, that is, the eye is in a closed state.
Fig. 3 is a schematic flow chart of a method for calculating eyelid fissure height according to an exemplary embodiment of the present application. As shown in fig. 3, step 300 may include:
step 310: and obtaining a first reference line according to the end points of the two ends of the upper eyelid.
Step 320: and obtaining a second reference line according to the end points of the two ends of the lower eyelid.
After the boundary line of the target area is obtained, the upper boundary and the lower boundary of the boundary line are images of the upper eyelid and the lower eyelid, and the first reference line and the second reference line can be obtained according to end points of the two ends of the upper eyelid and the lower eyelid. Specifically, the first reference line and the second reference line may be a connection line of end points of both ends of the upper eyelid and a connection line of end points of both ends of the lower eyelid, respectively.
Step 330: and determining the eyelid fissure height direction according to the first reference line and the second reference line.
Since each individual's eyelid is different and even some individual's eyes are tilted, there are instances when the calculation of the palpebral fissure height in both the horizontal and vertical directions can be inaccurate. Therefore, the present application determines a first reference line and a second reference line through the upper eyelid and the lower eyelid, and then determines the direction of the palpebral fissure height from the first reference line and the second reference line. Specifically, a reference direction may be obtained according to the first reference line and the second reference line, and the reference direction may be obtained by fitting the first reference line and the second reference line, that is, the sum of distances from all points on the first reference line and the second reference line to a straight line in which the reference direction is located is the minimum. After the reference direction is obtained, the direction of the height of the eyelid fissure can be determined to be the direction perpendicular to the reference direction.
Step 340: the maximum distance between the upper eyelid and the lower eyelid in the direction of the palpebral fissure height is selected as the palpebral fissure height.
After the eyelid fissure direction is determined, a plurality of distance values between the upper eyelid and the lower eyelid along the eyelid fissure direction can be calculated, and the maximum value can be selected as the eyelid fissure height to better reflect the current opening and closing state of the eye.
Fig. 4 is a flowchart illustrating a method for determining an opening/closing state of a target area according to an exemplary embodiment of the present application. As shown in fig. 4, step 400 may include:
step 410: the maximum and minimum values of all palpebral fissure heights of the video data are selected.
Since each person has different eye opening and eye closing states, for example, the height of the eyelid cleft between the upper eyelid and the lower eyelid of a person is not zero when the person closes the eye, after the height of the eyelid cleft in each frame of image is calculated, the maximum value and the minimum value of all the heights of the eyelid cleft are selected and are respectively used as the maximum eye opening state and the eye closing state of the patient.
Step 420: the maximum difference between the palpebral fissure height and the maximum value and the minimum difference between the palpebral fissure height and the minimum value are calculated.
And respectively calculating the maximum difference between the eyelid fissure height and the maximum value and the minimum difference between the eyelid fissure height and the minimum value in all the images so as to judge the opening and closing state corresponding to the eyelid fissure height.
Step 430: and when the maximum difference is smaller than or equal to a first preset difference, determining the opening and closing state of the target area as an eye opening state.
When the maximum difference is smaller than or equal to the first preset difference, that is, the difference between the eyelid fissure height of the target region in the current image and the maximum eye opening state is small, it may be determined that the eye opening/closing state corresponding to the current image is the eye opening state.
Step 440: and when the minimum difference is smaller than or equal to a second preset difference, determining the opening and closing state of the target area as an eye closing state.
When the minimum difference is less than or equal to the second preset difference, that is, the difference between the palpebral fissure height of the target region in the current image and the minimum eye opening state is small, it may be determined that the eye opening/closing state corresponding to the current image is the eye closing state. The second predetermined difference may be equal to the first predetermined difference, or may not be equal to the first predetermined difference.
Step 450: and when the maximum difference is larger than the first preset difference and the minimum difference is larger than the second preset difference, determining that the opening and closing state of the target area is a half-opening state.
When the maximum difference is greater than the first preset difference and the minimum difference is greater than the second preset difference, that is, the difference between the eyelid fissure height of the target region in the current image and the maximum eye opening state is relatively large, and the difference between the eyelid fissure height of the target region in the current image and the minimum eye opening state is relatively large, at this time, it may be determined that the eye opening/closing state corresponding to the current image is the half eye opening state.
Fig. 5 is a schematic structural diagram of a statistical apparatus for blink data according to an exemplary embodiment of the present application. As shown in fig. 5, the statistic device 50 for blink data includes: a video acquiring module 51, configured to acquire video data including eye movement within a preset time period; the image identification module 52 is configured to identify each frame of image of the video data to obtain a target area between the upper eyelid and the lower eyelid; the height calculating module 53 is configured to calculate a palpebral fissure height corresponding to the target area; the state determining module 54 is used for determining the opening and closing state of the target area according to the height of the eyelid fissure; the opening and closing states comprise an eye opening state, an eye closing state and a half eye opening state; and a data statistics module 55, configured to count blink data within a preset time period according to the open/close states of the target regions of all the frame images in the video data.
According to the blink data statistical device, video data containing eye movement in a preset time period are obtained through the video obtaining module 51, then the image recognition module 52 recognizes each frame of image of the video data to obtain a target area between an upper eyelid and a lower eyelid, the height calculation module 53 calculates the palpebral fissure height corresponding to the target area, and the state determination module 54 determines the opening and closing state of the target area according to the palpebral fissure height; finally, the data statistics module 55 performs statistics to obtain blink data in a preset time period according to the opening and closing states of the target areas of all the frame images in the video data; the blink data are counted by automatically acquiring video data within a period of time and identifying each frame of image in the video data to obtain a target area between an upper eyelid and a lower eyelid and determining the opening and closing state according to the height of the eyelid fissure, so that the difficulty of data statistics is reduced, the height of the eyelid fissure is calculated by automatically identifying the target area, the blink data can be accurately obtained, and an accurate data basis is provided for subsequent prejudgment.
In an embodiment, the image recognition module 52 may be further configured to: inputting each frame of image of the video data into an identification model to obtain a target area between an upper eyelid and a lower eyelid; wherein the recognition model comprises a neural network model. The recognition model may be trained from standard image data.
In an embodiment, the status determination module 54 may be further configured to: acquiring the complete blink times and the incomplete blink times in the video data according to the opening and closing state of the target area of all the frame images in the video data; the single complete blink indicates that the open-close state of the target region changes from the eye-open state to the eye-closed state and then changes to the eye-open state again, and the single incomplete blink indicates that the open-close state of the target region changes from the eye-open state to the half-open state and then changes to the eye-open state again.
Fig. 6 is a schematic structural diagram of a blink data statistics apparatus according to another exemplary embodiment of the present application. As shown in fig. 6, the image recognition module 52 may include: a seed selection unit 521, configured to select a seed region enclosed in each frame of image of the video data according to a pixel value of a region between the upper eyelid and the lower eyelid; a boundary line determining unit 522 configured to, when a seed region exists, expand around the seed region to determine a boundary line of the target region; a binarization unit 523 configured to perform binarization processing on each frame of image in which the boundary line of the target region is determined, to obtain a binarized image including the target region.
In an embodiment, the status determination module 54 may be further configured to: and when the seed area does not exist, determining that the target area is in a closed-eye state.
In one embodiment, as shown in fig. 6, the height calculating module 53 may include: a first obtaining unit 531, configured to obtain a first reference line according to end points of two ends of an upper eyelid; a second obtaining unit 532, configured to obtain a second reference line according to end points of both ends of the lower eyelid; a direction determining unit 533 configured to determine a palpebral fissure height direction according to the first reference line and the second reference line; and a height selecting unit 534 for selecting the maximum distance between the upper eyelid and the lower eyelid in the direction of the palpebral fissure height as the palpebral fissure height.
In one embodiment, as shown in fig. 6, the status determination module 54 may include: a maximum value selecting unit 541 configured to select a maximum value and a minimum value of all eyelid fissure heights of the video data; a difference calculating unit 542 for calculating a maximum difference between the palpebral fissure height and the maximum value and a minimum difference between the palpebral fissure height and the minimum value; the state determining unit 543 is configured to determine that the open-close state of the target region is an eye-open state when the maximum difference is smaller than or equal to a first preset difference, determine that the open-close state of the target region is an eye-close state when the minimum difference is smaller than or equal to a second preset difference, and determine that the open-close state of the target region is a half-open state when the maximum difference is larger than the first preset difference and the minimum difference is larger than the second preset difference.
Next, an electronic apparatus according to an embodiment of the present application is described with reference to fig. 7. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.
FIG. 7 illustrates a block diagram of an electronic device in accordance with an embodiment of the present application.
As shown in fig. 7, the electronic device 10 includes one or more processors 11 and memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 11 to implement the statistical methods of blink data and/or other desired functionality of the various embodiments of the application described above. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
When the electronic device is a stand-alone device, the input means 13 may be a communication network connector for receiving the acquired input signals from the first device and the second device.
The input device 13 may also include, for example, a keyboard, a mouse, and the like.
The output device 14 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present application are shown in fig. 7, and components such as buses, input/output interfaces, and the like are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
In addition to the above-described methods and apparatus, embodiments of the present application may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the statistical method of blink data according to various embodiments of the present application described in the "exemplary methods" section above of this specification.
The computer program product may be written with program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in the statistical method of blink data according to various embodiments of the present application described in the "exemplary methods" section above in this description.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. A statistical method of blink data, comprising:
acquiring video data containing eye movement in a preset time period;
identifying each frame of image of the video data to obtain a target area between an upper eyelid and a lower eyelid;
calculating the palpebral fissure height corresponding to the target area;
determining the opening and closing state of the target area according to the eyelid fissure height; wherein the open-close state comprises an eye opening state, an eye closing state and a half eye opening state; and
and counting to obtain blink data in the preset time period according to the opening and closing states of the target areas of all the frame images in the video data.
2. The statistical method of blink data according to claim 1, wherein the identifying each frame of image of the video data resulting in a target area between an upper eyelid and a lower eyelid comprises:
inputting each frame of image of the video data into a recognition model to obtain the target area between the upper eyelid and the lower eyelid;
wherein the recognition model comprises a neural network model.
3. The statistical method of blink data according to claim 1, wherein the identifying each frame of image of the video data resulting in a target area between an upper eyelid and a lower eyelid comprises:
selecting a seed region enclosed in each frame of image of the video data according to the pixel value of the region between the upper eyelid and the lower eyelid;
when the seed region exists, the boundary line of the target region is determined by taking the seed region as a center and expanding the seed region to the periphery; and
and carrying out binarization processing on each frame of image with the boundary line of the target area determined to obtain a binarized image containing the target area.
4. The statistical method of blink data according to claim 3, wherein the identifying each frame of image of the video data resulting in a target area between an upper eyelid and a lower eyelid comprises:
and when the seed area does not exist, determining that the target area is in a closed-eye state.
5. The statistical method of blink data according to claim 1, wherein the calculating the palpebral fissure height for the target region comprises:
obtaining a first reference line according to end points of two ends of the upper eyelid;
obtaining a second reference line according to end points of two ends of the lower eyelid;
determining the height direction of the eyelid fissure according to the first reference line and the second reference line; and
and selecting the maximum distance between the upper eyelid and the lower eyelid in the eyelid fissure height direction as the eyelid fissure height.
6. The statistical method of blink data according to claim 1, wherein the determining of the opening and closing state of the target area based on the palpebral fissure height comprises:
selecting the maximum value and the minimum value of all the palpebral fissure heights of the video data;
calculating a maximum difference between the palpebral fissure height and the maximum value and a minimum difference between the palpebral fissure height and the minimum value;
when the maximum difference is smaller than or equal to a first preset difference, determining that the opening and closing state of the target area is an eye opening state;
when the minimum difference is smaller than or equal to a second preset difference, determining that the opening and closing state of the target area is an eye closing state; and
and when the maximum difference is larger than the first preset difference and the minimum difference is larger than the second preset difference, determining that the opening and closing state of the target area is a half-opening state.
7. The method for counting blink data according to claim 1, wherein the counting blink data in the preset time period according to the opening and closing states of the target area of all the frame images in the video data comprises:
acquiring the complete blink times and the incomplete blink times in the video data according to the opening and closing states of the target areas of all the frame images in the video data; wherein, the single complete blink indicates that the opening and closing state of the target area is changed from the eye opening state to the eye closing state and then changed to the eye opening state again, and the single incomplete blink indicates that the opening and closing state of the target area is changed from the eye opening state to the eye half opening state and then changed to the eye opening state again.
8. A statistical apparatus for blink data, comprising:
the video acquisition module is used for acquiring video data containing eye movement in a preset time period;
the image identification module is used for identifying each frame of image of the video data to obtain a target area between an upper eyelid and a lower eyelid;
the height calculation module is used for calculating the palpebral fissure height corresponding to the target area;
the state determining module is used for determining the opening and closing state of the target area according to the eyelid fissure height; wherein the open-close state comprises an eye opening state, an eye closing state and a half eye opening state; and
and the data counting module is used for counting blink data in the preset time period according to the opening and closing states of the target areas of all the frame images in the video data.
9. A computer-readable storage medium, the storage medium storing a computer program for performing the statistical method of blink data as claimed in any one of the preceding claims 1 to 7.
10. An electronic device, the electronic device comprising:
a processor; and
a memory for storing the processor-executable instructions;
the processor is configured to perform the statistical method of blink data as claimed in any one of the preceding claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110915881.XA CN113616196A (en) | 2021-08-10 | 2021-08-10 | Blink data statistical method, device, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110915881.XA CN113616196A (en) | 2021-08-10 | 2021-08-10 | Blink data statistical method, device, medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113616196A true CN113616196A (en) | 2021-11-09 |
Family
ID=78384181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110915881.XA Pending CN113616196A (en) | 2021-08-10 | 2021-08-10 | Blink data statistical method, device, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113616196A (en) |
-
2021
- 2021-08-10 CN CN202110915881.XA patent/CN113616196A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2018385433B2 (en) | Digital visual acuity eye examination for remote physician assessment | |
| JP7070605B2 (en) | Focus range estimator, its method and program | |
| US20120323087A1 (en) | Affective well-being supervision system and method | |
| Valente et al. | Automatic diagnosis of strabismus in digital videos through cover test | |
| KR101984995B1 (en) | Artificial intelligence visual field alalysis method and apparatus | |
| JP6939533B2 (en) | Attention level estimation device, attention level estimation model learning device, attention level estimation method, attention level estimation model learning method, and program | |
| CN106526857B (en) | Focus adjustment method and device | |
| CN116491893B (en) | Method and device for evaluating change of ocular fundus of high myopia, electronic equipment and storage medium | |
| JP6479708B2 (en) | Feature amount extraction apparatus, estimation apparatus, method thereof, and program | |
| CN113662506B (en) | Cornea surface morphology measurement method and device, medium and electronic equipment | |
| US11583214B2 (en) | Adaptive psychological assessment tool | |
| CN113616196A (en) | Blink data statistical method, device, medium and electronic equipment | |
| JP6377545B2 (en) | Personal identification method, personal identification device, program | |
| KR102208508B1 (en) | Systems and methods for performing complex ophthalmic tratment | |
| CN113662566B (en) | Blink detection method and device based on electric signals, medium and electronic equipment | |
| CN113616195A (en) | Method, apparatus, medium, and device for distinguishing blink types based on state information | |
| Lee et al. | Eye-wearable head-mounted tracking and gaze estimation interactive machine system for human–machine interface | |
| Al-Oudat et al. | A smart intraocular pressure risk assessment framework using frontal eye image analysis | |
| CN115223232A (en) | Eye health comprehensive management system | |
| CN113827179B (en) | Eyelid pressure detection method and device, medium and electronic equipment | |
| JP2017202048A (en) | Feature amount extraction device, estimation device, method for the same and program | |
| CN112528714A (en) | Single light source-based gaze point estimation method, system, processor and equipment | |
| CN116309391B (en) | Image processing method and device, electronic equipment and storage medium | |
| KR20200012405A (en) | Terminal Usage Pattern Information Collection Management Method And System | |
| Wong | Instantaneous and Robust Pupil-Based Cognitive Load Measurement for Eyewear Computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20211109 |
|
| WD01 | Invention patent application deemed withdrawn after publication |