WO2021093744A1 - 一种瞳孔直径的测量方法、装置及计算机可读存储介质 - Google Patents

一种瞳孔直径的测量方法、装置及计算机可读存储介质 Download PDF

Info

Publication number
WO2021093744A1
WO2021093744A1 PCT/CN2020/127902 CN2020127902W WO2021093744A1 WO 2021093744 A1 WO2021093744 A1 WO 2021093744A1 CN 2020127902 W CN2020127902 W CN 2020127902W WO 2021093744 A1 WO2021093744 A1 WO 2021093744A1
Authority
WO
WIPO (PCT)
Prior art keywords
pupil
diameter
position information
video
image
Prior art date
Application number
PCT/CN2020/127902
Other languages
English (en)
French (fr)
Inventor
张玉楼
蔚鹏飞
黄康
张佳佳
王立平
Original Assignee
中国科学院深圳先进技术研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国科学院深圳先进技术研究院 filed Critical 中国科学院深圳先进技术研究院
Publication of WO2021093744A1 publication Critical patent/WO2021093744A1/zh

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/11—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils
    • A61B3/112—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for measuring interpupillary distance or diameter of pupils for measuring diameter of pupils
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • A61B3/145—Arrangements specially adapted for eye photography by video means

Definitions

  • This application relates to the field of computer technology, and in particular to a method, device and computer-readable storage medium for measuring pupil diameter.
  • the pupil is the small round hole in the center of the iris in the eye of an animal or human, which is a passage for light to enter the eye.
  • the expansion and contraction of the upper iris sphincter can make the pupils shrink or dilate, that is, the diameter of the pupils will become smaller or larger.
  • Measuring and analyzing changes in pupil diameter can help to study the way the neural circuits in the brain regulate behavior, and it can also directly reflect whether you have certain diseases. Therefore, how to quickly and accurately measure pupil diameter has become a problem. Problems that need to be solved urgently.
  • a common method of measuring the pupil diameter is to take multiple pictures of the pupil, place a scale on one side of the eyeball during the shooting process, perform edge detection and other processing on the picture, and calculate it by combining the scale on the scale.
  • the diameter of the pupil is to take multiple pictures of the pupil, place a scale on one side of the eyeball during the shooting process, perform edge detection and other processing on the picture, and calculate it by combining the scale on the scale.
  • the diameter of the pupil takes a long time and is likely to cause subjective errors. Therefore, the measurement efficiency is low and the measurement result is inaccurate.
  • the embodiments of the application provide a method, device, and computer-readable storage medium for measuring pupil diameter, which can greatly reduce subjective errors of experimenters and errors caused by measuring instruments. At the same time, the calculation is simple, and the measurement efficiency and measurement results are improved. Accuracy.
  • an embodiment of the present application provides a method for measuring pupil diameter, and the method includes:
  • a deep convolutional neural network is used to determine the position information of the M different feature points of the first pupil, where the first pupil is any pupil of the pupils included in the at least one frame, and the M is a positive value greater than or equal to 2.
  • the diameter of the first pupil is calculated according to the position information.
  • the M different feature points include feature points corresponding to the center of the first pupil and M-1 points on the circumference of the pupil;
  • the using the deep convolutional neural network to determine the position information of the M feature points of the first pupil includes:
  • the position information of the circle center is the position information of the occluded circle center identified or predicted by the deep convolutional neural network.
  • the location information includes the coordinates of the feature point and the confidence probability corresponding to the coordinates
  • the calculating the diameter of the first pupil according to the position information includes:
  • the first feature point is determined as an available feature point, and multiple available feature points corresponding to the first pupil are obtained ;
  • the diameter of the first pupil is calculated according to the coordinates of the plurality of available feature points.
  • the calculating the diameter of the first pupil according to the position information further includes:
  • the image in the previous frame adjacent to the image where the first pupil is located in the video to be measured is determined.
  • the available position information corresponding to the coordinates of the first feature point is determined as the position information of the available feature point corresponding to the first pupil in the image where the first pupil is located.
  • the calculating the diameter of the first pupil according to the coordinates of the multiple available feature points includes:
  • the diameter of the first pupil is calculated according to the coordinates of the plurality of available feature points and the first calculation method.
  • the method further includes:
  • the first image is an image that includes the first pupil in the video to be tested Any image in.
  • the method further includes:
  • an embodiment of the present application provides a pupil diameter measuring device, including:
  • the acquiring unit is configured to acquire a video to be tested, and there is at least one frame of images containing pupils in the video to be tested;
  • the determining unit is configured to use a deep convolutional neural network to determine the position information of M different feature points of the first pupil, where the first pupil is any pupil of the pupils included in the at least one frame, and the M is greater than Or a positive integer equal to 2;
  • the calculation unit is configured to calculate the diameter of the first pupil according to the position information.
  • the M different feature points include feature points corresponding to the center of the first pupil and M-1 points on the circumference of the pupil;
  • the determining unit is specifically used for:
  • the position information of the circle center is the position information of the occluded circle center identified or predicted by the deep convolutional neural network.
  • the location information includes the coordinates of the feature point and the confidence probability corresponding to the coordinates
  • the calculation unit is specifically used for:
  • the first feature point is determined as an available feature point, and multiple available feature points corresponding to the first pupil are obtained ;
  • the diameter of the first pupil is calculated according to the coordinates of the plurality of available feature points.
  • the calculation unit is further configured to:
  • the image in the previous frame adjacent to the image where the first pupil is located in the video to be measured is determined.
  • the available position information corresponding to the coordinates of the first feature point is determined as the position information of the available feature point corresponding to the first pupil in the image where the first pupil is located.
  • the calculation unit is specifically configured to:
  • the diameter of the first pupil is calculated according to the coordinates of the plurality of available feature points and the first calculation method.
  • the device further includes:
  • the labeling unit is configured to label the first pupil in the first image according to the position information corresponding to the first pupil in the first image, and the first image is the video to be tested containing the Any one of the images of the first pupil.
  • the device further includes:
  • the drawing unit is configured to draw a change curve of the diameter of the first pupil according to the diameter of the first pupil in the first image.
  • an embodiment of the present application provides an electronic device.
  • the electronic device includes a processor and a memory, and the processor and the memory are connected to each other.
  • the memory is used to store a computer program that supports the terminal device to execute the method provided by the foregoing first aspect and/or any one of the possible implementations of the first aspect
  • the computer program includes program instructions, and the processor is configured to call the foregoing
  • the program instruction executes the method provided in the first aspect and/or any one of the possible implementation manners of the first aspect.
  • an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program includes program instructions that, when executed by a processor, cause the processor to execute
  • the computer program includes program instructions that, when executed by a processor, cause the processor to execute
  • a video to be measured containing at least one frame of pupil images is input into a deep convolutional neural network to obtain the position information of the feature points corresponding to the pupils in the image, and the pupil diameter can be calculated according to the position information of the feature points corresponding to the pupils ,
  • the subjective error of the experimenter and the error caused by the measuring instrument can be greatly reduced, and the accuracy of the measurement result is improved.
  • the diameter is calculated by the position information, the calculation is simple, and the measurement efficiency is improved.
  • FIG. 1 is a schematic flowchart of a method for measuring pupil diameter provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of another pupil diameter measurement method provided by an embodiment of the present application.
  • Fig. 3 is a schematic structural diagram of a pupil diameter measuring device provided by an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of unobstructed pupil feature point position marks provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of the position mark of the feature point of a partially obscured pupil provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of feature point position marks of two consecutive frames of eye movements provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of marking pupils after fitting a circle by feature points of unobstructed pupils according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of marking the pupil after the feature points of the partially occluded pupil are fitted to a circle provided by an embodiment of the present application.
  • the term “if” can be construed as “when” or “once” or “in response to determination” or “in response to detection” depending on the context .
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.
  • FIG. 1 is a schematic flowchart of a method for measuring pupil diameter according to an embodiment of the present application. As shown in Figure 1, the pupil diameter measurement method includes:
  • the pupil diameter measuring device in the embodiment of the present application may include a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), a personal computer, a mobile Internet device (Mobile Internet Device, MID), These include, but are not limited to, the various electronic devices mentioned above that can invoke and execute software programs.
  • PDA Personal Digital Assistant
  • MID Mobile Internet Device
  • acquiring the video to be measured means that there is at least one frame of video containing pupil images.
  • the device for acquiring the video to be measured can be integrated into the above-mentioned pupil diameter measuring device, and the video to be measured can be obtained by collecting the video through a device such as a camera.
  • the terminal device that collects the video can be connected externally, and the video to be tested can be obtained through the external terminal device, or it can be the video to be tested obtained through data transmission through a wireless local area network or the Internet.
  • the above-mentioned video to be tested is a frame, it is an image including the pupil.
  • the image acquisition can be acquired by the device, or the received image, or during the acquisition or reception.
  • the images captured in the video are not limited here.
  • the above-mentioned video to be tested can be multiple types of videos that contain pupil images, can be a video that contains a pupil eye movement that is partially occluded, can also be a pupil eye movement video that is not blocked, or a moving pupil video.
  • the pupil video with fixed eye movements can also be a combination of the above videos, which is not limited here.
  • the aforementioned pupils may be human pupils or animal pupils, and there is no limitation here.
  • the pupil measurement model designed in this application can effectively measure the pupil diameter of a person or animal in a variety of scenarios, has a wide application range, and can meet the needs of measurement in various scenarios.
  • the obtained video is input into the trained deep convolutional neural network for prediction and recognition, and the position information of the pupil feature points in the video to be measured is obtained.
  • the pupil diameter can be calculated by the position information.
  • the diameter of the pupil can be obtained by measuring the video of the pupil appearing in the video for each period of time.
  • the above-mentioned deep convolutional neural network may be a variant network generated by the combination of a deep convolutional network and a residual network, or a specific target detection network, which is not limited here.
  • This application takes the variant network produced by the combination of deep convolutional neural network and residual network as an example to explain.
  • Using residual network can not only improve network efficiency, but it can also be effective when there are too many layers of convolutional neural network.
  • the advantage of avoiding gradient disappearance or gradient explosion Input the pupil video or pupil image obtained in step 101 into the variant network generated by the combination of the deep convolutional neural network and the residual network, and the multiple pupils of one or more pupils in the video identified in the video can be obtained.
  • the location information of a feature point is not only improve network efficiency, but it can also be effective when there are too many layers of convolutional neural network.
  • the feature points include the center of the circle and the points on the circle.
  • the actual output of the number of feature points is related to the training method.
  • the labeling method is the same at the time. Specifically, it can be marked by angle.
  • the pupil can be marked with 9 marks of 0 degree, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, and 360 degrees based on the center of the circle.
  • a feature point location marking method for example, a feature point at 0 degrees, or a feature point at 180 degrees, where you can mark according to the degree of pupil occlusion in the actual video
  • the marking method of the point position you can also choose the marking method of 9 characteristic point positions, which is not limited here. It is understandable that the method of calculating the pupil diameter is different if the number of the acquired characteristic points is different.
  • a variant network obtained by combining the deep neural network and the residual network needs to be trained to train the model.
  • the training image set there are many ways to obtain the training image set. For example, you can obtain the training video containing the marked pupils. First, the training video is segmented to obtain the multi-frame images contained in the video, and then the clustering method is used. , It can be through the k-means clustering algorithm, each frame of the obtained image is clustered according to the color histogram to obtain images in multiple categories, that is, each category represents the image of the pupil in a different state . Then extract a certain number of images from each category for annotation and input them into the above-mentioned variant network combining the deep neural network and the residual network for training, then the trained network model can be obtained, which is what this application needs to use model.
  • the above-mentioned k-means clustering can be obtained by first taking the color histogram (R, G, B histogram) of the first frame of the image obtained in the playback order in the training video as the centroid, and obtaining the R of the first frame of image,
  • the histograms of the three images G and B are 1*256 matrices; compare the second frame image and the first frame image as the initial centroids, that is, compare the distances of the RGB three centroids of the two images.
  • a distance threshold is preset here. When it is judged that the distance between the three centroids of the two images is less than the distance threshold, the second frame is added to the cluster of the first frame.
  • the third frame of image and the fourth frame of image Comparing with the initial test centroid, if it is judged that the distance is greater than the distance threshold, a new centroid is generated. If it is not similar, a new cluster is generated. In the next comparison, the new frame image will be compared with the centroids of all clusters, and the cluster to belong to is selected or a new cluster is generated. In this way, one or more clusters will be generated, and each frame of image in the training video will be attributable.
  • the above clustering according to histogram features is only an example. Actually, clustering can be performed according to the artificially divided pupil states in the image to obtain images of the pupils in each state. Clustering can also be performed according to other characteristics of the pupils in the image, which is not done here. limited.
  • each cluster represents the different states of the pupil, and a certain number of images are extracted from the images of each cluster, and divided into training set and validation set according to a certain ratio, where the ratio can be 7:3, or it can be 8:2, mark in the divided training set images.
  • the clustered images can also be divided into training set, validation set and test set.
  • the specific ratio can be set manually, and there is no limitation here.
  • the point position mark can also be the 9 feature point position marks of 0 degree, 45 degree, 90 degree, 135 degree, 180 degree, 225 degree, 270 degree, 315 degree, and 360 degree. It is understandable that during training The number of marked feature points corresponds to the number of feature points output during measurement.
  • the above-mentioned trained model is tested on the test set. If the correct rate of the test set and the training set is not much different, it means that the trained model is relatively stable and generalized. Ability is relatively strong. On the contrary, it means that the model is unavailable and needs to be analyzed to find the reasons.
  • the learning curve and the performance of generalization ability can be used to judge the problems encountered by the model and adopt corresponding methods to solve them. For example, the problem may be the under-fitting and over-fitting of the model. If the model is over-fitted, you can adjust it by increasing the amount of data, or by using batch normalization. Adjust by reducing network complexity, etc.
  • the model is under-fitting, you can add feature items to the model, make the model more complex, reduce the regularization parameters and other methods to adjust.
  • the training set is re-thrown into the variant network produced by the combination of the deep convolutional network and the residual network, and the network model training is performed until a model with good generalization ability is trained.
  • the program of the model can be packaged as a file with a suffix of .py, so that the model can be run on multiple different systems, which increases the application scope of the method. For example, it can run on a Linux system, or it can run on a Windows system. The portability of this solution is improved to meet the different needs of multiple users for the system.
  • the difference between the pupil video of a human, the pupil video of a mouse, and the pupil video of a puppy will be relatively large, and the image can be re-acquired to train a new model through the network.
  • These models respectively detect and track changes in pupil diameter to ensure that the best results are achieved.
  • the model training and parameter adjustment proposed in this application are also more convenient, which greatly improves the applicability of the model.
  • the video to be tested is input into the network.
  • the feature points of the pupil can be identified.
  • the network model is trained through neural
  • the network obtains pupil position information, spatial characteristic information, histogram characteristic information, and RGB characteristic information to train the model.
  • the model can predict the center position of the pupil based on these characteristics of the pupil in the video. Among them, if the video sample size is small and the occlusion is not serious, you can use the 9 features of marking the pupil center, 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, and 360 degrees. Point marking method, this method has certain requirements on computer performance, but it can get a more accurate pupil diameter.
  • the above-mentioned input video may be input one video at a time, or multiple videos at a time. If one video is input, the pupil diameter file in the video can be obtained. If multiple videos are input , You can get the pupil diameter file corresponding to each video separately, where the input video can be the file manually imported into the video, or the path of the input video file, and all the video files under this path are input to the trained Model.
  • the pupils of the multiple videos can be output It is a pupil diameter file, and there is no limit to the number and method of output files.
  • the above-mentioned location information includes the coordinates of the feature point and the confidence probability corresponding to each coordinate.
  • the feature points are the center of the circle and one or more points on the circumference.
  • the number of feature points is different, and the calculation methods that can be used are also different. There is a corresponding relationship between the number of feature points and the calculation method.
  • the pupil diameter calculation method can be determined according to the number of feature points, and the file of the feature point position information obtained above is input into In the calculation program of pupil diameter, it can be directly run by inputting it into matlab where the program has been written, and there is no limitation here. It is understandable that after the pupil diameter is calculated, the pupil diameter can be output in the form of a file, or can be output in a visual interface, which is not limited here.
  • a video to be measured containing at least one frame of pupil images is input into a deep convolutional neural network to obtain the position information of the feature points corresponding to the pupils in the image, and the pupil diameter can be calculated according to the position information of the feature points corresponding to the pupils ,
  • the subjective error of the experimenter and the error caused by the measuring instrument can be greatly reduced, and the accuracy of the measurement result is improved.
  • the diameter is calculated by the position information, the calculation is simple, and the measurement efficiency is improved.
  • FIG. 2 is a schematic flowchart of another pupil diameter measurement method provided by an embodiment of the present application.
  • the pupil diameter measurement method includes:
  • step 201 Determine whether the confidence probability corresponding to the first feature point coordinate of the first pupil is greater than or equal to a threshold, and if it is determined that the confidence probability corresponding to the first feature point coordinate of the first pupil is greater than or equal to the threshold, perform step 202.
  • the obtained position information of the pupil includes the coordinates of the characteristic points of the pupil and the confidence probability corresponding to each coordinate.
  • the first pupil is any pupil in at least one frame of the image containing the pupil in the video to be measured.
  • the method for calculating the pupil diameter may include the following steps: firstly, remove the outliers of the characteristic coordinates according to the confidence probability, and then select the coordinate characteristic points with the confidence probability greater than the threshold, that is, judge each pupil characteristic point in each frame of the image. Whether the confidence probability corresponding to the coordinate is greater than or equal to the threshold.
  • the above threshold may be 0.9, 0.93, or 0.95. This may be a probability threshold set by a technician, which is not limited here.
  • the number of feature points is determined according to the degree of occlusion of the pupil. If the occlusion is severe, only the center of the pupil and the position of 0 degrees directly above the pupil can be marked. If the amount of video data is small and the pupil occlusion is not serious, you can use 4 feature points to mark the pupil center, pupil 0 degree direction, 45 degree direction, and -45 degree direction. If the video sample size is small and the occlusion is not serious, you can use it Marking the 9 characteristic points of pupil center, 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, and 360 degrees.
  • the feature point After obtaining the location information of the feature point, by judging the confidence probability in the location information, when it is determined that the confidence probability corresponding to each feature point is greater than or equal to the threshold, the feature point is determined as a usable feature point, That is, when the pupil diameter is calculated next, the available feature points are used to calculate the diameter of the corresponding pupil.
  • the processing method is different according to the number of feature points: when the number of feature points is 2, only the center of the pupil and the 0 degree position directly above the pupil are marked ,
  • the pupil diameter in the previous frame image is used to determine the pupil diameter corresponding to the current frame image; when the number of feature points is greater than 3
  • the above feature point is determined to be an unusable feature point, that is, when the pupil diameter is calculated, the point is discarded.
  • the available position information corresponding to the coordinates of the first feature point in the previous frame image adjacent to the image It is determined as the position information of the available feature point corresponding to the first pupil in the image where the first pupil is located, that is, the pupil diameter calculated according to the previous frame of the video playback sequence to be measured is used as the pupil diameter of the frame of the image.
  • the corresponding calculation method can be used to calculate the pupil diameter.
  • the calculation method of pupil diameter which may be a method of calculating the diameter obtained according to the correspondence between the number of available feature points and the calculation method.
  • the distance between the two points can be calculated by Euclidean distance, and the obtained distance is the diameter corresponding to the pupil .
  • the number of available feature points is greater than 2, that is, one center feature point and at least two feature points at most 9 feature points are identified on the circumference, it is still possible to calculate the multiple center points and the maximum through the calculation method of Euclidean distance. The distance between multiple feature points on the circumference is obtained, and multiple distances are obtained, and the pupil diameter corresponding to the image is obtained by averaging.
  • the number of usable feature points on the circumference is at least 3, that is, a center feature point and at least 3 usable feature points on the circumference
  • the calculation method in this application is simple and efficient, so that the calculation time of the pupil diameter is greatly shortened, and the measurement efficiency of the pupil diameter is improved.
  • a sliding window function may be used to denoise, or other methods may be used to denoise, which is not limited here.
  • the position information of the feature point can be marked in the corresponding image.
  • Figure 5 is an image marked according to the position of the feature point.
  • the feature points can be marked with a preset color and marking method, which can mark the position of each feature point.
  • Figure 5 shows the situation without any occlusion. Under the marking method.
  • Figure 6 shows the marking method where the pupil is partially occluded
  • Figure 7 shows the marking method where the pupils of the eyes in two consecutive frames of images have moved separately.
  • the diameter of the pupil and the position information of multiple feature points are obtained, which can be verified in the visual interface, and can be visually observed according to the marked video output by the model to verify whether the marking is accurate, or through calculation
  • the diameter of the pupil and the coordinates of the characteristic points of the pupil fit a circle and mark it on the corresponding pupil in the image.
  • a circle can be drawn in the original image containing the pupil, which can be verified by intuitive observation.
  • the pupil when the pupil is partially occluded, it can also be verified by plotting the existing feature point coordinates and diameter on the image.
  • the pupil diameter of each frame in the video to be tested can also be drawn in accordance with the video playback sequence by drawing the change curve of the time and pupil diameter for verification.
  • the pupil diameter change curve can be the change curve of the pupil diameter in the video to be tested, or It may be that the pupil diameter in the video to be measured measured by other methods is compared with the pupil diameter of the video to be measured measured in this application and plotted in the diameter change curve. Both the marked image and the pupil diameter change curve can be displayed in the visual interface, and only one verification method can be displayed, or a combination of several verifications can be displayed, which is not limited here.
  • a video to be measured containing at least one frame of pupil images is input into a deep convolutional neural network to obtain the position information of the feature points corresponding to the pupils in the image, and the pupil diameter can be calculated according to the position information of the feature points corresponding to the pupils ,
  • the subjective error of the experimenter and the error caused by the measuring instrument can be greatly reduced, and the accuracy of the measurement result is improved.
  • the diameter is calculated by the position information, the calculation is simple, and the measurement efficiency is improved.
  • FIG. 3 is a schematic structural diagram of a pupil diameter measuring device provided by an embodiment of the present application.
  • the pupil diameter measuring device 3000 includes:
  • the acquiring unit 301 is configured to acquire a video to be tested, and there is at least one frame of images containing pupils in the video to be tested;
  • the determining unit 302 is configured to determine the position information of the M different feature points of the first pupil using a deep convolutional neural network, the first pupil is any pupil of the pupils included in the at least one frame, and the M is greater than or equal to A positive integer of 2;
  • the calculation unit 303 is configured to calculate the diameter of the first pupil according to the position information.
  • the foregoing M different feature points include feature points corresponding to the center of the first pupil and M-1 points on the circumference of the pupil;
  • the above determining unit 302 is specifically configured to:
  • the position information of the center of the circle is the position information of the center of the occluded circle recognized or predicted by the deep convolutional neural network.
  • the above-mentioned location information includes the coordinates of the above-mentioned feature points and the confidence probability corresponding to the above-mentioned coordinates;
  • the foregoing calculation unit 303 is specifically configured to:
  • the first feature point is determined as an available feature point, and multiple available feature points corresponding to the first pupil are obtained;
  • the diameter of the first pupil is calculated according to the coordinates of the multiple available feature points.
  • the foregoing calculation unit 303 is further configured to:
  • the first feature point in the previous frame of the image adjacent to the image where the first pupil is located in the video to be tested The available position information corresponding to the coordinates is determined as the position information of the available feature point corresponding to the first pupil in the image where the first pupil is located.
  • the foregoing calculation unit 303 is specifically configured to:
  • the diameter of the first pupil is calculated according to the coordinates of the multiple available feature points and the first calculation method.
  • the foregoing apparatus 3000 further includes:
  • the labeling unit 304 is configured to label the first pupil in the first image according to the position information corresponding to the first pupil in the first image, and the first image is an image containing the first pupil in the video to be tested Any image in.
  • the foregoing apparatus 3000 further includes:
  • the drawing unit 305 is configured to draw a change curve of the diameter of the first pupil according to the diameter of the first pupil in the first image.
  • the pupil diameter measuring device in the embodiment of the present application inputs the video to be measured containing at least one frame of pupil images into the deep convolutional neural network to obtain the position information of the feature points corresponding to the pupils in the image, and according to the position information of the feature points corresponding to the pupils.
  • the pupil diameter can be calculated, and the feature point position information obtained by inputting the video to be measured into the network can greatly reduce the subjective error of the experimenter and the error caused by the measuring instrument, and improve the accuracy of the measurement result.
  • the diameter is calculated by the position information, the calculation is simple, and the measurement efficiency is improved.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device in this embodiment may include: one or more processors 401, an input device 402, an output device 403, and a memory 404.
  • the aforementioned processor 401, input device 402, output device 403, and memory 402 are connected by a bus.
  • the memory 402 is used to store a computer program
  • the computer program includes program instructions
  • the processor 401 is used to execute the program instructions stored in the memory 402
  • the input device 402 is used to input data
  • the output device 403 is used to output data.
  • the aforementioned processor 401 is configured to invoke program instructions to execute the following steps:
  • the first pupil is any pupil of the pupils included in the at least one frame, and the M is a positive integer greater than or equal to 2;
  • the diameter of the first pupil is calculated based on the position information.
  • the foregoing M different feature points include feature points corresponding to the center of the first pupil and M-1 points on the circumference of the pupil;
  • the aforementioned processor 401 uses the aforementioned deep convolutional neural network to determine the position information of the M feature points of the first pupil, including:
  • the position information of the center of the circle is the position information of the center of the occluded circle recognized or predicted by the deep convolutional neural network.
  • the above-mentioned location information includes the coordinates of the above-mentioned feature points and the confidence probability corresponding to the above-mentioned coordinates;
  • the processor 401 calculates the diameter of the first pupil according to the position information, including:
  • the first feature point is determined as an available feature point, and multiple available feature points corresponding to the first pupil are obtained;
  • the diameter of the first pupil is calculated according to the coordinates of the multiple available feature points.
  • the processor 401 calculating the diameter of the first pupil according to the position information further includes:
  • the processor 401 calculates the diameter of the first pupil according to the coordinates of the multiple available feature points, including:
  • the diameter of the first pupil is calculated according to the coordinates of the multiple available feature points and the first calculation method.
  • processor 401 is configured to invoke program instructions to execute the following steps:
  • the first pupil is marked in the first image, and the first image is any image of the image including the first pupil in the video to be measured.
  • processor 401 is configured to invoke program instructions to execute the following steps:
  • the aforementioned processor 401 may be a central processing unit (CPU), and the processor may also be other general-purpose processors or digital signal processors (DSP). , Application specific integrated circuit (ASIC), ready-made programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 402 may include a read-only memory and a random access memory, and provides instructions and data to the processor 401.
  • a part of the memory 402 may also include a non-volatile random access memory.
  • the memory 402 may also store device type information.
  • the above-mentioned terminal device can execute the implementation manners provided in the steps in Figures 1 to 2 through its built-in functional modules.
  • the implementation manners provided in the above-mentioned steps please refer to the implementation manners provided in the above-mentioned steps, which will not be repeated here.
  • the electronic device in the embodiment of the present application inputs the to-be-tested video containing at least one frame of pupil image into the deep convolutional neural network to obtain the position information of the feature point corresponding to the pupil in the image, and can calculate according to the position information of the feature point corresponding to the pupil Obtaining the diameter of the pupil, by inputting the video to be measured into the network to obtain the position information of the characteristic points, the subjective error of the experimenter and the error caused by the measuring instrument can be greatly reduced, and the accuracy of the measurement result is improved. At the same time, the diameter is calculated by the position information, the calculation is simple, and the measurement efficiency is improved.
  • a computer-readable storage medium stores a computer program, and the computer program is executed by a processor to realize:
  • the aforementioned computer-readable storage medium may be the internal storage unit of the aforementioned terminal in any of the aforementioned embodiments, such as the hard disk or memory of the terminal.
  • the computer-readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, and a flash memory card equipped on the terminal. (Flash card) and so on.
  • the aforementioned computer-readable storage medium may also include both an internal storage unit of the aforementioned terminal and an external storage device.
  • the aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned terminal.
  • the above-mentioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
  • the disclosed system, server, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the above-mentioned units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.
  • the units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.
  • several instructions are included to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the above methods of the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Multimedia (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)

Abstract

一种瞳孔直径的测量方法，该方法包括：获取待测视频，待测视频中存在至少一帧包含瞳孔的图像；使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，第一瞳孔为至少一帧包含的瞳孔中的任一瞳孔，M为大于或等于2的正整数；根据位置信息计算第一瞳孔的直径。该测量方法极大的减少实验人员主观误差、测量仪器所造成的误差，同时计算简单，提升了测量的效率以及测量结果的准确性。

Description

一种瞳孔直径的测量方法、装置及计算机可读存储介质 技术领域
本申请涉及计算机技术领域，尤其涉及一种瞳孔直径的测量方法、装置及计算机可读存储介质。
 背景技术
瞳孔，是动物或人眼睛内虹膜中心的小圆孔，为光线进入眼睛的通道。虹膜上括约肌的伸缩，可以使瞳孔缩小或散大，即瞳孔的直径会发生变小或变大。测量并分析瞳孔直径的变化，可以有助于研究大脑中的神经环路调控行为的方式，也可以直接反应出是否患有某些疾病，因此，如何快速准确的测量出瞳孔直径已经成为了一个亟需解决的问题。
目前，一种常见的测量瞳孔直径的方法是通过拍摄多张瞳孔的照片，在拍摄的过程中在眼球一侧放置刻度尺，对照片进行边缘检测等处理，结合刻度尺上的刻度从而计算出瞳孔的直径。但是，这种方式测量耗费的时间长，容易造成人的主观误差，因此，测量的效率低，测量的结果不准确。
 技术问题
本申请实施例提供一种瞳孔直径的测量方法、装置及计算机可读存储介质，可以极大的减少实验人员主观误差、测量仪器的造成的误差，同时计算简单，提升了测量的效率以及测量结果的准确性。
 技术解决方案
第一方面，本申请实施例提供了一种瞳孔直径的测量方法，该方法包括：
获取待测视频，所述待测视频中存在至少一帧包含瞳孔的图像；
使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，所述第一瞳孔为所述至少一帧包含的瞳孔中的任一瞳孔，所述M为大于或等于2的正整数；
根据所述位置信息计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述M个不同特征点包括所述第一瞳孔的圆心对应的特征点以及所述瞳孔圆周上的M-1个点；
所述使用所述深度卷积神经网络确定第一瞳孔的M个特征点的位置信息，包括：
将所述待测视频输入所述深度卷积神经网络，得到所述视频中的每帧所述至少一帧包含瞳孔的图像中每一个瞳孔的圆心的位置信息以及圆周上的M-1个点的位置信息，所述圆心的位置信息为所述深度卷积神经网络识别或预测出的被遮挡的圆心位置信息。
在一种可能的实现方式中，所述位置信息包括所述特征点的坐标以及所述坐标对应的置信概率；
所述根据所述位置信息计算所述第一瞳孔的直径，包括：
判断所述第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，所述第一特征点为所述第一瞳孔的任一特征点；
在判断出所述第一特征点坐标对应的置信概率大于或等于所述阈值的情况下，将所述第一特征点确定为可用特征点，得到所述第一瞳孔对应的多个可用特征点；
根据所述多个可用特征点的坐标计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述根据所述位置信息计算所述第一瞳孔的直径，还包括：
在判断出所述第一瞳孔的第一特征点坐标对应的置信概率小于所述阈值的情况下，将所述待测视频中与所述第一瞳孔所在图像相邻的前一帧图像中所述第一特征点坐标对应的可用位置信息，确定为所述第一瞳孔在所述第一瞳孔所在图像中对应的可用特征点的位置信息。
在一种可能的实现方式中，所述根据所述多个可用特征点的坐标计算所述第一瞳孔的直径，包括：
根据特征点数量与直径计算方法的对应关系获取所述多个可用特征点对应的计算方法，得到第一计算方法；
根据所述多个可用特征点的坐标以及所述第一计算方法计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述方法还包括：
根据所述第一瞳孔在第一图像中对应的位置信息，在所述第一图像中标注所述第一瞳孔，所述第一图像为所述待测视频中包含所述第一瞳孔的图像中的任一图像。
在一种可能的实现方式中，所述方法还包括：
根据所述第一瞳孔在所述第一图像中的直径，绘制所述第一瞳孔的直径的变化曲线。
第二方面，本申请实施例提供了一种瞳孔直径的测量装置，包括：
获取单元，用于获取待测视频，所述待测视频中存在至少一帧包含瞳孔的图像；
确定单元，用于使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，所述第一瞳孔为所述至少一帧包含的瞳孔中的任一瞳孔，所述M为大于或等于2的正整数；
计算单元，用于根据所述位置信息计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述M个不同特征点包括所述第一瞳孔的圆心对应的特征点以及所述瞳孔圆周上的M-1个点；
所述确定单元，具体用于：
将所述待测视频输入所述深度卷积神经网络，得到所述视频中的每帧所述至少一帧包含瞳孔的图像中每一个瞳孔的圆心的位置信息以及圆周上的M-1个点的位置信息，所述圆心的位置信息为所述深度卷积神经网络识别或预测出的被遮挡的圆心位置信息。
在一种可能的实现方式中，所述位置信息包括所述特征点的坐标以及所述坐标对应的置信概率；
所述计算单元，具体用于：
判断所述第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，所述第一特征点为所述第一瞳孔的任一特征点；
在判断出所述第一特征点坐标对应的置信概率大于或等于所述阈值的情况下，将所述第一特征点确定为可用特征点，得到所述第一瞳孔对应的多个可用特征点；
根据所述多个可用特征点的坐标计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述计算单元，还用于：
在判断出所述第一瞳孔的第一特征点坐标对应的置信概率小于所述阈值的情况下，将所述待测视频中与所述第一瞳孔所在图像相邻的前一帧图像中所述第一特征点坐标对应的可用位置信息，确定为所述第一瞳孔在所述第一瞳孔所在图像中对应的可用特征点的位置信息。
在一种可能的实现方式中，所述计算单元，具体用于：
根据特征点数量与直径计算方法的对应关系获取所述多个可用特征点对应的计算方法，得到第一计算方法；
根据所述多个可用特征点的坐标以及所述第一计算方法计算所述第一瞳孔的直径。
在一种可能的实现方式中，所述装置还包括：
标注单元，用于根据所述第一瞳孔在第一图像中对应的位置信息，在所述第一图像中标注所述第一瞳孔，所述第一图像为所述待测视频中包含所述第一瞳孔的图像中的任一图像。
在一种可能的实现方式中，所述装置还包括：
绘制单元，用于根据所述第一瞳孔在所述第一图像中的直径，绘制所述第一瞳孔的直径的变化曲线。
第三方面，本申请实施例提供了一种电子设备，该电子设备包括处理器和存储器，该处理器和存储器相互连接。该存储器用于存储支持该终端设备执行上述第一方面和/或第一方面任一种可能的实现方式提供的方法的计算机程序，该计算机程序包括程序指令，该处理器被配置用于调用上述程序指令，执行上述第一方面和/或第一方面任一种可能的实现方式所提供的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序包括程序指令，该程序指令当被处理器执行时使该处理器执行上述第一方面和/或第一方面任一种可能的实现方式所提供的方法。
 有益效果
本申请实施例通过将包含至少一帧瞳孔图像的待测视频输入深度卷积神经网络得到图像中的瞳孔对应的特征点的位置信息，根据瞳孔对应的特征点的位置信息可以计算得到瞳孔的直径，通过将待测的视频输入网络中得到特征点位置信息可以极大的减少实验人员主观误差、测量仪器的造成的误差，提升了测量结果的准确性。同时通过位置信息计算直径，计算简单，提升了测量的效率。
 附图说明
为了更清楚地说明本申请实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍。
图1是本申请实施例提供的一种瞳孔直径的测量方法的流程示意图；
图2是本申请实施例提供的另一种瞳孔直径的测量方法流程示意图；
图3是本申请实施例提供的一种瞳孔直径的测量装置的结构示意图；
图4是本申请实施例提供的电子设备的结构示意图；
图5是本申请实施例提供的未被遮挡瞳孔特征点位置标记的示意图；
图6是本申请实施例提供的被部分遮挡瞳孔特征点位置标记的示意图；
图7是本申请实施例提供的连续两帧眼动的特征点位置标记的示意图；
图8是本申请实施例提供的未被遮挡瞳孔特征点拟合圆后标记瞳孔的示意图；
图9是本申请实施例提供的被部分遮挡瞳孔特征点拟合圆后标记瞳孔的示意图。
 本发明的最佳实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”和“包含”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/ 或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当... 时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
请参阅图1，图1是本申请实施例提供的一种瞳孔直径的测量方法的流程示意图。如图1所示，该瞳孔直径的测量方法包括：
101、获取待测视频。
在一种可能的实现方式中，本申请实施例中的瞳孔直径的测量装置可以包括平板电脑、个人数字助理（Personal Digital Assistant，PDA）、个人电脑、移动互联网设备（Mobile Internet Device，MID）、等包括不限于上述提到的各种能够调用执行软件程序的电子设备。
其中，获取待测视频为存在至少一帧包含瞳孔图像的视频，具体地，获取待测视频的装置可以集成在上述瞳孔直径的测量装置中，通过摄像头等装置可以采集视频得到待测视频，也可以外接采集视频的终端设备，通过外接的终端设备获取待测视频，还可以是通过无线局域网或者互联网通过数据传输获取到的待测视频。
可选地，上述待测视频若是为一帧的话，则为一幅包括瞳孔的图像，图像的获取可以是装置采集到的，也可以是接收到的图像，还可以是在采集到或者接收到的视频中截取的图像，这里不做限定。
上述待测视频可以是多种类型的包含瞳孔图像的视频，可以是包含有被部分遮挡住的瞳孔眼动视频，也可以是未被遮挡的瞳孔眼动视频，还可以是移动的瞳孔视频，以及固定眼动的瞳孔视频，也可以是上述视频的结合，这里不做限定。并且上述瞳孔可以是人的瞳孔也可以是动物的瞳孔，这里也不做限定。本申请设计的瞳孔测量模型能够有效的测量人或动物在多种场景下的瞳孔直径，适用范围较广，能够满足各种场景下测量的需求。
在本申请实施例中，将获取到的视频输入训练后的深度卷积神经网络中进行预测和识别，得到待测视频中瞳孔特征点的位置信息，通过位置信息计算可得瞳孔的直径，待测视频中出现的瞳孔的每一段时间的视频可以得到瞳孔的直径。
102、使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息。
在一种可能的实现方式中，上述深度卷积神经网络可以是深度卷积网络和残差网络结合产生的变体网络，也可以是特定的目标检测网络，这里不做限定。本申请以深度卷积神经网络和残差网络结合产生的变体网络为例进行讲解，使用残差网络不仅能够提升网络效率，在卷积神经网络的层数过多的情况下，也能有效的避免梯度消失或者梯度爆炸的优点。将步骤101获取到的包含瞳孔视频或者瞳孔图像输入到深度卷积神经网络和残差网络结合产生的变体网络中，可以获取到视频中识别到的视频中的一个或多个瞳孔分别的多个特征点位置信息。其中，特征点包括圆心以及圆周上的点，其中，根据训练时标注的圆周上的特征点数量以及待测视频中某一帧被遮挡程度进行输出，则实际输出的特征点数量的方式与训练时标注方式一致。具体地，可以按照角度进行标注，例如，可以按照以圆心为基准标注瞳孔的0度、45度、90度、135度、180度、225度、270度、315度、360度这9个标记方式中可以选一个特征点位置的标记方式（例如在0度位置的特征点、或者选择180度位置的特征点，这里可以根据实际视频中瞳孔被遮挡程度进行标记），也可以选择4个特征点位置的标注方式，还可以选择9个特征点位置的标记方式，这里不做限定，可以理解的是，获取到的特征点的数量不同，计算瞳孔直径的方法也不同。
在一种可能的实现方式中，在此之前，需要将深度神经网络与残差网络结合得到的变体网络对模型进行训练。其中，获取训练的图像集的方法有很多种，例如，可以获取包含标记了瞳孔的训练视频，首先，将训练视频进行镜头分割，得到该视频中包含的多帧图像，再通过聚类的方法，可以是通过k均值（k-means）聚类算法，将得到的每一帧图像按照彩色直方图进行聚类，得到多个类别下的图像，即每个类别代表瞳孔在不同状态下的图像。再从每个类别中抽取一定数量的图像进行标注并输入到上述深度神经网络与残差网络结合的变体网络中进行训练，则可以得到训练后的网络模型，则是本申请需要用到的模型。
其中，上述k-means聚类可以是首先将以训练视频中按照播放顺序获取到的第一帧图像彩色直方图（R，G，B直方图）为质心，获取到第一帧图像的R，G，B三幅图像的直方图，分别为1*256的矩阵；对比第二帧图像与第一帧图像为初始质心，即比较两个图像的RGB三种质心的距离，可以理解的是，这里预设了一个距离阈值，在判断出两个图像的三种质心的距离小于距离阈值的情况下，第二帧加入第一帧的聚类，同理，第三帧图像以及第四帧图像与初试质心比较在判断出距离大于距离阈值的情况下，生成新的质心，若不类似，则生成新的聚类。下次比较时新的帧图像将会与所有的聚类的质心进行比较，选择归属的聚类或者生成新的聚类。如此往复，将会生成一个或多个聚类，同时训练视频中的每一帧图像都会有所归属。上述按照直方图特征聚类仅为举例，实际可以根据图像中的人为划分的瞳孔状态进行聚类，得到各个状态下瞳孔的图像，还可以根据图像中的瞳孔其他特征进行聚类，这里不做限定。
进一步地，每个聚类代表瞳孔的不同状态，从每个聚类的图像中分别抽取一定数量的图像，按照一定比例分为训练集和验证集，其中比例可以是7:3，也可以是8:2，在划分出的训练集图像中标记，也可以将聚类得到的图像分为训练集、验证集以及测试集，具体的比例可以人为设置，这里也不做限定。
其中，可以有几种标记方式可以选择，其中可以标记圆心特征点和圆周上0度的特征点位置，也可以选择圆心特征点以及瞳孔0度方向，45度方向，-45度方向4个特征点位置标记，还可以是0度、45度、90度、135度、180度、225度、270度、315度、360度这9个特征点位置标记，可以理解的是，在训练的时候标记的特征点个数对应测量时输出的特征点个数。将标记好的图像输入上述深度卷积神经网络以及残差网络组合的变体网络中进行训练，在训练的过程中不断调整网络超参数，比如学习率，迭代次数，训练轮数，网络层数，卷积核大小，mini-batch等，训练过程中可以通过迭代的方式来调整权重，从而使得该网络能够准确识别、预测还有追踪瞳孔特征点的位置信息。
在一种可能的实现方式中，在训练完成之后，将上述训练好的模型在测试集上进行测试，如果测试集和训练集正确率相差不大，则说明训练出的模型比较稳定，泛化能力比较强。反之，说明模型不可用，需要分析查找原因，可以通过学习曲线，以及泛化能力的表现，判断模型遇到的问题，并采取相应方法加以解决。例如，遇到问题可能是模型的欠拟合和过拟合，如果模型发生过拟合，则可以通过增加数据量的方式进行调整，也可以通过使用批量归一化的方式进行调整，还可以通过降低网络复杂度等的方式进行调节。如果模型发生欠拟合，则可以向模型中添加特征项、使得模型变得更加复杂、减小正则化参数等方法进行调节。在调整之后，把训练集重新丢入深度卷积网络和残差网络结合产生的变体网络中，进行网络模型训练，直到训练出泛化能力好的模型为止。通过深度卷积神经网络以及残差网络的结合，既能够准确识别出瞳孔的特征点，又能够避免梯度消失。其中，该模型的程序可以打包为一个后缀为.py的文件，使得该模型可以在多个不同的系统上运行，增加了该方法的应用范围。例如，可以在Linux系统上运行，也可以在Windows系统上运行。提高了本方案的可移植性，满足多种使用者对系统的不同需求。
进一步地，可以根据待测的视频中瞳孔的类型，例如人的瞳孔视频与小老鼠的瞳孔视频以及小狗的瞳孔视频差异会比较大，则可以重新获取图像通过网络训练新的模型，利用多种模型分别进行检测和追踪瞳孔直径变化，以确保达到最好的效果，同时本申请提出的模型训练和调参也比较方便，从而大大提高了该模型的适用性。
在一种可能的实现方式中，将待测视频输入到该网络中，通过结合对象识别以及语义分割算法这两个关键成分，能够识别瞳孔的特征点，同时，因为训练网络模型的时候通过神经网络获取到瞳孔的位置信息、空间特征信息、直方图特征信息以及RGB特征信息对模型进行训练，模型可以根据视频中的瞳孔的这些特征预测出瞳孔的圆心位置信息。其中，如果视频样本量较小，遮挡情况不严重，可以采用标记瞳孔圆心、0度、45度、90度、135度、180度、225度、270度、315度、360度这9个特征点的标记方式，这种方式对计算机性能有一定的要求，但是能够得到精确度较高的瞳孔直径。
在一种可能的实现方式中，上述输入视频可以是一次输入一个视频，也可以一次性输入多个视频，输入的一个视频，则可以得到该视频中瞳孔的直径文件，若输入的多个视频，则可以分别得到每一个视频对应的瞳孔直径文件，其中，输入的视频可以是手动传入视频的文件，也可以是输入的视频文件的路径，将该路径下的视频文件都输入至训练好的模型。
进一步地，可以根据文件命名方式等方法进行判断多个视频是否可合并为同一视频，当判断出多个视频为同一个视频分割为成的多个视频时，可以将上述多个视频的瞳孔输出为一个瞳孔直径文件，这里对输出文件的数量和方式不做限定。
103、根据上述位置信息计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述位置信息包括特征点的坐标以及每一个坐标对应的置信概率。根据特征点的个数，特征点为圆心以及圆周上的一个或多个点。其中，特征点的个数不同，可以采用的计算的方法也不同。特征点的个数与计算方法有一个对应关系，在确定了特征点的标记方式之后，可以根据特征点的个数确定瞳孔直径的计算方法，并将上述得到的特征点位置信息的文件输入到瞳孔直径的计算程序中，可以是输入到写好程序的matlab中直接运行，这里不做限定。可以理解的是，在计算得到了瞳孔直径后，可以将该瞳孔直径以文件的形式输出，也可以以可视化的界面输出，这里不做限定。
本申请实施例通过将包含至少一帧瞳孔图像的待测视频输入深度卷积神经网络得到图像中的瞳孔对应的特征点的位置信息，根据瞳孔对应的特征点的位置信息可以计算得到瞳孔的直径，通过将待测的视频输入网络中得到特征点位置信息可以极大的减少实验人员主观误差、测量仪器的造成的误差，提升了测量结果的准确性。同时通过位置信息计算直径，计算简单，提升了测量的效率。
请参阅图2，图2是本申请实施例提供的另一种瞳孔直径的测量方法的流程示意图。如图2所示，该瞳孔直径的测量方法包括：
201、判断第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，在判断出第一瞳孔的第一特征点坐标对应的置信概率大于或等于阈值的情况下，执行步骤202。
在一种可能的实现方式中，得到的瞳孔的位置信息包括瞳孔的特征点坐标以及每一个坐标对应的置信概率。其中第一瞳孔为待测视频中至少一帧包含瞳孔的图像中的任一瞳孔。其中，瞳孔直径的计算方法可以包括以下步骤：首先，根据置信概率剔除特征坐标离群点，然后选取置信概率大于阈值的坐标特征点出来，即判断上述每一帧图像中的每一个瞳孔特征点坐标对应的置信概率是否大于或等于阈值。其中，上述阈值可以是0.9，也可以是0.93，还可以是0.95，这个可以是技术人员设置的概率阈值，在这里不做限定。
202、将上述第一特征点确定为可用特征点，得到上述第一瞳孔对应的多个可用特征点。
在一种可能的实现方式中，根据瞳孔的被遮挡程度确定特征点的数量，若是遮挡比较严重，可以采用只标记瞳孔圆心和瞳孔正上方0度位置。如果视频数据量小，瞳孔遮挡情况不严重，可以采用标记瞳孔圆心，瞳孔0度方向，45度方向，-45度方向4个特征点，如果视频样本量较小，遮挡情况不严重，可以采用标记瞳孔圆心、0度、45度、90度、135度、180度、225度、270度、315度、360度这9个特征点的标记方式。在得到特征点的位置信息后，通过对位置信息中的置信概率进行判断，在判断出每个特征点对应的置信概率大于或等于阈值的情况下，则将该特征点确定为可用特征点，即在接下来计算瞳孔直径的时候通过可用特征点用于计算对应瞳孔的直径。若判断出该特征点对应的置信概率小于阈值时，根据特征点的个数，处理方式不同：在特征点的个数为2的情况下，即只标记了瞳孔圆心和瞳孔正上方0度位置，当确定有一个或一个以上的特征点置信概率小于上述阈值时，则用上一帧图像中瞳孔的直径确定为当前帧图像瞳孔对应的直径；在特征点的个数大于3的情况下，当判断出有一个或多个特征点的置信概率小于上述阈值时，则确定上述特征点为不可用特征点，即在计算瞳孔直径时，舍弃该点不用。
在一种可能的实现方式中，当该帧图像中的可用特征点的个数小于2的情况下，将该图像相邻的前一帧图像中所述第一特征点坐标对应的可用位置信息，确定为所述第一瞳孔在所述第一瞳孔所在图像中对应的可用特征点的位置信息，即将按照待测视频播放顺序的前一帧计算得到的瞳孔直径作为该帧图像的瞳孔直径。
在上述存在至少两个可用特征点的情况下，可以采用对应的计算方法计算出瞳孔的直径。
203、根据上述多个可用特征点的坐标计算上述第一瞳孔的直径。
在一种可能的实现方式中，特征点个数与瞳孔直径的计算方法有一个对应关系，可以是根据可用特征点的个数与计算方法对应关系获取到的计算直径的方法。
具体地，在可用特征点个数为2，即一个圆心特征点以及圆周上只有一个特征点的情况下，可以通过欧式距离计算两点之间的距离，得到的距离即为该瞳孔对应的直径。在可用特征点个数大于2的情况下，即一个圆心特征点以及圆周上识别到至少两个特征点最多9个特征点的情况下，可以还是通过欧式距离的计算方法计算得到多个圆心与圆周上的多个特征点之间的距离，得到多个距离，通过求平均的方式，得到该图像对应的瞳孔直径。在圆周上的可用特征点的个数至少为3时，即一个圆心特征点以及圆周上至少有3个可用特征点，可以选取其中3个圆周上的特征点可以按照三点拟合圆的方法计算瞳孔的直径，也可以分别计算圆心与圆周上的可用特征点来计算平均直径。
本申请中的计算方法简单高效，使得大大缩短了瞳孔直径的计算时间，提高了瞳孔直径的测量效率。
进一步地，对计算得到的瞳孔的直径进行去噪，可以采用滑窗函数去噪，也可以采用其他方式去噪，这里不做限定。
在一种可能的实现方式中，根据瞳孔在待测视频中某一帧图像中的位置信息，可以在对应的图像中标注出特征点的位置信息。如图5所示，图5为根据特征点位置标记后的图像，可以将特征点以预设的颜色以及标记方式标记出来，可以是标记每一个特征点的位置，图5为没有任何遮挡情况下的标记方式。如图6瞳孔被部分遮挡的标记方式，以及图7为连续两帧图像中眼睛瞳孔移动后分别标记的标记方式。
进一步地，得到了瞳孔的直径以及多个特征点的位置信息，可以在可视化的界面中进行验证，可以根据模型输出的标记好的视频进行直观的观察是否标记的准确进行验证，也可以通过计算出的瞳孔直径以及瞳孔的特征点坐标拟合一个圆，标记在图像中对应的瞳孔上，如图8所示，可以在原包含瞳孔的图像中绘制圆形，通过直观的观察进行验证。如图9所示，在瞳孔被部分遮挡时，也能够通过已有的特征点坐标以及直径绘制在图像上进行验证。还可以将待测视频中每一帧的瞳孔直径按照视频的播放顺序绘制有关时间和瞳孔直径的变化曲线进行验证，其中，瞳孔直径变化曲线可以是待测视频中的瞳孔直径的变化曲线，也可以是将其他方式测量得到的待测视频中的瞳孔直径与本申请测量得到的待测视频的瞳孔直径对比起来画在直径变化曲线中。上述标记的图像以及瞳孔直径变化曲线都可以显示在上述可视化界面中，可以只显示一种验证的方式，也可以显示几种验证的结合，这里不做限定。
本申请实施例通过将包含至少一帧瞳孔图像的待测视频输入深度卷积神经网络得到图像中的瞳孔对应的特征点的位置信息，根据瞳孔对应的特征点的位置信息可以计算得到瞳孔的直径，通过将待测的视频输入网络中得到特征点位置信息可以极大的减少实验人员主观误差、测量仪器的造成的误差，提升了测量结果的准确性。同时通过位置信息计算直径，计算简单，提升了测量的效率。
请参阅图3，图3是本申请实施例提供的一种瞳孔直径的测量装置的结构示意图。如图3所示，该瞳孔直径的测量装置3000包括：
获取单元301，用于获取待测视频，上述待测视频中存在至少一帧包含瞳孔的图像；
确定单元302，用于使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，上述第一瞳孔为上述至少一帧包含的瞳孔中的任一瞳孔，上述M为大于或等于2的正整数；
计算单元303，用于根据上述位置信息计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述M个不同特征点包括上述第一瞳孔的圆心对应的特征点以及上述瞳孔圆周上的M-1个点；
上述确定单元302，具体用于：
将上述待测视频输入上述深度卷积神经网络，得到上述视频中的每帧上述至少一帧包含瞳孔的图像中每一个瞳孔的圆心的位置信息以及圆周上的M-1个点的位置信息，上述圆心的位置信息为上述深度卷积神经网络识别或预测出的被遮挡的圆心位置信息。
在一种可能的实现方式中，上述位置信息包括上述特征点的坐标以及上述坐标对应的置信概率；
上述计算单元303，具体用于：
判断上述第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，上述第一特征点为上述第一瞳孔的任一特征点；
在判断出上述第一特征点坐标对应的置信概率大于或等于上述阈值的情况下，将上述第一特征点确定为可用特征点，得到上述第一瞳孔对应的多个可用特征点；
根据上述多个可用特征点的坐标计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述计算单元303，还用于：
在判断出上述第一瞳孔的第一特征点坐标对应的置信概率小于上述阈值的情况下，将上述待测视频中与上述第一瞳孔所在图像相邻的前一帧图像中上述第一特征点坐标对应的可用位置信息，确定为上述第一瞳孔在上述第一瞳孔所在图像中对应的可用特征点的位置信息。
在一种可能的实现方式中，上述计算单元303，具体用于：
根据特征点数量与直径计算方法的对应关系获取上述多个可用特征点对应的计算方法，得到第一计算方法；
根据上述多个可用特征点的坐标以及上述第一计算方法计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述装置3000还包括：
标注单元304，用于根据上述第一瞳孔在第一图像中对应的位置信息，在上述第一图像中标注上述第一瞳孔，上述第一图像为上述待测视频中包含上述第一瞳孔的图像中的任一图像。
在一种可能的实现方式中，上述装置3000还包括：
绘制单元305，用于根据上述第一瞳孔在上述第一图像中的直径，绘制上述第一瞳孔的直径的变化曲线。
本申请实施例的瞳孔直径的测量装置通过将包含至少一帧瞳孔图像的待测视频输入深度卷积神经网络得到图像中的瞳孔对应的特征点的位置信息，根据瞳孔对应的特征点的位置信息可以计算得到瞳孔的直径，通过将待测的视频输入网络中得到特征点位置信息可以极大的减少实验人员主观误差、测量仪器的造成的误差，提升了测量结果的准确性。同时通过位置信息计算直径，计算简单，提升了测量的效率。
可以理解的是，本实施例的服务器的获取单元301、确定单元302、计算单元303、标注单元304、绘制单元305的功能可以根据上述方法实施例中的方法具体实现，其具体实现过程可以参照上述方法实施例的相关描述，此处不再赘述。
请参阅图4，图4是本申请实施例提供的电子设备的结构示意图。如图4所示，本实施例中的电子设备可以包括：一个或多个处理器401、输入设备402、输出设备403和存储器404。上述处理器401、输入设备402、输出设备403和存储器402通过总线连接。存储器402用于存储计算机程序，该计算机程序包括程序指令，处理器401用于执行存储器402存储的程序指令，输入设备402用于输入数据，输出设备403用于输出数据。其中，上述处理器401被配置用于调用程序指令执行以下步骤：
获取待测视频，上述待测视频中存在至少一帧包含瞳孔的图像；
使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，上述第一瞳孔为上述至少一帧包含的瞳孔中的任一瞳孔，上述M为大于或等于2的正整数；
根据上述位置信息计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述M个不同特征点包括上述第一瞳孔的圆心对应的特征点以及上述瞳孔圆周上的M-1个点；
上述处理器401使用上述深度卷积神经网络确定第一瞳孔的M个特征点的位置信息，包括：
将上述待测视频输入上述深度卷积神经网络，得到上述视频中的每帧上述至少一帧包含瞳孔的图像中每一个瞳孔的圆心的位置信息以及圆周上的M-1个点的位置信息，上述圆心的位置信息为上述深度卷积神经网络识别或预测出的被遮挡的圆心位置信息。
在一种可能的实现方式中，上述位置信息包括上述特征点的坐标以及上述坐标对应的置信概率；
上述处理器401根据上述位置信息计算上述第一瞳孔的直径，包括：
判断上述第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，上述第一特征点为上述第一瞳孔的任一特征点；
在判断出上述第一特征点坐标对应的置信概率大于或等于上述阈值的情况下，将上述第一特征点确定为可用特征点，得到上述第一瞳孔对应的多个可用特征点；
根据上述多个可用特征点的坐标计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述处理器401根据上述位置信息计算上述第一瞳孔的直径，还包括：
在判断出上述第一瞳孔的第一特征点坐标对应的置信概率小于上述阈值的情况下，将上述待测视频中与上述第一瞳孔所在图像相邻的前一帧图像中上述第一特征点坐标对应的可用位置信息，确定为上述第一瞳孔在上述第一瞳孔所在图像中对应的可用特征点的位置信息。
在一种可能的实现方式中，上述处理器401根据上述多个可用特征点的坐标计算上述第一瞳孔的直径，包括：
根据特征点数量与直径计算方法的对应关系获取上述多个可用特征点对应的计算方法，得到第一计算方法；
根据上述多个可用特征点的坐标以及上述第一计算方法计算上述第一瞳孔的直径。
在一种可能的实现方式中，上述处理器401被配置用于调用程序指令执行以下步骤：
根据上述第一瞳孔在第一图像中对应的位置信息，在上述第一图像中标注上述第一瞳孔，上述第一图像为上述待测视频中包含上述第一瞳孔的图像中的任一图像。
在一种可能的实现方式中，上述处理器401被配置用于调用程序指令执行以下步骤：
根据上述第一瞳孔在上述第一图像中的直径，绘制上述第一瞳孔的直径的变化曲线。
应当理解，在一些可行的实施方式中，上述处理器401可以是中央处理单元 (central processing unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器 (digital signal processor，DSP)、专用集成电路 (application specific integrated circuit，ASIC)、现成可编程门阵列 (field-programmable gate array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该存储器402可以包括只读存储器和随机存取存储器，并向处理器401提供指令和数据。存储器402的一部分还可以包括非易失性随机存取存储器。例如，存储器402还可以存储设备类型的信息。
具体实现中，上述终端设备可通过其内置的各个功能模块执行如上述图1至图2中各个步骤所提供的实现方式，具体可参见上述各个步骤所提供的实现方式，在此不再赘述。
本申请实施例中的电子设备通过将包含至少一帧瞳孔图像的待测视频输入深度卷积神经网络得到图像中的瞳孔对应的特征点的位置信息，根据瞳孔对应的特征点的位置信息可以计算得到瞳孔的直径，通过将待测的视频输入网络中得到特征点位置信息可以极大的减少实验人员主观误差、测量仪器的造成的误差，提升了测量结果的准确性。同时通过位置信息计算直径，计算简单，提升了测量的效率。
在本申请的另一实施例中提供一种计算机可读存储介质，上述计算机可读存储介质存储有计算机程序，上述计算机程序被处理器执行时实现：
上述计算机可读存储介质可以是前述任一实施例上述的终端的内部存储单元，例如终端的硬盘或内存。上述计算机可读存储介质也可以是上述终端的外部存储设备，例如上述终端上配备的插接式硬盘，智能存储卡（smart media card, SMC），安全数字（secure digital, SD）卡，闪存卡（flash card）等。进一步地，上述计算机可读存储介质还可以既包括上述终端的内部存储单元也包括外部存储设备。上述计算机可读存储介质用于存储上述计算机程序以及上述终端所需的其他程序和数据。上述计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、服务器和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，上述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本申请实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
上述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备 ( 可以是个人计算机，服务器，或者网络设备等 ) 执行本申请各个实施例上述方法的全部或部分步骤。而前述的存储介质包括：U 盘、移动硬盘、只读存储器 (read-only memory，ROM)、随机存取存储器 (random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上上述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (10)

  1. 一种瞳孔直径的测量方法，其特征在于，包括：
    获取待测视频，所述待测视频中存在至少一帧包含瞳孔的图像；
    使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，所述第一瞳孔为所述至少一帧包含的瞳孔中的任一瞳孔，所述M为大于或等于2的正整数；
    根据所述位置信息计算所述第一瞳孔的直径。
  2. 根据权利要求1所述的方法，其特征在于，所述M个不同特征点包括所述第一瞳孔的圆心对应的特征点以及所述瞳孔圆周上的M-1个点；
    所述使用所述深度卷积神经网络确定第一瞳孔的M个特征点的位置信息，包括：
    将所述待测视频输入所述深度卷积神经网络，得到所述视频中的每帧所述至少一帧包含瞳孔的图像中每一个瞳孔的圆心的位置信息以及圆周上的M-1个点的位置信息，所述圆心的位置信息为所述深度卷积神经网络识别或预测出的被遮挡的圆心位置信息。
  3. 根据权利要求2所述的方法，其特征在于，所述位置信息包括所述特征点的坐标以及所述坐标对应的置信概率；
    所述根据所述位置信息计算所述第一瞳孔的直径，包括：
    判断所述第一瞳孔的第一特征点坐标对应的置信概率是否大于或等于阈值，所述第一特征点为所述第一瞳孔的任一特征点；
    在判断出所述第一特征点坐标对应的置信概率大于或等于所述阈值的情况下，将所述第一特征点确定为可用特征点，得到所述第一瞳孔对应的多个可用特征点；
    根据所述多个可用特征点的坐标计算所述第一瞳孔的直径。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述位置信息计算所述第一瞳孔的直径，还包括：
    在判断出所述第一瞳孔的第一特征点坐标对应的置信概率小于所述阈值的情况下，将所述待测视频中与所述第一瞳孔所在图像相邻的前一帧图像中所述第一特征点坐标对应的可用位置信息，确定为所述第一瞳孔在所述第一瞳孔所在图像中对应的可用特征点的位置信息。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述多个可用特征点的坐标计算所述第一瞳孔的直径，包括：
    根据特征点数量与直径计算方法的对应关系获取所述多个可用特征点对应的计算方法，得到第一计算方法；
    根据所述多个可用特征点的坐标以及所述第一计算方法计算所述第一瞳孔的直径。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述方法还包括：
    根据所述第一瞳孔在第一图像中对应的位置信息，在所述第一图像中标注所述第一瞳孔，所述第一图像为所述待测视频中包含所述第一瞳孔的图像中的任一图像。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    根据所述第一瞳孔在所述第一图像中的直径，绘制所述第一瞳孔的直径的变化曲线。
  8. 一种瞳孔直径的测量装置，包括：
    获取单元，用于获取待测视频，所述待测视频中存在至少一帧包含瞳孔的图像；
    确定单元，用于使用深度卷积神经网络确定第一瞳孔的M个不同特征点的位置信息，所述第一瞳孔为所述至少一帧包含的瞳孔中的任一瞳孔，所述M为大于或等于2的正整数；
    计算单元，用于根据所述位置信息计算所述第一瞳孔的直径。
  9. 一种电子设备，其特征在于，包括处理器、存储器、输入设备、输出设备、摄像头，所述处理器、所述存储器、所述输入设备、所述输出设备、所述摄像头相互连接，其中，所述存储器用于存储支持所述瞳孔直径的测量装置执行上述进程识别方法的计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，执行如权利要求1-7任一项所述的瞳孔直径的测量方法。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行如权利要求1-7任一项所述的瞳孔直径的测量方法。
PCT/CN2020/127902 2019-11-11 2020-11-10 一种瞳孔直径的测量方法、装置及计算机可读存储介质 WO2021093744A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911095982.6 2019-11-11
CN201911095982.6A CN110934565B (zh) 2019-11-11 2019-11-11 一种瞳孔直径的测量方法、装置及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2021093744A1 true WO2021093744A1 (zh) 2021-05-20

Family

ID=69907539

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/127902 WO2021093744A1 (zh) 2019-11-11 2020-11-10 一种瞳孔直径的测量方法、装置及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN110934565B (zh)
WO (1) WO2021093744A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110934565B (zh) * 2019-11-11 2021-11-26 中国科学院深圳先进技术研究院 一种瞳孔直径的测量方法、装置及计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1686051A (zh) * 2005-05-08 2005-10-26 上海交通大学 基于vpf和改进的susan的眼角和瞳孔的定位方法
CN101246544A (zh) * 2008-01-24 2008-08-20 电子科技大学中山学院 一种基于边界点搜索和susan边缘检测的虹膜定位方法
CN101513342A (zh) * 2009-03-26 2009-08-26 中国人民解放军第四军医大学 全视野瞳孔分析测量方法
JP2016087291A (ja) * 2014-11-10 2016-05-23 パナソニックＩｐマネジメント株式会社 瞳孔・視線計測装置および照明システム
CN108509908A (zh) * 2018-03-31 2018-09-07 天津大学 一种基于双目立体视觉的瞳孔直径实时测量方法
CN110934565A (zh) * 2019-11-11 2020-03-31 中国科学院深圳先进技术研究院 一种瞳孔直径的测量方法、装置及计算机可读存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007094906A (ja) * 2005-09-29 2007-04-12 Toshiba Corp 特徴点検出装置および方法
US7938785B2 (en) * 2007-12-27 2011-05-10 Teledyne Scientific & Imaging, Llc Fusion-based spatio-temporal feature detection for robust classification of instantaneous changes in pupil response as a correlate of cognitive response
CN103198301B (zh) * 2013-04-08 2016-12-28 北京天诚盛业科技有限公司 虹膜定位方法及装置
US9414745B2 (en) * 2014-02-05 2016-08-16 Andrew Elliott Neice Pupillometry systems, methods, and devices
CN107341447A (zh) * 2017-06-13 2017-11-10 华南理工大学 一种基于深度卷积神经网络和证据k近邻的人脸核实方法
CN109919981B (zh) * 2019-03-11 2022-08-02 南京邮电大学 一种基于卡尔曼滤波辅助的多特征融合的多目标跟踪方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1686051A (zh) * 2005-05-08 2005-10-26 上海交通大学 基于vpf和改进的susan的眼角和瞳孔的定位方法
CN101246544A (zh) * 2008-01-24 2008-08-20 电子科技大学中山学院 一种基于边界点搜索和susan边缘检测的虹膜定位方法
CN101513342A (zh) * 2009-03-26 2009-08-26 中国人民解放军第四军医大学 全视野瞳孔分析测量方法
JP2016087291A (ja) * 2014-11-10 2016-05-23 パナソニックＩｐマネジメント株式会社 瞳孔・視線計測装置および照明システム
CN108509908A (zh) * 2018-03-31 2018-09-07 天津大学 一种基于双目立体视觉的瞳孔直径实时测量方法
CN110934565A (zh) * 2019-11-11 2020-03-31 中国科学院深圳先进技术研究院 一种瞳孔直径的测量方法、装置及计算机可读存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHAO XUEPENG, MENG CHUN-NING; FENG MING-KUI; CHANG SHENG-JIANG: "Fatigue Detection Based on Cascade Convolutional Neural Network", GUANGDIANZI-JIGUANG - JOURNAL OF OPTRONICS-LASER, TIANJIN DAXUE JIDIAN FENXIAO, TIANJIN, CN, vol. 28, no. 5, 1 May 2017 (2017-05-01), CN, pages 497 - 502, XP055811983, ISSN: 1005-0086, DOI: 10.16136/j.joel.2017.05.0485 *

Also Published As

Publication number Publication date
CN110934565B (zh) 2021-11-26
CN110934565A (zh) 2020-03-31

Similar Documents

Publication Publication Date Title
JP7058373B2 (ja) 医療画像に対する病変の検出及び位置決め方法、装置、デバイス、及び記憶媒体
WO2020248387A1 (zh) 基于多路摄像的人脸识别方法、装置、终端及存储介质
WO2020155518A1 (zh) 物体检测方法、装置、计算机设备及存储介质
WO2017181899A1 (zh) 一种人脸活体验证方法及装置
CN107194361A (zh) 二维姿势检测方法及装置
CN112101124B (zh) 一种坐姿检测方法及装置
CN111444555B (zh) 一种测温信息显示方法、装置及终端设备
CN112328822B (zh) 图片预标注方法、装置及终端设备
JP2022542199A (ja) キーポイントの検出方法、装置、電子機器および記憶媒体
CN111126268A (zh) 关键点检测模型训练方法、装置、电子设备及存储介质
CN113763348A (zh) 图像质量确定方法、装置、电子设备及存储介质
CN111782529B (zh) 辅助诊断系统的测试方法、装置、计算机设备及存储介质
WO2023103285A1 (zh) 定位方法、装置、机器人及计算机可读存储介质
CN113780145A (zh) 精子形态检测方法、装置、计算机设备和存储介质
CN113450329B (zh) 一种微循环图像血管分支红细胞流速计算方法及系统
WO2022068228A1 (zh) 病灶标注的验证方法、装置、计算机设备及存储介质
CN114519401A (zh) 一种图像分类方法及装置、电子设备、存储介质
WO2021093744A1 (zh) 一种瞳孔直径的测量方法、装置及计算机可读存储介质
WO2020244076A1 (zh) 人脸识别方法、装置、电子设备及存储介质
CN114864093B (zh) 基于眼底图像进行疾病预测的装置、方法及存储介质
JP2023549865A (ja) 児童を対象とした両眼距離測定の方法及びシステム
US12026879B2 (en) Method for detecting the presence of pneumonia area in medical images of patients, detecting system, and electronic device employing method
CN111767829B (zh) 一种活体检测方法、装置、系统及存储介质
CN112784691B (zh) 一种目标检测模型训练方法、目标检测方法和装置
CN112580462A (zh) 一种特征点选择方法、终端和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20888219

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20888219

Country of ref document: EP

Kind code of ref document: A1