CN116246332B - Eyeball tracking-based data labeling quality detection method, device and medium - Google Patents
Eyeball tracking-based data labeling quality detection method, device and medium Download PDFInfo
- Publication number
- CN116246332B CN116246332B CN202310526432.5A CN202310526432A CN116246332B CN 116246332 B CN116246332 B CN 116246332B CN 202310526432 A CN202310526432 A CN 202310526432A CN 116246332 B CN116246332 B CN 116246332B
- Authority
- CN
- China
- Prior art keywords
- information
- eye
- data
- eye movement
- condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
Abstract
The application relates to the technical field of data annotation, and particularly provides a data annotation quality detection method, equipment and medium based on eyeball tracking, wherein the method comprises the following steps: acquiring eye movement information, wherein the eye movement information is change information of eye gazing positions in a primary image data labeling process; acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information; judging whether the labeling data are qualified or not according to whether the condition information meets preset conditions or not; the method can effectively improve the accuracy and the evaluation efficiency of the data labeling quality detection method and reduce the evaluation cost of the data labeling quality detection method.
Description
Technical Field
The application relates to the technical field of data annotation, in particular to a method, equipment and medium for detecting data annotation quality based on eyeball tracking.
Background
In the field of artificial intelligence and robotics, it is necessary to train a machine learning model using images (hereinafter referred to as image annotation data) subjected to human annotation data, for example, an image recognition model is necessary to train using images annotated with recognition targets. Because the quality of the image annotation data can influence the learning effect of the machine learning model, in order to improve the learning effect of the robot learning model, the quality of the image annotation data needs to be evaluated by a data annotation quality detection method in the prior art, the quality of the image annotation data is evaluated by a manual rechecking mode in the existing data annotation quality detection method, and if the quality of the image annotation data is qualified, the machine learning model is trained by the image annotation data; if the quality of the image marking data is not qualified, notifying a marking person to re-mark the image. Because the manual review is easily influenced by personal factors such as personal preference and judgment capability, namely, the auditing standards of each person are different, the conventional data labeling quality detection method does not use a unified auditing standard to evaluate the image labeling data, so that the accuracy of the data labeling quality detection method is low, and the conventional data labeling quality detection method also has the problems of low evaluation efficiency and high evaluation cost (using personnel cost) because the evaluation is required by the manual review.
In view of the above problems, no effective technical solution is currently available.
Disclosure of Invention
The purpose of the application is to provide a data labeling quality detection method, equipment and medium based on eyeball tracking, which can effectively improve the accuracy and evaluation efficiency of the data labeling quality detection method and reduce the evaluation cost of the data labeling quality detection method.
In a first aspect, the present application provides a method for detecting quality of data annotation based on eye tracking, including the steps of:
acquiring eye movement information, wherein the eye movement information is change information of eye gazing positions in a primary image data labeling process;
acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information;
judging whether the labeling data are qualified or not according to whether the condition information meets the preset condition.
According to the data labeling quality detection method based on eyeball tracking, whether the labeling data are qualified is judged by judging whether the condition information meets the preset condition or not, the method is equivalent to selecting an auditing standard corresponding to the condition information according to the condition information to detect the quality of standard data, and the auditing standard is preset and cannot be interfered by human factors, so that the accuracy of the data labeling quality detection method can be effectively improved, and the quality of the labeling data is not required to be detected in a manual rechecking mode, so that compared with the prior art, the evaluation efficiency of the data labeling quality detection method can be effectively improved, and the evaluation cost of the data labeling quality detection method can be reduced.
Optionally, the condition information includes eye movement area information, the preset condition is that an actual intersection ratio of the eye movement area information and the labeling data area information is greater than or equal to a preset intersection ratio threshold, and the labeling data area information is an area where the labeling data is located.
Optionally, the calculation formula of the actual blending ratio is:
；
wherein IoU represents the actual cross ratio, S A Representing the information of the marked data area S B Representing eye movement region information, S A ∩S B Representing the intersection of the labeling data region information and the eye movement region information, S A ∪S B And representing the union of the labeling data region information and the eye movement region information.
Optionally, the condition information includes total gazing time information, the preset condition is that the effective gazing rate is greater than or equal to a preset gazing rate threshold, the effective gazing rate is a ratio of the effective gazing time information to the total gazing time information, and the effective gazing time information is total time that the eye gazing position is located in the marked data area information in a process of marking the image data.
Optionally, the condition information includes total eye jump number information, the preset condition is that the effective eye jump rate is greater than or equal to a preset eye jump rate threshold, the effective eye jump rate is a ratio of the effective eye jump number information to the total eye jump number information, and the effective eye jump number information is a difference value between the total eye jump number information and the back vision eye jump number information.
Optionally, the number of the back-looking type hops is the number of the back-looking type hops, and if the included angle between the current direction of the back-looking type hops and the direction of the last time of the back-looking type hops is greater than 90 degrees, the current eye hops are considered to be the back-looking type hops.
Optionally, the condition information includes eye movement track length information, the preset condition is that an effective track rate is greater than or equal to a preset track rate threshold, the effective track rate is a ratio of the effective track length information to the eye movement track length information, and the effective track length information is a total length of the eye movement track in the labeling data area information in a primary image data labeling process.
Optionally, the condition information includes eye movement area information and total gaze time information, and the preset condition is that an actual intersection ratio of the eye movement area information and the labeling data area information is greater than or equal to a preset intersection ratio threshold value and the effective gaze rate is greater than or equal to a preset gaze rate threshold value.
In a second aspect, the present application provides an electronic device comprising a processor and a memory storing computer readable instructions which, when executed by the processor, perform the steps of the method as provided in the first aspect above.
In a third aspect, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs steps in a method as provided in the first aspect above
As can be seen from the above, the method, the device and the medium for detecting the quality of the data marking based on eye tracking provided by the present application determine whether the marking data is qualified by determining whether the condition information satisfies the preset condition, and the method is equivalent to selecting an audit standard corresponding to the condition information according to the condition information to detect the quality of the standard data.
Drawings
Fig. 1 is a flowchart of a method for detecting quality of data labeling based on eye tracking according to an embodiment of the present application.
Fig. 2 is a schematic diagram of gaze point, eye movement area information, and labeling data area information according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a back vision eye jump according to an embodiment of the present application.
Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Reference numerals: 101. a processor; 102. a memory; 103. a communication bus.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
In a first aspect, as shown in fig. 1-3, the present application provides a method for detecting quality of data annotation based on eye tracking, which includes the following steps:
s1, acquiring eye movement information, wherein the eye movement information is change information of an eye gazing position in a primary image data labeling process;
s2, acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information;
and S3, judging whether the labeling data are qualified or not according to whether the condition information meets the preset condition.
The process of the annotator for the image data annotation of the image comprises the following steps: the method comprises the steps of searching and determining the position of data to be marked (target objects to be marked) in an image, and marking the data to be marked. When the data to be marked is positioned and marked, the eye gazing position of the marker can be changed, and the eye gazing position can be changed only when the eyeballs move, so that the eye gazing information obtained in the step S1 is equivalent to the change information of the eye gazing position in the process of marking the image data, specifically, the step S1 can firstly collect the eye image information by utilizing a vision sensor or a camera, and then acquire the eye gazing information according to the eye image information by utilizing the existing eye tracking model or eye tracking algorithm. It should be understood that if the eye-gazing position falls within a circle range with a certain point as a center and a parameter r as a radius within a preset time (more than 80 ms), the center is the gazing point, and all the eye-gazing positions within the area are replaced by the gazing point, and since the eye-gazing information is information of changing the eye-gazing position during one image data labeling process, the information of changing the eye-gazing position during one image data labeling process can be replaced by a plurality of gazing points (refer to C in fig. 2), and thus the eye-gazing information can reflect the position information of the gazing point with respect to time change.
Since the eye movement information can reflect the position information of the gaze point with respect to time variation, step S2 may acquire the eye movement region information, the total gaze time information, the total eye jump number information, and the eye movement track length information from the eye movement information. The eye movement area information may be a pattern surrounding all the eye points generated according to all the eye points based on the existing surrounding algorithm, or the eye movement area information may be a pattern surrounding all the eye trajectories generated according to all the eye trajectories based on the existing surrounding algorithm, and in this embodiment, the eye movement area information is preferably a minimum convex polygon (refer to B in fig. 2) surrounding all the eye points or a minimum convex polygon surrounding all the eye trajectories, the total eye movement time information is a total eye movement time (total time of looking at an image) of a person marked in one image data marking process, each eye point has corresponding eye movement time information, the total eye movement time information is equal to a sum of all the eye movement time information, the total eye movement number information is a total eye movement number in one image marking process, if the eye movement point moves from the current position to the other position, one eye movement is considered to occur, that is one eye movement number corresponds to two eye points, so the total eye movement track length information=the number of eye movement positions-1, the eye movement track length information is a total eye movement track length in one image data marking process, each eye movement track length corresponds to the total length of the eye track, and the eye movement track length between the two eye movement tracks is the corresponding to the length of the eye track. It should be understood that the eye movement information and the condition information are both information determined for a display screen of the display device.
The working principle of the step S3 is as follows: in the process of labeling data of an image, the data to be labeled in the image can be accurately labeled only when the eye gazing position is mainly concentrated on the data to be labeled, and whether the eye gazing position is mainly concentrated on the data to be labeled can be judged through the condition information obtained in the step S2, so that whether the labeling data are qualified can be judged through a mode of judging whether the condition information meets preset conditions or not in the step S3. Specifically, the preset condition in the step S3 is a preset condition, the preset condition is equivalent to an audit standard of the labeling data, and if the condition information meets the preset condition, the labeling data is considered to be qualified; if the condition information does not meet the preset condition, the marking data is considered to be unqualified. It should be understood that the eye movement area information, the total eye gaze time information, the total eye jump number information and the eye movement track length information correspond to different auditing standards, that is, the eye movement area information, the total eye gaze time information, the total eye jump number information and the eye movement track length information correspond to different preset conditions, respectively, so that if the information included in the condition information changes, the preset condition corresponding to the condition information also changes.
The working principle of the embodiment is as follows: the method judges whether the marked data is qualified or not by judging whether the condition information meets the preset condition, the method is equivalent to selecting an auditing standard corresponding to the condition information according to the condition information to detect the quality of the standard data, and the auditing standard is preset and is not interfered by human factors, so that the accuracy of the method for detecting the marked quality of the data can be effectively improved, and the method does not need to detect the quality of the marked data in a manual rechecking mode, so that compared with the prior art, the method can effectively improve the evaluation efficiency of the method for detecting the marked quality of the data and reduce the evaluation cost of the method for detecting the marked quality of the data.
In some embodiments, the condition information includes eye movement region information, the preset condition is that an actual intersection ratio of the eye movement region information and labeling data region information (refer to a in fig. 2) is equal to or greater than a preset intersection ratio threshold, and the labeling data region information is a region where the labeling data is located. The working principle of the embodiment is as follows: because the data to be marked in the image can be accurately marked only when the eye gazing position is mainly concentrated on the data to be marked, the higher the coincidence degree of the eye movement area information and the marking data area information is, the more concentrated the eye gazing position is on the data to be marked, the embodiment can judge whether the marking data are qualified or not by calculating the actual coincidence ratio and comparing the actual coincidence ratio with a coincidence ratio threshold value, and particularly, if the actual coincidence ratio is larger than or equal to the coincidence ratio threshold value, the marking data are considered to be qualified; and if the actual cross-over ratio is smaller than the cross-over ratio threshold, the marked data is considered to be unqualified. The cross ratio threshold value of the embodiment is a preset value, and a person skilled in the art can adjust the cross ratio threshold value according to actual needs, the cross ratio threshold value of the embodiment is preferably 0.8, and the embodiment is equivalent to judging whether the labeling data is qualified according to the coincidence degree of the eye movement area information and the labeling data area information.
Preferably, in some embodiments, the eye movement region information is region information of eye movement information between marking a first mark point and a last mark point. The embodiment is equivalent to starting to acquire eye movement information when marking data to be marked is started and stopping acquiring eye movement information when marking the data to be marked is ended, namely the embodiment sets corresponding time nodes for starting to acquire eye movement information and stopping to acquire eye movement information respectively, so that the embodiment can effectively remove useless eye movement information generated by a marker looking for the data to be marked after opening an image and useless eye movement information generated by the marker finishing marking after finishing marking, and the problem that the useless eye movement information influences the eye movement area information is solved by removing the useless eye movement information because the useless eye movement information can generate invalid gazing points and invalid eye movement tracks, and the eye movement area information of the embodiment is the smallest convex polygon surrounding all gazing points or surrounding all eye movement tracks, namely the eye movement area information of the embodiment can surround the invalid gazing points or invalid eye movement tracks. It should be appreciated that since this embodiment removes unwanted eye movement information, this embodiment is also effective to avoid situations where image annotation data is misjudged as unacceptable due to too much unwanted eye movement information.
In some embodiments, the actual overlap ratio is calculated as:
；
wherein IoU represents the actual cross ratio, S A Representing the information of the marked data area S B Representing eye movement region information, S A ∩S B Representing the intersection of the labeling data region information and the eye movement region information, S A ∪S B And representing the union of the labeling data region information and the eye movement region information.
In some embodiments, the condition information includes total gaze time information, the preset condition is that an effective gaze rate is equal to or greater than a preset gaze rate threshold, the effective gaze rate is a ratio of the effective gaze time information to the total gaze time information, and the effective gaze time information is a total time when the eye gaze position is located in the labeling data region information in a single image data labeling process. The effective fixation time of this embodiment corresponds to the sum of fixation time information corresponding to the fixation positions of the eyes located in the labeling data region information. The working principle of the embodiment is as follows: because the data to be marked in the image can be accurately marked only when the eye gazing position is mainly concentrated on the data to be marked, and when the eye gazing position is positioned in the marking data area information, the gazing time information corresponding to the eye gazing position is effective gazing time, so that the larger the duty ratio of the effective gazing time information in the total gazing time information is, the more concentrated the eye gazing position is on the data to be marked, the embodiment can judge whether the marking data are qualified or not by calculating the effective gazing rate and comparing the effective gazing rate with the gazing rate threshold value, and particularly, if the effective gazing rate is larger than or equal to the gazing rate threshold value, the marking data are considered to be qualified; and if the effective gazing rate is smaller than the gazing rate threshold, the marked data is considered to be unqualified. The gaze rate threshold value of this embodiment is a preset value, and a person skilled in the art can adjust the magnitude of the gaze rate threshold value according to actual needs, and the gaze rate threshold value of this embodiment is preferably 0.9.
In some embodiments, the condition information includes total number of hops information, the preset condition is that an effective number of hops is equal to or greater than a preset threshold of hops, the effective number of hops is a ratio of the effective number of hops information to the total number of hops information, and the effective number of hops information is a difference between the total number of hops information and the back-looking number of hops information. The working principle of the embodiment is as follows: because the back-looking type eye hops are generated for looking back at the content (for example, the image comprises a plurality of pieces of data to be marked, after the marking of part of the data to be marked is finished, the back-looking type eye hops have no meaning on the marking of the image data, and the larger the proportion of the back-looking type eye hops in the total eye hops quantity information is, the less attention is paid to the data to be marked in the marked image by a marker, so that the embodiment can judge whether the marked data are qualified or not by calculating the effective eye hops and comparing the effective eye hops with the eye hops threshold value, and particularly, if the effective eye hops are larger than or equal to the eye hops threshold value, the marked data are considered to be qualified; and if the effective eye jump rate is smaller than the eye jump rate threshold value, the marked data is considered to be unqualified. The threshold value of the eye jump rate in this embodiment is a preset value, and a person skilled in the art can adjust the size of the threshold value of the eye jump rate according to actual needs.
In some embodiments, the number of back-looking type hops information is the number of back-looking type hops, and if the angle between the current direction of the current hop and the direction of the last hop is greater than 90 °, the current hop is considered to be the back-looking type hop. As shown in fig. 3, the arrow in fig. 3 is the direction of the eye jump, D in fig. 3 is the current eye jump, and E in fig. 3 is the last eye jump, and since the angle between the direction of the current eye jump and the direction of the last eye jump is greater than 90 °, the current eye jump is considered as a back vision eye jump.
In some embodiments, the condition information includes eye movement track length information, the preset condition is that an effective track rate is greater than or equal to a preset track rate threshold, the effective track rate is a ratio of the effective track length information to the eye movement track length information, and the effective track length information is a total length of the eye movement track in the labeling data region information in a single image data labeling process. The working principle of the embodiment is as follows: because the data to be marked in the image can be accurately marked only when the eye gazing position is mainly concentrated on the data to be marked, and when the eye gazing position is concentrated on the data to be marked, the eye movement track generated by eye jump is also positioned in the marking data area information, if the eye gazing position is mainly concentrated on the data to be marked, the occupation ratio of the effective track length information in the eye movement track length information is large, therefore, the embodiment can judge whether the marking data is qualified or not by calculating the effective track rate and comparing the effective track rate with the track rate threshold value, and particularly, if the effective track rate is larger than or equal to the track rate threshold value, the marking data is considered to be qualified; and if the effective track rate is smaller than the track rate threshold value, the marked data is considered to be unqualified. The track rate threshold value of this embodiment is a preset value, and a person skilled in the art can adjust the size of the track rate threshold value according to actual needs, and the track rate threshold value of this embodiment is preferably 0.9.
In some embodiments, the condition information includes eye movement region information and total gaze time information, and the preset condition is that an actual intersection ratio of the eye movement region information and the labeling data region information is equal to or greater than a preset intersection ratio threshold and the effective gaze rate is equal to or greater than a preset gaze rate threshold. Since the condition information of this embodiment includes the eye movement region information and the total injection time information, the preset condition of this embodiment needs to include the condition corresponding to the eye movement region information and the condition corresponding to the total injection time information.
In some embodiments, the condition information includes eye movement area information, total gaze time information, and total eye jump number information, and the preset condition is that the actual intersection ratio is equal to or greater than a preset intersection ratio threshold, the effective gaze rate is equal to or greater than a preset gaze rate threshold, and the effective eye jump rate is equal to or greater than a preset eye jump rate threshold.
In some embodiments, the condition information includes a plurality of the eye movement region information, the total eye-gaze time information, the total eye-gaze number information, and the eye movement trace length information, and the preset condition corresponding to the condition information includes a plurality of the condition corresponding to the eye movement region information, the condition corresponding to the total eye-gaze time information, the condition corresponding to the total eye-gaze number information, and the condition corresponding to the eye movement trace length information, and the number of the information included in the condition information is equal to the number of the conditions included in the preset condition.
In some embodiments, the condition information includes eye movement area information, total eye gaze time information, total eye jump number information, and eye movement track length information, and the preset condition is any three of an actual intersection ratio being equal to or greater than a preset intersection ratio threshold, an effective gaze rate being equal to or greater than a preset gaze rate threshold, an effective eye jump rate being equal to or greater than a preset eye jump rate threshold, and an effective track rate being equal to or greater than a preset track rate threshold. This embodiment is equivalent to the case where the condition information includes four pieces of information, and only any three of the preset conditions need to be satisfied to qualify the annotation data. It should be understood that the condition information in the present application may be any combination of the eye movement area information, the total eye gaze time information, the total eye jump number information, and the eye movement track length information, so that a person skilled in the art may adjust the information included in the condition information and adjust the preset condition corresponding to the condition information according to actual needs.
In some embodiments, the condition information includes any one or any two or any three of eye movement area information, total eye gaze time information, total eye jump number information and eye movement track length information, and if the condition information is judged to be unqualified according to the preset condition corresponding to the condition information, the condition corresponding to other information in the condition information is utilized to review the label data. Because the embodiment rechecks the labeling data by using the conditions corresponding to other information in the condition information, the embodiment can effectively reduce the situation of misjudgment on the labeling data.
As can be seen from the above, the method for detecting the quality of the data marking based on eyeball tracking provided by the application judges whether the marking data is qualified or not by judging whether the condition information meets the preset condition, and the method is equivalent to selecting an audit standard corresponding to the condition information according to the condition information to detect the quality of the standard data.
In a second aspect, referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device includes: processor 101 and memory 102, the processor 101 and memory 102 being interconnected and in communication with each other by a communication bus 103 and/or other form of connection mechanism (not shown), the memory 102 storing computer readable instructions executable by the processor 101, which when executed by an electronic device, the processor 101 executes the computer readable instructions to perform the methods in any of the alternative implementations of the above embodiments to perform the functions of: s1, acquiring eye movement information, wherein the eye movement information is change information of an eye gazing position in a primary image data labeling process; s2, acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information; and S3, judging whether the labeling data are qualified or not according to whether the condition information meets the preset condition.
In a third aspect, embodiments of the present application further provide a computer readable storage medium, which when executed by a processor, performs a method in any of the alternative implementations of the above embodiments to implement the following functions: s1, acquiring eye movement information, wherein the eye movement information is change information of an eye gazing position in a primary image data labeling process; s2, acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information; and S3, judging whether the labeling data are qualified or not according to whether the condition information meets the preset condition. The computer readable storage medium may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM), electrically erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk.
As can be seen from the above, the method, the device and the medium for detecting the quality of the data marking based on eye tracking provided by the present application determine whether the marking data is qualified by determining whether the condition information satisfies the preset condition, and the method is equivalent to selecting an audit standard corresponding to the condition information according to the condition information to detect the quality of the standard data.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above described embodiments of the apparatus are only illustrative, e.g. the above described division of units is only one logical function division, and there may be another division in practice, and e.g. multiple units or components may be combined or integrated into another robot, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may rise to one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Furthermore, functional modules in various embodiments of the present application may be integrated together to form a single portion, or each module may exist alone, or two or more modules may be integrated to form a single portion.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (6)
1. The data labeling quality detection method based on eyeball tracking is characterized by comprising the following steps of:
acquiring eye movement information, wherein the eye movement information is change information of eye gazing positions in a primary image data labeling process;
acquiring condition information according to the eye movement information, wherein the condition information comprises any one or more of eye movement area information, total eye-gaze time information, total eye jump quantity information and eye movement track length information;
judging whether the labeling data are qualified or not according to whether the condition information meets preset conditions or not;
if the condition information comprises the eye movement area information, the preset condition is that the actual intersection ratio of the eye movement area information and the marked data area information is greater than or equal to a preset intersection ratio threshold value, and the marked data area information is the area where the marked data is located;
if the condition information comprises the total gaze time information, the preset condition is that the effective gazing rate is larger than or equal to a preset gazing rate threshold, the effective gazing rate is the ratio of the effective gazing time information to the total gaze time information, and the effective gazing time information is the total time that the eye gazing position is positioned in the marked data area information in the process of marking the image data;
if the condition information comprises the total eye jump quantity information, the preset condition is that the effective eye jump rate is larger than or equal to a preset eye jump rate threshold value, the effective eye jump rate is the ratio of the effective eye jump quantity information to the total eye jump quantity information, and the effective eye jump quantity information is the difference value between the total eye jump quantity information and the back vision eye jump quantity information;
if the condition information includes the eye movement track length information, the preset condition is that the effective track rate is greater than or equal to a preset track rate threshold, the effective track rate is a ratio of the effective track length information to the eye movement track length information, and the effective track length information is the total length of the eye movement track in the marking data area information in a one-time image data marking process.
2. The method for detecting the labeling quality of data based on eye tracking according to claim 1, wherein the calculation formula of the actual blending ratio is:
；
wherein IoU represents the actual cross ratio, S A Representing the information of the marked data area S B Representing eye movement region information, S A ∩S B Representing the intersection of the labeling data region information and the eye movement region information, S A ∪S B And representing the union of the labeling data region information and the eye movement region information.
3. The method for detecting quality of data labeling based on eye tracking according to claim 1, wherein the number of eye jumps is the number of eye jumps of the back view type, and if the angle between the current eye jump direction and the previous eye jump direction is greater than 90 °, the current eye jump is regarded as the eye jump of the back view type.
4. The method for detecting quality of data labeling based on eye tracking according to claim 1, wherein the condition information includes the eye movement area information and the total gaze time information, and the preset condition is that an actual ratio of the eye movement area information to the labeling data area information is equal to or greater than a preset ratio threshold and an effective gaze rate is equal to or greater than a preset gaze rate threshold.
5. An electronic device comprising a processor and a memory storing computer readable instructions that, when executed by the processor, perform the steps in the method of any of claims 1-4.
6. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, performs the steps of the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310526432.5A CN116246332B (en) | 2023-05-11 | 2023-05-11 | Eyeball tracking-based data labeling quality detection method, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310526432.5A CN116246332B (en) | 2023-05-11 | 2023-05-11 | Eyeball tracking-based data labeling quality detection method, device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116246332A CN116246332A (en) | 2023-06-09 |
| CN116246332B true CN116246332B (en) | 2023-07-28 |
Family
ID=86633474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310526432.5A Active CN116246332B (en) | 2023-05-11 | 2023-05-11 | Eyeball tracking-based data labeling quality detection method, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116246332B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107515677A (en) * | 2017-08-31 | 2017-12-26 | 杭州极智医疗科技有限公司 | Notice detection method, device and storage medium |
| WO2020042678A1 (en) * | 2018-08-28 | 2020-03-05 | 北京七鑫易维信息技术有限公司 | Oculomotorius information-based alarming method and apparatus, device and storage medium |
| CN112860059A (en) * | 2021-01-08 | 2021-05-28 | 广州朗国电子科技有限公司 | Image identification method and device based on eyeball tracking and storage medium |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103339628A (en) * | 2011-03-30 | 2013-10-02 | 日本电气株式会社 | Data relatedness assessment device, data relatedness assessment method, and recording medium |
| US20160106315A1 (en) * | 2014-05-30 | 2016-04-21 | Umoove Services Ltd. | System and method of diagnosis using gaze and eye tracking |
| CN108520218A (en) * | 2018-03-29 | 2018-09-11 | 深圳市芯汉感知技术有限公司 | A kind of naval vessel sample collection method based on target tracking algorism |
| CN108805017A (en) * | 2018-04-27 | 2018-11-13 | 淘然视界(杭州)科技有限公司 | A kind of face automatic marking method and system |
| US20200201966A1 (en) * | 2018-12-21 | 2020-06-25 | Oath Inc. | Biometric based self-sovereign information management |
| CN109816013A (en) * | 2019-01-17 | 2019-05-28 | 陆宇佳 | It is tracked based on eye movement and carries out image pattern quick obtaining device and method |
| US11221671B2 (en) * | 2019-01-31 | 2022-01-11 | Toyota Research Institute, Inc. | Opengaze: gaze-tracking in the wild |
| CN109925678A (en) * | 2019-03-01 | 2019-06-25 | 北京七鑫易维信息技术有限公司 | A kind of training method based on eye movement tracer technique, training device and equipment |
| CN109993315B (en) * | 2019-03-29 | 2021-05-18 | 联想(北京)有限公司 | Data processing method and device and electronic equipment |
| CN110807364B (en) * | 2019-09-27 | 2022-09-30 | 中国科学院计算技术研究所 | Modeling and capturing method and system for three-dimensional face and eyeball motion |
| DE102019217730A1 (en) * | 2019-11-18 | 2021-05-20 | Volkswagen Aktiengesellschaft | Method for operating an operating system in a vehicle and operating system for a vehicle |
| CN111309144B (en) * | 2020-01-20 | 2022-02-01 | 北京津发科技股份有限公司 | Method and device for identifying injection behavior in three-dimensional space and storage medium |
| CN111563633A (en) * | 2020-05-15 | 2020-08-21 | 上海乂学教育科技有限公司 | Reading training system and method based on eye tracker |
| KR102537695B1 (en) * | 2020-12-22 | 2023-05-26 | 주식회사 에스원 | Automatic Data Labeling Method based on Deep learning Object Detection amd Trace and System thereof |
| CN114332860A (en) * | 2021-11-29 | 2022-04-12 | 北京机械设备研究所 | Natural interaction condition event related electroencephalogram marking method, device, medium and equipment |
| CN114820456A (en) * | 2022-03-30 | 2022-07-29 | 图湃(北京)医疗科技有限公司 | Image processing method and device |
| CN114879845A (en) * | 2022-05-23 | 2022-08-09 | 南京理工大学 | Picture label voice labeling method and system based on eye tracker |
-
2023
- 2023-05-11 CN CN202310526432.5A patent/CN116246332B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107515677A (en) * | 2017-08-31 | 2017-12-26 | 杭州极智医疗科技有限公司 | Notice detection method, device and storage medium |
| WO2020042678A1 (en) * | 2018-08-28 | 2020-03-05 | 北京七鑫易维信息技术有限公司 | Oculomotorius information-based alarming method and apparatus, device and storage medium |
| CN112860059A (en) * | 2021-01-08 | 2021-05-28 | 广州朗国电子科技有限公司 | Image identification method and device based on eyeball tracking and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116246332A (en) | 2023-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7268879B2 (en) | Tracking Surgical Items Predicting Duplicate Imaging | |
| JP6994101B2 (en) | Target recognition methods, devices, storage media and electronic devices | |
| EP3153092B1 (en) | Pupil detection system, gaze detection system, pupil detection method, and pupil detection program | |
| CN106412573B (en) | A kind of method and apparatus of detector lens stain | |
| CN110623629A (en) | Visual attention detection method and system based on eyeball motion | |
| US20160095511A1 (en) | Eye gaze detecting device and eye gaze detection method | |
| CN114018160B (en) | Tab imaging device and tab size detection method | |
| US20190066333A1 (en) | Information processing apparatus, method for controlling information processing apparatus, and storage medium | |
| Simonelli et al. | Are we missing confidence in pseudo-lidar methods for monocular 3d object detection? | |
| JP2008192100A (en) | Eyelid detector and program | |
| US20130243251A1 (en) | Image processing device and image processing method | |
| CN109298786B (en) | Method and device for evaluating marking accuracy | |
| CN103530623A (en) | Information observation method and information observation device | |
| CN109697719A (en) | A kind of image quality measure method, apparatus and computer readable storage medium | |
| US10642353B2 (en) | Non-transitory computer-readable storage medium, information processing apparatus, and information processing method | |
| US20200250553A1 (en) | Out-of-vehicle communication device, out-of-vehicle communication method, information processing device, and computer readable medium | |
| CN110135420A (en) | Dial plate state identification method and device, readable storage medium storing program for executing and electronic equipment | |
| WO2016172340A1 (en) | System and methods for assessing vision using a computing device | |
| EP2926721B1 (en) | Information processing technique for eye gaze movements | |
| CN116246332B (en) | Eyeball tracking-based data labeling quality detection method, device and medium | |
| CN111767820A (en) | Method, device, equipment and storage medium for identifying object concerned | |
| US20170243061A1 (en) | Detection system and detection method | |
| JP2015015031A (en) | Advertising effect measurement server, advertising effect measurement system, and program | |
| CN109583266A (en) | A kind of object detection method, device, computer equipment and storage medium | |
| US20220395206A1 (en) | Cognitive assessment system based on eye movement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |