KR102500052B1 - Method and apparatus for scoring motion using eye tracking system - Google Patents

Abstract

Disclosed are a method and an apparatus for grading a motion using an eye tracking system. According to an embodiment of the present invention, disclosed is the method for grading a motion using an eye tracking system, which comprises the following steps of: acquiring one or more pieces of gaze information in which information obtained by tracking the gaze of a grader for the motion and an image obtained by capturing the motion are combined; and generating grading information for the motion by linking one or more points input from the grader with one or more of the gaze information to which the points are applied.

Description

Method and apparatus for scoring motion using eye tracking system {METHOD AND APPARATUS FOR SCORING MOTION USING EYE TRACKING SYSTEM}

The present invention relates to a method and apparatus for scoring a motion, and more particularly, to a method and apparatus for scoring a motion by combining a scorer's gaze detected by a gaze tracking system with a point of the scorer.

The information described in this section simply provides background information on the present invention and does not constitute prior art.

After Taekwondo Poomsae was adopted as an official event at the Asian Games held in Jakarta in 2018, the population practicing Taekwondo Poomsae is expanding. In addition, since the 2022 World Taekwondo Poomsae Championships will be held in Goyang, Korea, interest in Taekwondo Poomsae is increasing.

In a Taekwondo Poomsae match, the scorer (judge) sees the player's Poomsae movements and inputs the evaluation (points) to the scoring device, and the deducted points are displayed on the monitor (electronic board).

Competition events of Taekwondo Poomsae can be divided into official Poomsae and free Poomsae. Deduction points are applied to the official Poomsae, and scoring points are applied to the free Poomsae.

Specifically, in certified Poomsae, deduction points are applied for accuracy (basic movements, detailed movements for each Poomsae, balance) and expressiveness (speed and power, harmony (strong, slow, rhythm), expression of energy), Scores for expression and expressiveness are added together.

In the case of accuracy, whether or not the movement meets the basic Taekwondo motions and the performance standards for each Poomsae motion prescribed by Kukkiwon is evaluated. A deduction point of 0.1 points is deducted for small mistakes and 0.3 points for big mistakes. Applied. Deduction points for accuracy are entered and applied immediately during the process of Taekwondo Poomsae movement.

In the case of expressiveness, the ability to express the unique meaning of each poomsae, the characteristics of techniques, and the connection characteristics of motions is evaluated on the basis of 2.0 points. Deduction points for expressiveness are input and applied after the progress of Taekwondo Poomsae movements is completed.

For free poomsae, scoring points are applied based on 6.0 points for technical skills and 4.0 points for directing skills. In the case of technical skills, scoring points are applied based on 5.0 points for kicking difficulty and 1.0 points for accuracy of movements and completeness of poomsae, and the points for technical skills and directing skills are added together.

Since this scoring method provides only deduction points or scoring points, it is difficult to present a clear basis for deductions to players and coaches in the process of expressing deductions for mistakes in Poomsae movements or incorrect postures. Due to these limitations, the conventional scoring method causes a problem in which players and coaches often raise objections to the judgment.

An embodiment of the present invention provides a method and apparatus capable of inducing a more accurate evaluation as well as providing an accurate basis for evaluation by providing information tracking the scorer's gaze together with the scorer's evaluation. there is

According to an embodiment of the present invention, a method for scoring a motion using a gaze tracking system includes obtaining one or more gaze information in which information tracking a scorer's gaze for the motion and an image of the motion are combined. step; and generating scoring information for the motion by linking one or more points input from the scorer with one or more gaze information to which the points are applied.

According to another embodiment of the present invention, an apparatus for scoring a motion using an eye tracking system includes a memory storing one or more programs for scoring the motion; and a processor executing programs stored in the memory, wherein the processor obtains one or more pieces of gaze information in which information tracking a scorer's gaze for the motion and an image of the motion are combined, and receives information from the scorer. An apparatus characterized in generating scoring information for the motion by linking one or more input points with one or more gaze information to which the points are applied.

Since an embodiment of the present invention utilizes the information obtained by tracking the scorer's gaze as a basis for the scorer's evaluation, it is possible to provide an accurate basis for the evaluation and induce a more accurate evaluation.

In addition, since another embodiment of the present invention provides accurate data on motions and judgments, it can be used to efficiently implement scorer education and player training.

1 is an exemplary diagram for explaining a situation in which a scoring device may be utilized.
2 is an exemplary block diagram for explaining the configuration of a scoring device.
3 is a flowchart for explaining an example of a scoring method.
4 is an exemplary diagram for explaining an example of a scoring method.
5 is a flowchart for explaining another example of a scoring method.
6 is an exemplary diagram for explaining another example of a scoring method.
7 is a flowchart for explaining another example of a scoring method.
8 is an exemplary diagram for explaining another example of a scoring method.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

In addition, in describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. Throughout the specification, when a part 'includes' or 'includes' a certain component, it means that it may further include other components without excluding other components unless otherwise stated. . In addition, the '... Terms such as 'unit' and 'module' refer to a unit that processes at least one function or operation, and may be implemented as hardware, software, or a combination of hardware and software.

Prior to the detailed description of the present invention, 'operation' referred to in this specification will be described.

Motions targeted for scoring in the present invention may include various motions such as taekwondo poomsae motions, gymnastics motions, figure skating motions, dance motions, diving motions, and martial arts motions.

That is, if a player or participant (hereinafter referred to as 'athlete') performs a motion that moves his or her body, and a referee or scorer (hereinafter referred to as 'scorer') performs an evaluation, it does not matter what the name is referred to. It can be included in the operation of the present invention without.

Hereinafter, for convenience of explanation, the present invention will be described with a focus on Taekwondo Poomsae motion.

1 is an exemplary diagram for explaining a situation in which a device 100 for scoring motions using an eye tracking system (hereinafter, referred to as a 'scoring device') 100 may be utilized.

When the player performs a Taekwondo Poomsae motion, the eye tracking device 110 tracks the scorer's gaze for the Taekwondo Poomsae motion and generates information (information tracking the scorer's gaze), and the photographing device 120 captures the player's gaze. Taekwondo Poomsae motions can be photographed and related information (videos of Taekwondo Poomsae motions) can be created.

The eye tracking device 110 may be a wearable device such as goggles worn by a scorer, a Head Mounted Display (HMD), or glasses. In this case, the number of eye tracking devices 110 may be plural according to the number of scorers (110-1, 110-2, ...). In addition, the eye tracking device 110 may be a camera installed in a terminal such as a desktop or tablet (eg, a scoring device), or a device disposed at a predetermined distance from a scorer to track the scorer's gaze.

As illustrated in FIG. 1 , the photographing device 120 and the eye tracking device 110 may be implemented as separate physical components or included in one physical component. For example, the eye tracking device 110 and the photographing device 120 may be installed together in a wearable device worn by a scorer. As another example, the eye tracking device 110 and the photographing device 120 may be installed together in a terminal such as a desktop or tablet.

The eye-tracking device 110 and the photographing device 120 may form eye-tracking systems 110 and 120 . The gaze tracking systems 110 and 120 may generate gaze information by combining information obtained by tracking the gaze of the scorer with images of Taekwondo Poomsae motions.

Therefore, gaze information may include information on tracking the scorer's gaze and images of Taekwondo Poomsae motions. In addition, the gaze information may further include the degree of time (degree of interest) for which the scorer's gaze is fixed at a specific position.

The scoring device 100 is a device that generates scoring information based on evaluation (points) input from a scorer and gaze information (information in which the gaze tracking device tracks the grader's gaze and an image captured by a photographing device). corresponds to

The scoring device 100 may be plural according to the number of scorers (100-1, 100-2, ...), and may be implemented as a desktop or a tablet capable of receiving points from scorers.

Depending on the embodiment, the scoring device 100 may be physically separated from a device (input device) capable of receiving points from a scorer. For example, the input device is implemented as a separate component including a button or a touch pad, and the scoring device 100 is implemented as a server that generates scoring information by receiving points input by each scorer from the input device. It can be.

The scoring device 100 transmits the generated scoring information to the monitor 130, and the monitor 130 may display the scoring information to the outside.

2 is an exemplary block diagram for explaining the configuration of the scoring device 100, and FIG. 3 is a flowchart for explaining an example of a scoring method.

As shown in FIG. 2 , the scoring device 100 may include a communication unit 210 , a memory 220 and a processor 230 .

The communication unit 210 may perform wired/wireless communication with the eye tracking device 110 , the photographing device 120 , and the monitor 130 . Gaze information in which information tracking the scorer's eye gaze and images of Taekwondo Poomsae motions are combined may be transmitted from the gaze tracking systems 110 and 120 to the communication unit 210 . Scoring information in which the points input by the scorer and gaze information are linked may be transmitted from the communication unit 210 to the monitor 130 .

One or more programs for scoring Taekwondo Poomsae movements may be stored in the memory 220 . In addition, current or past gaze information may be further stored in the memory 220, or scoring information in which points are combined with current or past gaze information may be further stored.

The processor 330 may execute programs stored in the memory 220 to generate scoring information as follows.

The processor 330 may obtain gaze information in which information tracking the scorer's eye gaze and images of Taekwondo Poomsae motions are combined (S310).

For example, the processor 330 may obtain gaze information through a method of receiving gaze information from the gaze tracking systems 110 and 120 through the communication unit 210 . As another example, the processor 330 may obtain gaze information through a method of accessing the memory 220 and reading gaze information stored in the memory 220 .

The former example corresponds to the case where the Taekwondo Poomsae motion is scored during or immediately after the Taekwondo Poomsae motion is performed, and the latter example corresponds to the case where the video stored in the memory 220 is played after the Taekwondo Poomsae motion is completed. This refers to the case where the Taekwondo Poomsae movement is scored. In the case of the latter example, the scoring device 100 may further include a display (not shown) for playing an image.

The processor 330 may generate scoring information for Taekwondo Poomsae motions by linking one or more points input from the scorer with one or more gaze information to which the points are applied (S320).

An exemplary diagram for explaining an example of a scoring method for Taekwondo Poomsae motions is shown in FIG. 4 .

Figure 4 (a) shows an example of gaze information in which information tracking the scorer's eyes and images of Taekwondo Poomsae are combined, and Figure 4 (b) shows points and gaze information to which these points are applied are linked. shows an example of what has been done.

As shown in (a) of FIG. 4, it is assumed that the scorer's gaze moves in the order of gaze information 1 → gaze information 2 → gaze information 3 → gaze information 5. As shown in (b) of FIG. 4, the first point P1 is applied to gaze information 1, the second point P2 is applied to gaze information 3, and the third point P3 is applied to gaze information 5. Assume applied.

In this case, the processor 230 associates the first point P1 with gaze information 1, the second point P2 with gaze information 3, and the third point P3 with gaze information 5. Scoring information can be created.

In this way, since the present invention provides data capable of accurately determining which motion of the player the point is given to by the scorer's eyes, it is possible to provide a clear basis for the scorer's decision.

Depending on the embodiment, the points input from the scorer may include deduction points for deducting points and scoring points for scoring points. In this case, the processor 330 creates scoring information (scoring information linked with deducted points) by linking deduction points and gaze information, and links scoring points and gaze information to score information (scoring information linked with scoring points). information) can be created.

Depending on the embodiment, the processor 230 may select only some (one or more) of the gaze information (S322) and generate scoring information by linking the selected gaze information with points (S324).

Among the gaze information, gaze information selected by the processor 230 is referred to as 'target gaze information'. The criterion for selecting the target gaze information may be the time when the scorer's gaze is fixed or the point at which points are input. An embodiment of selecting target gaze information will be described in detail below.

5 is a flowchart for explaining an example of a method of selecting target gaze information.

As described above, points for the accuracy of certified Poomsae are immediately input and applied during the course of Taekwondo Poomsae movements. In this case, since the scorer inputs a point using a button or touch pad while watching Taekwondo Poomsae, the point at which the point is input and the point at which the point is actually applied (gaze information) may be different. That is, the point of view of the gaze information to which the point is applied precedes the point of time when the point is input. Here, the point of view of the gaze information means the point of time when the grader's gaze gazes at a specific motion.

Therefore, if gaze information at the same point in time as the point is input is selected among the gaze information, gaze information to which the point is not applied (gaze information after the point is input) is selected, not the gaze information to which the point is actually applied. problem arises.

In consideration of the time difference between the point at which the points are input and the point at which the point is actually applied, the processor 230 may select, as the target gaze information, gaze information prior to a preset time from the point at which the points are input (S510). . The target gaze information means gaze information to which points are actually applied among gaze information.

Here, the preset time is variable considering the time difference estimated based on the data generating the past scoring information (the time difference between the point at which the point is input and the point at which the point is actually applied) and the individual time difference of each scorer. It can be.

When the target gaze information is selected, the processor 230 may generate scoring information by linking the points with the target gaze information (S520).

Depending on the embodiment, there may be a plurality of gaze information (candidate gaze information) within a preset time from when points are input. In this case, the processor 230 may select the most preceding candidate gaze information, the least lagging candidate gaze information, or the candidate gaze information located in the center among a plurality of candidate gaze information as the target gaze information. As another example, the processor 230 may display all of a plurality of candidate gaze information through a display, request selection of one of them, and select the selected candidate gaze information as the target gaze information.

An exemplary diagram for explaining an example of a method of selecting target gaze information is shown in FIG. 6 . FIG. 6(a) shows gaze information, and FIG. 6(b) shows target gaze information.

As shown in (a) of FIG. 6, it is assumed that the scorer's gaze moves in the order of gaze information 1 → gaze information 2 → gaze information 3 → gaze information 5. As shown in (b) of FIG. 6, there is a viewpoint of gaze information 1 within a preset time from the input of the first point P1, and gaze information 3 within a preset time of input of the second point P2. It is assumed that a viewpoint of , and a viewpoint of gaze information 5 exist within a preset time from the input of the third point P3.

In this case, the processor 230 selects gaze information 1 as the target gaze information and associates it with the first point P1, selects gaze information 2 as the target gaze information and associates it with the second point P2, , Gaze information 5 may be selected as the target gaze information, and scoring information may be generated by linking it with the third point P3.

7 is a flowchart for explaining another example of selecting target gaze information.

As described above, the expression power of the certified Poomsae and the directing power of the free Poomsae are input after the Taekwondo Poomsae motion is completed. In the case of expressive power and directing power, the object of judgment is not the individual judgment of Taekwondo Poomsae motion at a specific moment, but the overall process of Taekwondo Poomsae motion. There is a need to select information.

To this end, the processor 230 may select target gaze information according to the degree of time (interest) that the scorer's gaze is fixed on a specific position of the motion (S710). That is, the processor 230 may select, as target gaze information, gaze information having an interest level equal to or greater than a preset value from among gaze information.

The degree of interest is a value indicating how much the scorer pays attention to a specific motion, and as the scorer pays more attention, the degree of interest in the corresponding motion or position may increase.

The preset value is an estimated value based on data that generated past scoring information (data comparing past data on whether a scorer gave actual points for a specific action and the level of interest in the past for that action), scorer It can be varied by the difference in individual interest of each of them.

When the target gaze information is selected, the processor 230 may generate scoring information by linking the points with the target gaze information (S720).

As described above, since the present invention provides data by which the scorer can accurately determine which motion of the player he or she has focused on and has given points to, a clear basis for the scorer's decision can be provided.

An exemplary diagram for explaining another example of selecting target gaze information is shown in FIG. 8 . FIG. 8(a) shows gaze information (circles on a white background), and FIG. 8(b) shows target gaze information. The larger the value of the degree of interest, the larger the size of the circle on a white background representing gaze information was expressed.

As shown in (a) of FIG. 8, it is assumed that the scorer's gaze moves in the order of gaze information 1 → gaze information 2 → gaze information 3 → gaze information 5. In addition, as shown in (a) of FIG. 8, it is assumed that the degree of interest increases in the order of gaze information 2 → gaze information 5 → gaze information 1 → gaze information 4 → gaze information 3. Furthermore, it is assumed that gaze information 3, gaze information 4, and gaze information 1 are equal to or greater than preset values.

As shown in (b) of FIG. 8, since the degree of interest in gaze information 1 is equal to or greater than the preset value, gaze information 1 is selected as the target gaze information, and since the degree of interest in gaze information 4 is greater than the preset value, gaze information 4 is selected as the target gaze information. and since the degree of interest in the gaze information 3 is equal to or greater than a preset value, the gaze information 3 may be selected as the target gaze information. Since the degree of interest of gaze information 5 and gaze information 2 is less than a preset value, gaze information 5 and gaze information 2 may not be selected as target gaze information.

In this case, the processor 230 may generate scoring information by linking gaze information 1, gaze information 4, and gaze information 3 (target gaze information) with points.

In FIGS. 3, 5, and 7, it is described that each process is sequentially executed, but this is merely an example of the technical idea of one embodiment of the present invention. In other words, those skilled in the art to which an embodiment of the present invention belongs may change and execute the order described in FIGS. 3, 5, and 7 without departing from the essential characteristics of the embodiment of the present invention, or each One or more of the processes may be executed in parallel, and various modifications and variations may be applied. Therefore, FIGS. 3, 5, and 7 are not limited to a time-sequential order.

Meanwhile, each of the processes shown in FIGS. 3, 5 and 7 can be implemented as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. That is, computer-readable recording media include magnetic storage media (eg, ROM, floppy disk, hard disk, etc.), optical reading media (eg, CD-ROM, DVD, etc.) and carrier waves (eg, Internet Transmission through) and the same storage medium. In addition, the computer-readable recording medium may be distributed to computer systems connected through a network to store and execute computer-readable codes in a distributed manner.

The above description is merely an example of the technical idea of the present embodiment, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be construed according to the claims below, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of this embodiment.

100: scoring device
110: eye tracking device
120: shooting device
130: monitor

Claims (7)

As a method for scoring motions using an eye tracking system,
acquiring one or more pieces of gaze information in which information obtained by tracking the scorer's gaze for the motion and an image obtained by capturing the motion are combined; and
generating scoring information for the motion by linking one or more points input from the scorer with one or more gaze information to which the points are applied;
The generating step is
selecting one or more pieces of target gaze information from among the gaze information according to a degree of time when the scorer's gaze is fixed or a time point at which the points are input; and
generating the scoring information by associating the points with the target gaze information;
In the selection step,
Among the gaze information, selecting gaze information prior to a preset time from the point at which each of the points is input as the target gaze information
method. delete delete According to claim 1,
In the selection step,
Among the gaze information, gaze information in which a degree of time during which the scorer's gaze is fixed is equal to or greater than a preset value is selected as the target gaze information. According to claim 1,
These points are
Including deduction points and scoring points;
The generating step is
and generating scoring information linked to the deduction points and scoring information linked to the scoring points. A computer-readable recording medium recording a program for executing the method of any one of claims 1, 4, and 5 in a computer. A device for scoring motions using an eye tracking system,
a memory storing one or more programs for scoring the operation; and
A processor for executing programs stored in the memory;
the processor,
Obtaining one or more gaze information in which information tracking the scorer's gaze for the motion and an image of the motion are combined;
generating scoring information for the motion by linking one or more points input from the scorer with one or more gaze information to which the points are applied;
The scoring information,
Selecting one or more target gaze information from among the gaze information according to the degree of time when the scorer's gaze is fixed or the point at which the points are input;
It is created by linking the points and the target gaze information,
Selecting the gaze information,
Characterized in that, among the gaze information, gaze information prior to a preset time from the point at which each of the points is input is selected as the target gaze information.
Device.

Images