CN114419526B - Foul behavior detection method and apparatus, electronic device and storage medium - Google Patents
Foul behavior detection method and apparatus, electronic device and storage medium Download PDFInfo
- Publication number
- CN114419526B CN114419526B CN202210328144.4A CN202210328144A CN114419526B CN 114419526 B CN114419526 B CN 114419526B CN 202210328144 A CN202210328144 A CN 202210328144A CN 114419526 B CN114419526 B CN 114419526B
- Authority
- CN
- China
- Prior art keywords
- player
- foul
- dimensional distance
- distance
- key point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30221—Sports video; Sports image
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application relates to the technical field of machine vision, and discloses a foul behavior detection method, a foul behavior detection device, electronic equipment and a storage medium, wherein the foul behavior detection method comprises the following steps: intercepting a target frame of the color video to obtain a first image, and determining a first player and a second player in the first image; wherein the suspected foul behavior exists in the target frame, the first player is a suspected foul player, and the second player is a suspected foul player; determining a homonymous frame with the same time sequence as the target frame in the depth video, and intercepting the homonymous frame to obtain a second image; the depth video and the color video are obtained by shooting by the same camera at the same time; determining a three-dimensional distance between the first player and the second player based on the second image; under the condition that the three-dimensional distance is smaller than the preset safety distance, the first player is determined to have the foul behavior, so that the interpretability of penalty of the foul behavior is enhanced, and the phenomena of misjudgment and missed judgment are avoided as much as possible.
Description
Technical Field
The embodiment of the application relates to the technical field of machine vision, in particular to a foul behavior detection method and device, electronic equipment and a storage medium.
Background
Sports is a consciously cultivated variety of activities on self physical quality gradually developed in the process of human development, including various forms of physical activities such as walking, running, jumping, throwing, dancing and the like, and is developed into various projects such as track and field, balls, swimming, martial arts, aerobics, mountaineering, skating, weightlifting, wrestling, judo, bicycles and the like.
In order to maintain fairness and safety of sports competitions, each sports competition project is provided with certain rules and rules, behaviors of violating various rules and rules are called as 'foul' behaviors, the foul behaviors do not accord with the spirit of fair competition in the sports competitions, serious foul behaviors can even threaten the physical health and life safety of athletes, the foul behaviors of the athletes need to be pointed out and corrected in time, and referees are the roles of pointing out and correcting the foul behaviors of the athletes in each sports competition.
The inventor of the application finds that physical contact is caused, so that the foul behaviors which cause the fall and injury of the foul athletes are easily found by referees and high-speed high-definition cameras, but some actions which are not in physical contact with other athletes are made by the athletes, so that the safety of other athletes is threatened actually, other athletes have to avoid, the situations of fall or unsatisfactory match performance occur, the actions with the safety threats are dangerous foul behaviors actually, but the foul behaviors are not easy to be found by the referees, whether the penalty depends on the personal experience of the referees, the interpretability is poor, the missing judgment and the misjudgment of the foul behaviors are easily caused, and the requirement on fairness in the sports competition cannot be met.
Disclosure of Invention
The embodiment of the application aims to provide a method, a device, an electronic device and a storage medium for detecting an unlawful behavior, which can scientifically and accurately detect the unlawful behavior without physical contact, enhance the interpretability of penalty judgment of the unlawful behavior and avoid the occurrence of misjudgment and missed judgment as much as possible.
In order to solve the above technical problem, an embodiment of the present application provides a foul behavior detection method, including the following steps: intercepting a target frame of a color video to obtain a first image, and determining a first player and a second player in the first image; wherein the target frame has a suspected foul behavior, the first player is a suspected foul player, and the second player is a suspected foul player; determining a homonymous frame with the same time sequence as the target frame in the depth video, and intercepting the homonymous frame to obtain a second image; the depth video and the color video are shot by the same camera at the same time; determining a three-dimensional distance between the first player and the second player based on the second image; determining that the first athlete has a foul behavior if the three-dimensional distance is less than a preset safe distance.
An embodiment of the present application further provides a foul behavior detection apparatus, including: the device comprises a camera module, a positioning module, a calculating module and a detecting module, wherein the camera module is used for shooting a color video and a depth video simultaneously; the positioning module is used for intercepting a target frame of the color video to obtain a first image, determining a first player and a second player in the first image, determining a same-name frame with the same time sequence as the target frame in the depth video, and intercepting the same-name frame to obtain a second image, wherein the target frame has suspected foul behaviors, the first player is a suspected foul player, and the second player is a suspected foul player; the computing module is to determine a three-dimensional distance between the first player and the second player based on the second image; the detection module is used for determining that the first athlete has a foul behavior under the condition that the three-dimensional distance is smaller than a preset safety distance.
An embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described foul behavior detection method.
Embodiments of the present application further provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned foul behavior detection method.
The method, the device, the electronic equipment and the storage medium for detecting the foul behavior provided by the embodiment of the application intercept a target frame of a color video shot by a camera in a sports game, namely a frame with suspected foul behavior, obtain a first image, respectively determine a first player suspected of foul and a second player suspected of foul in the first image, then determine a same-name frame with the same time sequence as the target frame in a depth video shot by the same camera with the color video, intercept the same-name frame to obtain a second image, then determine a three-dimensional distance between the first player and the second player based on the second image, finally judge whether the three-dimensional distance between the first player and the second player is smaller than a preset safety distance, determine that the first player has the foul behavior under the condition that the three-dimensional distance between the first player and the second player is smaller than the preset safety distance, considering that some foul actions which are not in physical contact with other athletes and are made by the athletes are not easy to be discovered by the referees, and a typical characteristic of the foul actions with the security threat is that the foul actions infringe the security zones of other athletes, in the embodiment of the application, the server detects whether the suspected foul athlete really infringes the security zones of the suspected foul athlete by calculating the three-dimensional distance between the suspected foul athlete and the suspected foul athlete, so that the referee is prompted to judge the foul actions, the foul actions without physical contact can be scientifically and accurately detected, the occurrence of misjudgment and missed judgment is avoided as much as possible, meanwhile, the calculation process of the three-dimensional distance provides scientific data support, and the solvability of the foul action judgment is enhanced.
Additionally, the determining a three-dimensional distance between the first player and the second player based on the second image comprises: determining a first keypoint of the first player and a second keypoint of the second player in the second image, calculating a two-dimensional distance between the first keypoint and the second keypoint according to the coordinates of the first keypoint and the coordinates of the second keypoint; calculating a depth value difference value between the first key point and the second key point according to the depth value of the first key point and the depth value of the second key point; according to the two-dimensional distance and the depth value difference value, the three-dimensional distance between the first key point and the second key point is calculated, and the three-dimensional distance between the first key point and the second key point is used as the three-dimensional distance between the first player and the second player, so that considering that in a fierce sports competition, close-range countermeasures are indispensable, the approaching or even touching of most body parts does not threaten the safety of the players, but a small part of body parts is fragile and easy to be injured, the three-dimensional distance between the first player and the second player is calculated by selecting the first key point of the first player and the second key point of the second player, and the calculation process is simple and fast and is more consistent with the actual situation of the sports competition.
Additionally, the first keypoints comprise a left foot of the first player and a right foot of the first player, the second keypoints comprise a left foot of the second player and a right foot of the second player, and the three-dimensional distances between the first player and the second player comprise a first three-dimensional distance between the left foot of the first player and the left foot of the second player, a second three-dimensional distance between the left foot of the first player and the right foot of the second player, a third three-dimensional distance between the right foot of the first player and the left foot of the second player, and a fourth three-dimensional distance between the left foot of the first player and the left foot of the second player; the determining that the first player has foul behavior in the case that the three-dimensional distance is smaller than a preset safety distance comprises: respectively judging whether the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance are smaller than a preset safety distance; in the case that at least one of the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance is smaller than the preset safety distance, it is determined that the first player has a foul behavior, and considering that in a sports competition such as basketball, football and the like, players often need to take off to complete actions such as shooting, grabbing a backboard, fighting a ball and the like, during the process from taking off to landing, feet and ankle joints of the players are fragile, and dangerous foul behaviors such as foot cushion easily occur in the violent sports competition, the padded players find that the feet are normally landed and must step on the feet of the opposite side, so that the players can avoid from the ground to fall, and although the players who take the foot cushion do not contact with the padded players, the foot cushion action actually exists and need to penalize, so the embodiment of the application detects four three-dimensional distances between the two feet of the first player and the second player, therefore, whether the foul behaviors exist or not is determined, the accuracy of penalty judgment of the foul behaviors can be further improved, and the fairness of the sports competition is improved.
Additionally, the first keypoint comprises the first player's left foot and the first player's right foot, the second keypoint comprises the second player's head, and the three-dimensional distance between the first player and the second player comprises a fifth three-dimensional distance between the first player's left foot and the second player's head, and a sixth three-dimensional distance between the first player's right foot and the second player's head; the determining that the first player has a foul behavior in the case that the three-dimensional distance is smaller than a preset safe distance comprises: respectively judging whether the fifth three-dimensional distance and the sixth three-dimensional distance are smaller than a preset safety distance; under the condition that the fifth three-dimensional distance is smaller than the preset safety distance, and/or under the condition that the sixth three-dimensional distance is smaller than the preset safety distance, it is determined that the first player has a foul action, and considering that the first player wears shoes for playing in a football sports competition and the second player wears skates for playing in an ice sports competition such as a hockey, the spikes and the skates are very sharp and dangerous, so that the spikes and the skates are dangerous to perform the foul action, and especially the spikes and the skates are more dangerous to approach the heads of other players, therefore, the embodiment of the application detects the distance between the two feet of the first player and the head of the second player, determines whether the foul action exists, can further improve the accuracy of penalty of the foul action, and improve the fairness of the competition sports.
Additionally, after the determining that the first athlete has had a foul act, the method further comprises: the three-dimensional distance between the first player and the second player and the preset safety distance are output, the penalty of the referee needs to be based on the fact, strong data evidence can greatly enhance the interpretability of the penalty of the unlawful act, and the detection basis, namely the three-dimensional distance between the first player and the second player and the preset safety distance can be output after the first player is determined to have the unlawful act, so that the penalty is reasonable.
In addition, the color video that obtains is shot including a plurality of different angles, the depth video that obtains is shot including a plurality of different angles, different angles include main viewing angle at least, overlook angle and side view angle, consider that a camera carries out the shooting based on the competition probably can not satisfy the needs that the foul detected, some too remote corners shoot not to arrive, consequently the embodiment of this application chooses the color video that obtains for use of a plurality of different angles, also be exactly set up a plurality of cameras of shooing at different angles to further promote the accuracy that the foul action detected.
In addition, the determining that the first player has foul behavior in the case that the three-dimensional distance is less than a preset safe distance includes: acquiring the injury state information of the second athlete under the condition that the three-dimensional distance is smaller than a preset safety distance; wherein the injury status information includes injured and non-injured; determining that the first player has a common foul act if the second player is not injured; if the second player is injured, the first player is determined to have malicious unlawful behaviors, and in order to protect the unlawful player, result-oriented penalty criteria are generally adopted in the sports competition, namely, unlawful penalty criteria of different degrees are carried out on the unlawful player according to the injury condition of the unlawful player, so that the rights and interests of the unlawful player can be further protected, and the fairness of the sports competition is further improved.
Drawings
One or more embodiments are illustrated by the corresponding figures in the drawings, which are not meant to be limiting.
FIG. 1 is a first flowchart of a foul behavior detection method according to one embodiment of the present application;
FIG. 2 is a flow chart for determining a three-dimensional distance between a first player and a second player based on a second image, according to one embodiment of the present application;
FIG. 3 is a schematic illustration of an offending action provided in accordance with an embodiment of the present application;
FIG. 4 is a schematic illustration of another foul act provided in accordance with an embodiment of the present application;
FIG. 5 is a flowchart II of a foul behavior detection method according to another embodiment of the present application;
FIG. 6 is a flowchart III of a foul behavior detection method according to another embodiment of the present application;
FIG. 7 is a schematic structural diagram of an foul behavior detection apparatus according to another embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that in the examples of the present application, numerous technical details are set forth in order to provide a better understanding of the present application. However, the technical solutions claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present application, and the embodiments may be mutually incorporated and referred to without contradiction.
An embodiment of the present application relates to a method for detecting a foul behavior, which is applied to an electronic device, where the electronic device may be a terminal or a server, and the electronic device in this embodiment and the following embodiments are described by taking the server as an example.
The specific flow of the foul behavior detection method of this embodiment may be as shown in fig. 1, and includes:
In a specific implementation, a camera for shooting a match in a sports competition field in real time is arranged beside the sports competition field, the camera has a color lens and an infrared lens, and can be used for simultaneously shooting a color video and a depth video in a target field, the shot color video and the shot depth video are aligned in time sequence, when the foul behavior detection is required, a server can intercept a target frame of the color video to obtain a first image and determine a first player and a second player in the first image, wherein the suspected foul behavior exists in the target frame, namely the target frame is a frame in which the foul behavior possibly exists, the target frame can be selected by a judge or a video judge group according to the real-time condition on the competition field, the first player determined by the server in the first image is a suspected foul player, the second player determined by the server in the first image is a suspected foul player, that is, in the first image, the first athlete is likely to violate the second athlete's safety zone.
It is understood that the target frame is a frame in a color video, and the server intercepts the target frame in the color video, and the resulting first image is a color image.
In one example, a color camera and an infrared camera for shooting the match in the sports competition field in real time can be arranged beside each other, the color camera can shoot the color video in the target field, and the infrared camera can shoot the depth video in the target field.
And 102, determining a frame with the same name in the time sequence as the target frame in the depth video, and intercepting the frame with the same name to obtain a second image.
Specifically, after determining the target frame, the server may find a frame with the same name in the depth video captured by the same camera as the color video, and intercept the frame with the same name to obtain the second image.
It can be understood that the homonymous frame of the target frame is a frame in the depth video, the second image obtained by intercepting the homonymous frame in the depth video by the server is a depth image, and each pixel point has depth data.
In one example, the server may find a frame with the same name as the target frame in the depth video according to the ranking of the target frame in the color video, such as: the target frame is the 5589 th frame in the color video, and the server can find the 5589 th frame of the depth video in the depth video, wherein the 5589 th frame of the depth video is the same-name frame of the target frame.
In another example, the server may find a homonymous frame of the target frame in the depth video according to the timestamp of the target frame, such as: the timestamp of the target frame is 16 minutes 37 seconds 42, the server finds the frame at 16 minutes 37 seconds 42 in the depth video, and the frame at 16 minutes 37 seconds 42 in the depth video is the same-name frame of the target frame.
Based on the second image, a three-dimensional distance between the first player and the second player is determined, step 103.
Specifically, after the server captures the second image, the three-dimensional distance between the first player and the second player may be determined based on the second image.
In a specific implementation, since the color video and the depth video are aligned in time sequence, the position of the first player in the first image is the position of the first player in the second image, and the position of the second player in the first image is the position of the second player in the second image, which is not changed, and the server does not need to search the second image for the first player and the second player.
In an example, the server may input coordinates of each pixel point corresponding to the first athlete, depth values of each pixel point corresponding to the first athlete, coordinates of each pixel point corresponding to the second athlete, and depth values of each pixel point corresponding to the second athlete into the ranging model to be trained, so as to obtain a three-dimensional distance between the first athlete and the second athlete output by the ranging model, where the ranging model is obtained by training based on a large number of training samples, and the ranging model may be configured by a person skilled in the art according to actual needs.
And 104, determining that the first player has foul behaviors under the condition that the three-dimensional distance is smaller than the preset safety distance.
In a specific implementation, after determining the three-dimensional distance between the first player and the second player, the server may determine whether the three-dimensional distance between the first player and the second player is smaller than a preset safety distance, and in a case that the server determines that the three-dimensional distance between the first player and the second player is smaller than the preset safety distance, the server determines that the first player has a foul behavior, where the preset safety distance may be set by a person skilled in the art according to a sporting event, and this is not specifically limited by the embodiments of the present application.
In one example, the server may determine that the first player has not committed an act if it determines that the three-dimensional distance between the first player and the second player is greater than or equal to a preset safe distance.
In an example, the color video in this application embodiment includes the color video that a plurality of different angles were shot and are obtained, the depth video includes the depth video that a plurality of different angles were shot and are obtained, different shooting angles include main viewing angle at least, overlook angle and side viewing angle, consider that only using a camera to carry out the shooting of sports competition probably can not satisfy the needs that the foul detected, some too remote corners can not shoot, consequently, the embodiment of this application chooses the color video that a plurality of different angles were shot for use, namely set up a plurality of cameras of shooing at different angles, thereby further promote the accuracy that the foul action detected.
In one example, the color videos and the depth videos are both N segments, the server may intercept a target frame in each color video to obtain N first images, the server finds a same-name frame of the target frame in the depth video corresponding to each color video and intercepts the same-name frame to obtain N second images, the server determines a three-dimensional distance between the first player and the second player in each second image, determines that the first player has a foul behavior when the three-dimensional distance between the first player and the second player in at least one second image is smaller than a preset safety distance, and determines that the first player does not have the foul behavior when the three-dimensional distances between the first player and the second player in all the second images are greater than or equal to the preset safety distance.
In one example, the preset safety distance comprises a first safety distance, a second safety distance and a third safety distance, the first safety distance is smaller than the second safety distance, the second safety distance is smaller than the third safety distance, and the server penalizes the first-level foul of the first player under the condition that the server judges that the three-dimensional distance between the first player and the second player is smaller than the first safety distance; in the case that the three-dimensional distance between the first player and the second player is determined to be greater than or equal to the first safety distance and less than the second safety distance, penalizing the first player a secondary foul; under the condition that the three-dimensional distance between the first player and the second player is judged to be greater than or equal to the second safety distance and smaller than the third safety distance, the first player is penalized with a three-level foul, the penalty of the first-level foul is more severe than that of the second-level foul, and the penalty of the second-level foul is more severe than that of the third-level foul.
In this embodiment, the server captures a target frame of a color video shot by a camera in a sports game, that is, a frame with a suspected foul behavior, to obtain a first image, determines a first player suspected of foul and a second player suspected of foul in the first image, determines a same-name frame with the same time sequence as the target frame in a depth video shot by the same camera with the color video, captures the same-name frame to obtain a second image, then determines a three-dimensional distance between the first player and the second player based on the second image, finally determines whether the three-dimensional distance between the first player and the second player is smaller than a preset safety distance, determines that the first player has the foul behavior in the case that the three-dimensional distance between the first player and the second player is smaller than the preset safety distance, and considers that some foul motions made by the players without physical contact with other players are not easily discovered by the referee, the server detects whether the suspected unlawful player really infringes the safety zone of the suspected unlawful player or not by calculating the three-dimensional distance between the suspected unlawful player and the suspected unlawful player, so that the judge is prompted to perform the penalty of the unlawful action, the unlawful action without physical contact can be scientifically and accurately detected, the occurrence of misjudgment and missed judgment is avoided as much as possible, meanwhile, the calculation process of the three-dimensional distance provides scientific data support, and the interpretability of the penalty of the unlawful action is enhanced.
In one embodiment, the server determines the three-dimensional distance between the first player and the second player based on the second image, which may be implemented by the steps shown in fig. 2, specifically including:
in step 201, a first keypoint of a first player and a second keypoint of a second player are determined in a second image.
In a particular implementation, considering that the position of the first player in the first image is the position of the first player in the second image, and the position of the second player in the first image is the position of the second player in the second image, this is not changed, while, in vigorous sporting competitions, close play is an essential part, where the close proximity of a large part of the athlete's body, or even touching it, is not a safety threat to the athlete, but a small part of the body of the athlete is fragile and easy to be injured, so that the server can detect the parts to carry out scientific and reasonable unlawful behavior detection, in this embodiment, therefore, the server, in determining the three-dimensional distance between the first player and the second player, first and second keypoints for the first and second players may be determined in the second image.
In one example, where the face of a player is fragile, such as the brow bone and the nose bridge bone are both more vulnerable to injury, and thus attempts to hit the face, poke the eyes, etc., are dangerous violations, the server may select the first player's hand as the first keypoint and the second player's nose tip as the second keypoint.
Specifically, after the server determines a first key point of the first player and a second key point of the second player in the second image, a two-dimensional rectangular coordinate system may be established with a lower left corner of the second image as an origin, coordinates of the first key point and coordinates of the second key point are obtained, and a two-dimensional distance between the first key point and the second key point is calculated according to the coordinates of the first key point and the coordinates of the second key point.
Specifically, since the second image is captured from the depth video, the second image is a depth image, and includes depth data of each pixel point, the server may obtain a depth value of the first key point and a depth value of the second key point after determining the first key point of the first player and the second key point of the second player in the second image, and calculate a depth value difference between the first key point and the second key point according to the depth values of the first key point and the second key point.
In a specific implementation, the server may first perform step 202 and then perform step 203, may first perform step 203 and then perform step 202, and may also perform step 202 and step 203 at the same time.
And 204, calculating the three-dimensional distance between the first key point and the second key point according to the two-dimensional distance and the depth value difference value, and taking the three-dimensional distance between the first key point and the second key point as the three-dimensional distance between the first player and the second player.
Specifically, after calculating the two-dimensional distance between the first and second key points and the depth value difference between the first and second key points, the server may calculate the three-dimensional distance between the first and second key points according to the two-dimensional distance between the first and second key points and the depth value difference between the first and second key points based on the principle of triangulation, and use the three-dimensional distance between the first and second key points as the three-dimensional distance between the first and second players.
In the embodiment, considering that in a fierce sports competition, close-distance confrontation is an essential part, the safety of athletes is not threatened by the approach and even touch of most body parts, but a small part of body parts are fragile and easy to be injured, the three-dimensional distance between a first athlete and a second athlete is calculated by selecting a first key point of the first athlete and a second key point of the second athlete, and the calculation process is simple and quick and is more suitable for the actual situation of the sports competition.
In one embodiment, the first keypoints of the first player selected by the server include a left foot of the first player and a right foot of the first player, the second keypoints of the second player selected by the server include a left foot of the second player and a right foot of the second player, so that the three-dimensional distances between the first player and the second player include a first three-dimensional distance between the left foot of the first player and the left foot of the second player, a second three-dimensional distance between the left foot of the first player and the right foot of the second player, a third three-dimensional distance between the right foot of the first player and the left foot of the second player, and a fourth three-dimensional distance between the left foot of the first player and the left foot of the second player, and the server, when determining whether the first player has a foul behavior, may determine whether the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance, and the fourth distance are smaller than a preset safety distance, respectively, the server determines that the first player has a foul behavior under the condition that at least one of the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance is smaller than a preset safety distance, and determines that the first player does not have the foul behavior under the condition that the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance are smaller than the preset safety distance.
In a specific implementation, as shown in fig. 3, considering that in the sports competition of basketball, football and the like, a player often needs to take off to complete actions of shooting, snatching a backboard, fighting a ball and the like, in the process of jumping to landing, feet and ankle joints of athletes are fragile, dangerous foul behaviors like foot cushion are easy to occur in fierce sports competition, the athlete under foot cushion can step on the feet of the opponent when finding that the athlete normally lands, so that the player can avoid the falling by instinct, although the player doing the action of foot pad does not contact with the player to be foot-padded, but the stepping action is real and requires a penalty, so embodiments of the present application capture four three-dimensional distances between the first player's feet and the second player's feet, therefore, whether the foul behaviors exist or not is determined, the accuracy of penalty judgment of the foul behaviors can be further improved, and the fairness of the sports competition is improved.
In one embodiment, the first keypoints of the first player selected by the server include a left foot of the first player and a right foot of the first player, the second keypoints of the second player selected by the server include a head of the second player, so that the three-dimensional distance between the first player and the second player determined by the server includes a fifth three-dimensional distance between the left foot of the first player and the head of the second player and a sixth three-dimensional distance between the right foot of the first player and the head of the second player, the server may determine whether the fifth three-dimensional distance and the sixth three-dimensional distance are smaller than a preset safety distance when determining whether the first player has a foul behavior, respectively, the server may determine whether the fifth three-dimensional distance and the sixth three-dimensional distance are smaller than the preset safety distance when the fifth three-dimensional distance is smaller than the preset safety distance, and/or when the sixth three-dimensional distance is smaller than the preset safety distance, and determining that the first player has a foul behavior, and determining that the first player has no foul behavior under the condition that the fifth three-dimensional distance and the sixth three-dimensional distance are both smaller than a preset safety distance.
In a specific implementation, as shown in fig. 4, considering that in a soccer sports competition, players wear spiked shoes to play, and in an ice sports competition such as a hockey, players wear skates to play, the spikes and the skates are sharp and are dangerous, and therefore, the spikes and the skates are dangerous foul behaviors, and are especially dangerous when being close to the heads of other players, the embodiment of the present application detects the distance between the two feet of the first player and the head of the second player, so as to determine whether the first player has the foul behaviors, and thus, the accuracy of penalty judgment of the foul behaviors can be further improved, and the fairness of the sports competition is improved.
Another embodiment of the present application relates to a method for detecting a foul behavior, and the following details of implementation of the method for detecting a foul behavior of the present embodiment are specifically described, and the following details are only provided for facilitating understanding, and are not necessary for implementing the present solution, and a specific process of the method for detecting a foul behavior of the present embodiment may be as shown in fig. 5, and includes:
Based on the second image, a three-dimensional distance between the first player and the second player is determined, step 303.
And step 304, determining that the first player has a foul behavior under the condition that the three-dimensional distance between the first player and the second player is smaller than the preset safety distance.
And 305, outputting the three-dimensional distance between the first player and the second player and a preset safety distance.
In particular, considering that the penalty of the referee needs to be based on the fact, the interpretability of the penalty can be greatly enhanced by the strong data evidence, so that the embodiment of the application allows detection to be output according to the three-dimensional distance between the first player and the second player and the preset safety distance after the first player is determined to have the penalty, so that the penalty is reasonable.
Another embodiment of the present application relates to a method for detecting a foul behavior, and the implementation details of the method for detecting a foul behavior of the present embodiment are specifically described below, the following are only implementation details provided for easy understanding, and are not necessary for implementing the present solution, and a specific flow of the method for detecting a foul behavior of the present embodiment may be as shown in fig. 6, and includes:
And step 402, determining a frame with the same name in the time sequence as the target frame in the depth video, and intercepting the frame with the same name to obtain a second image.
Based on the second image, a three-dimensional distance between the first player and the second player is determined, step 403.
And step 404, acquiring the injury state information of the second athlete under the condition that the three-dimensional distance is smaller than the preset safety distance.
At step 406, it is determined that the first player has a common foul.
In specific implementation, the server can acquire the injury state information of the second player under the condition that the three-dimensional distance between the first player and the second player is determined to be smaller than the preset safety distance, if the second player is judged to be not injured, the first player is determined to have a common foul behavior, if the second player is judged to be injured, the first player is determined to have a malicious foul behavior, in order to protect the foul player, result-oriented penalty criteria are generally adopted in the sports competition, namely, different degrees of foul penalty are carried out on the foul player according to the injury condition of the foul player, so that the rights and interests of the foul player can be further protected, and the fairness of the sports competition is further improved.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of this patent to add insignificant modifications or introduce insignificant designs to the algorithms or processes, but not to change the core designs of the algorithms and processes.
Another embodiment of the present application relates to a foul behavior detection apparatus, details of implementation of the foul behavior detection apparatus of this embodiment are specifically described below, the following are only details provided for easy understanding, and are not necessary for implementing the present solution, and a schematic diagram of the foul behavior detection apparatus of this embodiment may be as shown in fig. 7, and includes: a camera module 501, a positioning module 502, a calculation module 503 and a detection module 504.
The camera module 501 is used to capture color video and depth video simultaneously.
The positioning module 502 is configured to capture a target frame of a color video to obtain a first image, determine a first athlete and a second athlete in the first image, determine a frame with the same name in time sequence as the target frame in a depth video, and capture the frame with the same name to obtain a second image, where the target frame has a suspected foul behavior, the first athlete is a suspected foul athlete, and the second athlete is a suspected foul athlete.
The calculation module 503 is configured to determine a three-dimensional distance between the first player and the second player based on the second image.
The detection module 504 is configured to determine that the first athlete has a foul behavior if the three-dimensional distance is less than a preset safety distance.
It should be noted that, all the modules involved in this embodiment are logic modules, and in practical application, one logic unit may be one physical unit, may also be a part of one physical unit, and may also be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, a unit that is not so closely related to solving the technical problem proposed by the present application is not introduced in the present embodiment, but this does not indicate that there is no other unit in the present embodiment.
Another embodiment of the present application relates to an electronic device, as shown in fig. 8, including: at least one processor 601; and a memory 602 communicatively coupled to the at least one processor 601; wherein the memory 602 stores instructions executable by the at least one processor 601, the instructions being executable by the at least one processor 601 to enable the at least one processor 601 to perform the foul behavior detection method in the above embodiments.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, etc., which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
Another embodiment of the present application relates to a computer-readable storage medium storing a computer program. The computer program realizes the above-described method embodiments when executed by a processor.
That is, as can be understood by those skilled in the art, all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the present application, and that various changes in form and details may be made therein without departing from the spirit and scope of the present application in practice.
Claims (8)
1. A method for detecting an offending behavior, comprising:
intercepting a target frame of a color video to obtain a first image, and determining a first player and a second player in the first image; wherein a suspected foul act exists in the target frame, the first player is a suspected foul player, and the second player is a suspected foul player;
determining a homonymous frame with the same time sequence as the target frame in the depth video, and intercepting the homonymous frame to obtain a second image; the depth video and the color video are shot by the same camera at the same time;
determining a three-dimensional distance between the first player and the second player based on the second image;
determining that the first player has a foul behavior when the three-dimensional distance is smaller than a preset safety distance;
the determining, based on the second image, a three-dimensional distance between the first player and the second player, comprising:
determining a first keypoint of the first player and a second keypoint of the second player in the second image;
calculating a two-dimensional distance between the first key point and the second key point according to the coordinates of the first key point and the coordinates of the second key point;
calculating a depth value difference value between the first key point and the second key point according to the depth value of the first key point and the depth value of the second key point;
calculating a three-dimensional distance between the first key point and the second key point according to the two-dimensional distance and the depth value difference value, and taking the three-dimensional distance between the first key point and the second key point as the three-dimensional distance between the first player and the second player;
the first keypoints comprise the first player's left foot and the first player's right foot, the second keypoints comprise the second player's left foot and the second player's right foot, the three-dimensional distances between the first player and the second player comprise a first three-dimensional distance between the first player's left foot and the second player's left foot, a second three-dimensional distance between the first player's left foot and the second player's right foot, a third three-dimensional distance between the first player's right foot and the second player's left foot, and a fourth three-dimensional distance between the first player's left foot and the second player's left foot;
the determining that the first player has foul behavior in the case that the three-dimensional distance is smaller than a preset safety distance comprises:
respectively judging whether the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance are smaller than a preset safety distance;
determining that the first player has foul behavior if at least one of the first, second, third, and fourth three-dimensional distances is less than the preset safe distance.
2. The foul behavior detection method of claim 1, wherein the first keypoints comprise a left foot of the first player and a right foot of the first player, the second keypoints comprise a head of the second player, and the three-dimensional distance between the first player and the second player comprises a fifth three-dimensional distance between the left foot of the first player and the head of the second player and a sixth three-dimensional distance between the right foot of the first player and the head of the second player;
the determining that the first player has a foul behavior in the case that the three-dimensional distance is smaller than a preset safe distance comprises:
respectively judging whether the fifth three-dimensional distance and the sixth three-dimensional distance are smaller than a preset safety distance;
determining that the first athlete has a foul behavior if the fifth three-dimensional distance is less than the preset safe distance and/or if the sixth three-dimensional distance is less than the preset safe distance.
3. A foul behaviour detection method according to any one of claims 1-2, wherein after said determination that there is a foul behaviour in the first player, the method further comprises:
outputting the three-dimensional distance between the first player and the second player and the preset safety distance.
4. Foul behaviour detection method according to any one of claims 1 to 2, characterized in that said colour video comprises colour videos taken at several different angles, said depth videos comprising depth videos taken at several different angles, said different angles comprising at least a main view angle, a top view angle and a side view angle.
5. The method according to any one of claims 1 to 2, wherein said determining that the first player has a foul act in case the three-dimensional distance is less than a preset safety distance comprises:
acquiring the injury state information of the second athlete under the condition that the three-dimensional distance is smaller than a preset safety distance; wherein the injury status information includes injured and non-injured;
determining that the first player has a common foul act if the second player is not injured;
if the second player is injured, determining that the first player has malicious foul behavior.
6. An foul behavior detection apparatus, characterized in that the apparatus comprises a camera module, a positioning module, a calculation module and a detection module;
the camera module is used for shooting a color video and a depth video simultaneously;
the positioning module is used for intercepting a target frame of the color video to obtain a first image, determining a first player and a second player in the first image, determining a same-name frame with the same time sequence as the target frame in the depth video, and intercepting the same-name frame to obtain a second image, wherein the target frame has suspected foul behaviors, the first player is a suspected foul player, and the second player is a suspected foul player;
the computing module is to determine a three-dimensional distance between the first player and the second player based on the second image;
the detection module is used for determining that the first athlete has a foul behavior under the condition that the three-dimensional distance is smaller than a preset safety distance;
the determining, based on the second image, a three-dimensional distance between the first player and the second player, comprising:
determining a first keypoint of the first player and a second keypoint of the second player in the second image;
calculating a two-dimensional distance between the first key point and the second key point according to the coordinates of the first key point and the coordinates of the second key point;
calculating a depth value difference value between the first key point and the second key point according to the depth value of the first key point and the depth value of the second key point;
calculating a three-dimensional distance between the first key point and the second key point according to the two-dimensional distance and the depth value difference value, and taking the three-dimensional distance between the first key point and the second key point as the three-dimensional distance between the first player and the second player;
the first keypoints comprise the first player's left foot and the first player's right foot, the second keypoints comprise the second player's left foot and the second player's right foot, the three-dimensional distances between the first player and the second player comprise a first three-dimensional distance between the first player's left foot and the second player's left foot, a second three-dimensional distance between the first player's left foot and the second player's right foot, a third three-dimensional distance between the first player's right foot and the second player's left foot, and a fourth three-dimensional distance between the first player's left foot and the second player's left foot;
the determining that the first player has foul behavior in the case that the three-dimensional distance is smaller than a preset safety distance comprises:
respectively judging whether the first three-dimensional distance, the second three-dimensional distance, the third three-dimensional distance and the fourth three-dimensional distance are smaller than a preset safety distance;
determining that the first player has foul behavior if at least one of the first, second, third, and fourth three-dimensional distances is less than the preset safe distance.
7. An electronic device, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a foul behaviour detection method as claimed in any one of claims 1 to 5.
8. A computer-readable storage medium, storing a computer program, characterized in that the computer program, when executed by a processor, implements a foul behaviour detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210328144.4A CN114419526B (en) | 2022-03-31 | 2022-03-31 | Foul behavior detection method and apparatus, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210328144.4A CN114419526B (en) | 2022-03-31 | 2022-03-31 | Foul behavior detection method and apparatus, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114419526A CN114419526A (en) | 2022-04-29 |
| CN114419526B true CN114419526B (en) | 2022-09-09 |
Family
ID=81262978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210328144.4A Active CN114419526B (en) | 2022-03-31 | 2022-03-31 | Foul behavior detection method and apparatus, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114419526B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115869608A (en) * | 2022-11-29 | 2023-03-31 | 京东方科技集团股份有限公司 | Referee method, device and system for fencing competition and computer readable storage medium |
| CN117612242A (en) * | 2023-05-30 | 2024-02-27 | 黑龙江大学 | Monitoring display system for player motion state tracking foul |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108837478B (en) * | 2018-06-27 | 2020-03-27 | 重庆文理学院 | Foul automatic discrimination system for hurdling competition based on machine vision |
| CN110348524B (en) * | 2019-07-15 | 2022-03-04 | 深圳市商汤科技有限公司 | Human body key point detection method and device, electronic equipment and storage medium |
| CN110942006B (en) * | 2019-11-21 | 2023-04-18 | 中国科学院深圳先进技术研究院 | Motion gesture recognition method, motion gesture recognition apparatus, terminal device, and medium |
-
2022
- 2022-03-31 CN CN202210328144.4A patent/CN114419526B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114419526A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10628678B2 (en) | Classification of activity derived from multiple locations | |
| CN114419526B (en) | Foul behavior detection method and apparatus, electronic device and storage medium | |
| US20210166010A1 (en) | Systems and methods for determining reduced player performance in sporting events | |
| Young et al. | The importance of a sport-specific stimulus for training agility | |
| US20100030350A1 (en) | System and Method for Analyzing Data From Athletic Events | |
| CN111444890A (en) | Sports data analysis system and method based on machine learning | |
| CN112969513B (en) | System and method for determining reduced athlete performance in a sporting event | |
| US10086251B2 (en) | Systems and methods for determining penalties | |
| US20160361593A1 (en) | System, apparatus and method for determining and analyzing sport related activities in conjunction with low latency transmission and processing | |
| Frias et al. | Man-to-man or zone defense? Measuring team dispersion behaviors in small-sided soccer games | |
| Nojima et al. | Augmented dodgeball: an approach to designing augmented sports | |
| CN114842550B (en) | Foul behavior detection method and apparatus, electronic device and storage medium | |
| US20200054930A1 (en) | Method, system and computer program for automatic refereeing | |
| Piergiovanni et al. | Early detection of injuries in mlb pitchers from video | |
| US11024053B1 (en) | User analytics using a mobile device camera and associated systems and methods | |
| CN114842372A (en) | Contact type foul detection method and device, electronic equipment and storage medium | |
| EP4325448A1 (en) | Data processing apparatus and method | |
| EP4325443A1 (en) | Data processing apparatus and method | |
| Tanaka et al. | Automatic Edge Error Judgment in Figure Skating Using 3D Pose Estimation from a Monocular Camera and IMUs | |
| US20190206278A1 (en) | A system and method for sportsman training | |
| KR102500052B1 (en) | Method and apparatus for scoring motion using eye tracking system | |
| Peek et al. | Association between position-specific impact and movement characteristics of professional rugby union players during game play | |
| Murthy et al. | Injury Mechanism Classification in Soccer Videos. | |
| Gupta | Study on Advanced Technologies in Professional Sports Performance | |
| CN112784726A (en) | Method and device for determining target data information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |