CN117953010A - Motion trail tracking method and system - Google Patents

Motion trail tracking method and system Download PDF

Info

Publication number
CN117953010A
CN117953010A CN202410070839.6A CN202410070839A CN117953010A CN 117953010 A CN117953010 A CN 117953010A CN 202410070839 A CN202410070839 A CN 202410070839A CN 117953010 A CN117953010 A CN 117953010A
Authority
CN
China
Prior art keywords
track
target
information
target information
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410070839.6A
Other languages
Chinese (zh)
Inventor
黄文斌
孙耀然
何赛灵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202410070839.6A priority Critical patent/CN117953010A/en
Publication of CN117953010A publication Critical patent/CN117953010A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The application discloses a motion trail tracking method and a motion trail tracking system; the method comprises the steps of obtaining first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of a moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; matching the first target information and the second target information one by one in view to obtain third target information; carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. The high-precision, purely automatic and noninductive moving target tracking is realized.

Description

Motion trail tracking method and system
Technical Field
The present application relates to the field of motion trajectory tracking technologies, and in particular, to a motion trajectory tracking method and system.
Background
Analysis of athletic data provides important support for, and greatly facilitates the development of, many team sports such as football, basketball, rugby, and the like. The motion trail of the moving object is the most basic and key item in motion data analysis. To gain a competitive advantage and reduce the risk of injury, a large number of teams are studying how to design a set of tracking systems that can quantify training and competition characteristics. Currently, tracking systems of moving targets are mainly divided into two types, namely a wearable scheme based on GPS or UWB and a non-wearable scheme based on a camera.
The wearable technical scheme based on GPS is as follows: before a race, each moving object needs to pick up a wearing device, or is bound on an arm, or is fixed on the back. The device comprises a GPS positioning chip, a Bluetooth transmission module or a wifi transmission module. During the competition, the position of the moving object is acquired in real time through a positioning chip on the wearable equipment, and the position information is transmitted back to the server through communication between the transmission module and the field side server. Thereby acquiring the positions of all moving targets in real time. The technical scheme has the following defects: each match is preceded by a wearable device distributed for each moving object, and each wearable device needs to be charged and debugged in advance before being worn. The match is easy to cause the loss of the corresponding moving object information due to the hardware failure of the wearable device. The scheme is a wearing scheme, and can generate burden to the movement of the moving object, so that the movement experience is poor. The acquisition of the location information is based on a GPS chip. However, the positioning accuracy of the GPS is not high, and thus the accuracy of the running information obtained is also not high. And cannot be used for calculating intensity information such as speed and acceleration.
The wearable technical scheme based on UWB is as follows: UWB base stations are required to be built around a movement area, and the UWB base stations can be fixed or movable, and each movement target is required to pick up a wearing device, be bound on an arm or be fixed on the back. The device incorporates UWB locating tags. During the competition, the position of the moving target is acquired through communication between the UWB positioning tags and the surrounding UWB positioning base stations, and the UWB base stations communicate with the field side server through the transmission module to transmit the position information back to the server. Thereby acquiring the positions of all moving targets in real time. The technical scheme has the following defects: each match is preceded by a wearable device distributed for each moving object, and each wearable device needs to be charged and debugged in advance before being worn. The match is easy to cause the loss of the corresponding moving object information due to the hardware failure of the wearable device. The scheme is a wearing scheme, and can generate burden to the motion of the moving object, so that the motion experience is poor. The acquisition of the location information is based on UWB positioning. Because of the principle, when the shielding occurs between the tag and the base station, the positioning information can fluctuate and even jump, and the obtained positioning information is not particularly accurate and cannot be used for calculating the intensity information such as speed, acceleration and the like.
The non-wearing scheme based on the camera is as follows: and a plurality of high-definition cameras are arranged around the movement area to pointedly shoot each area of the movement area. During the competition, the AI algorithm is used for semi-automatic information extraction of the video, and then the data correction and connection are carried out by operators in cooperation with corresponding software, so that the complete tracking of the moving target track is completed. The technical scheme has the following defects: the existing algorithm cannot be completely automated, and a large amount of manual operations are needed to complete tracking of the motion trail of the moving target. Because the camera cannot accurately acquire depth information of a moving object, the position information acquired based on the scheme is the least accurate, and the more distant from the camera, the more inaccurate. And therefore cannot be used to calculate intensity information such as speed and acceleration.
Both the above wearing and non-wearing solutions are currently imperfect. The cumbersome pre-match debugging and unavoidable hardware wear of wearable solutions gives the moving object a poor moving experience. However, the vision-based scheme cannot achieve complete automation because the camera cannot obtain accurate depth information, and a large amount of manpower is often required for access. In addition, the two traditional methods are poor in performance in terms of accuracy and consistency, and tracking tracks of moving targets cannot be accurately acquired.
Disclosure of Invention
The application mainly solves the technical problem of providing a motion trail tracking method and solving the problem that the tracking trail of a moving target cannot be accurately acquired.
In order to solve the technical problems, the application provides a motion trail tracking method, which comprises the following steps:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of the moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; and matching the first target information and the second target information from view to view based on the position information in the motion area to obtain third target information.
Step S2: performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for connecting the track segments according to the starting and ending positions and the ending time of the track segments, and matching the track segments by using the characteristic detection information to obtain the complete tracking track of the moving object.
In some embodiments, the first camera is a lidar or millimeter wave radar capable of determining a distance characteristic of the moving object; the second shooting device is a camera or a camera capable of shooting the external display characteristics of the moving target; the first shooting device and the second shooting device are installed around the movement area in groups, and the shooting areas of the first shooting device and the second shooting device installed around the movement area can completely cover the movement area.
In some embodiments, a motion region coordinate system is established by using one corner point of the motion region as a coordinate source point, and the intrinsic coordinate system of each first photographing device is measuredTransformation matrix/>, with the motion region coordinate system C F Wherein n is the number of the first photographing devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T; calculating the picture plane coordinate system/>, of each second shooting device by using a Zhengyou calibration methodMapping parameters with the motion region coordinate system C F M is the number of second photographing devices; and calibrating the first shooting device, the second shooting device and the moving area coordinate system in a three-in-one way.
In some embodiments, the point clouds at the view angles of a plurality of the first photographing devices will be calculated according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to the motion region coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices, after projectionDirectly fusing and adding the point cloud data projected from each view angle under the motion area coordinate system to form complete target point cloud data P; determining the ground height and a target movement area according to the movement area coordinate system; filtering ground point clouds belonging to the ground and off-site point clouds not belonging to the target motion area to obtain target point cloud data;
Inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises all 3D detection frames of the moving targets in the moving area and centroid positions of the moving targets.
In some embodiments, the 2D neural network is used to sequentially identify and extract the features of the pictures of each view angle, so as to obtain 2D detection frames, appearance features and/or clothes number information of the moving target under each second shooting device picture; after detection results of all visual angles are obtained, projecting the center coordinates of the bottom of the 2D detection frame in the second shooting device picture to the position below the motion area coordinate system; and obtaining second target information, wherein the second target information comprises projection coordinates of the 2D detection frames in the motion area coordinate system under the view angles of the second shooting devices, and appearance characteristics and clothes number information corresponding to each 2D detection frame.
In some embodiments, the track segment with identity consistency is generated using only the position information of the 3D detection box in the third target information using a multi-target tracking algorithm based on euclidean distance criteria and a Kalman filter algorithm.
In some embodiments, creating a plurality of track areas for connecting the track segments according to the starting and ending positions and time of each track segment, and setting the target detection information of the first frame as a starting node and the target detection information of the last frame as an ending node for each track segment t; scribing the temporally adjacent and spatially adjacent nodes into the same track area according to equation (3);
Wherein, For start node or end node,/>To start node,/>To end node,/>For calculating a time difference between two different nodes; /(I)For computing the re-recognition feature similarity between two different nodes.
In some embodiments, for each of the track areas, if only an end node of one of the track segments is within the track area, then that track segment is the input track segment for that track area; if only the starting node is in the track area, the track section is an output track section; if the start node and the end node are both in the track area, the track area is an internal track section; track segment connection is carried out in the track area, and only the input track segment is matched with the output track segment; comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing matching of the input track segment and the output track segment based on the similarity matrix; completing the connection of all the track segments of each moving target to obtain the complete tracking track;
Simi=θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Wherein: phi (t i,tj) is a similarity model, and is used for extracting representative re-identification features of two track segments and calculating the similarity of the re-identification features between the two; sigma (t i) is used to count the number of clothes contained in the track segment and calculate representative number of clothes information, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which the track segment ends.
The application also provides a motion trail tracking system, which comprises a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a motion target in the motion area, the second shooting device is used for acquiring second target information of the motion target in the motion area, and the edge server is used for executing the motion trail tracking method.
In some embodiments, the edge server is connected to a cloud server, the edge server receives the first target information and the second target information shot by the first shooting device and the second shooting device, and sends the received first target information and second target information to the cloud server, and the cloud server is used for executing the motion trail tracking method.
The beneficial effects of the application are as follows: according to the application, the first target information corresponding to the first shooting device and the second target information corresponding to the second shooting device are matched and fused, so that the final third target information of running perception of the moving target is obtained. Performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. Therefore, the problem that the wearing equipment is required in the wearing scheme is solved, and the problem that automatic tracking and high-precision tracking cannot be processed in the traditional wearable scheme is solved. The high-precision, purely automatic and noninductive moving target tracking is realized.
Drawings
FIG. 1 is a flow chart according to an embodiment of the application;
FIG. 2 is a schematic diagram of a structure according to an embodiment of the application;
FIG. 3 is a schematic diagram of a track area according to an embodiment of the application;
Fig. 4 is a schematic diagram of feature detection information between an input track segment and an output track segment according to an embodiment of the present application.
Detailed Description
In order that the application may be readily understood, a more particular description thereof will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Preferred embodiments of the present application are shown in the drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and/or" as used in this specification includes any and all combinations of one or more of the associated listed items.
Fig. 1 shows an embodiment of a motion trajectory tracking method of the present application, including:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of a moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; based on the position information in the motion area, matching the first target information and the second target information from one view angle to another to obtain third target information;
Step S2: carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object.
According to the application, the first target information corresponding to the first shooting device and the second target information corresponding to the second shooting device are matched and fused, so that the final third target information of running perception of the moving target is obtained. Carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. Therefore, the problem that the wearing equipment is required in the wearing scheme is solved, and the problem that automatic tracking and high-precision tracking cannot be processed in the traditional wearable scheme is solved. The high-precision, purely automatic and noninductive moving target tracking is realized.
The first imaging device 11 is a device such as a laser radar or a millimeter wave radar capable of measuring characteristics such as a distance from a moving object. The first camera 11 is used for optical detection and ranging, has extremely high accuracy, and is the most complex and reliable way to measure the speed, moving object and position of an object. The first camera 11 may be used indoors, outdoors, in any stadium and in any training sport area 100 and provides accurate, consistent and reliable results. The second photographing device 12 is a video camera, a camera, or the like capable of photographing an external feature of a moving object.
In some embodiments, as shown in fig. 2, an edge server 13 is further disposed around the movement area 100, where the edge server 13 is communicatively connected to the first camera 11 and the second camera 12, and may be connected through wired communication through a network cable, or may be connected through wireless communication such as WiFi, bluetooth, or the like. The above-described motion trajectory tracking method may be performed in the edge server 13.
In some embodiments, the edge server 13 is connected to the cloud server 14, the edge server 13 receives only the first target information and the second target information shot by the first shooting device 11 and the second shooting device 12, sends the received first target information and second target information to the cloud server 14, and performs the above-mentioned motion trail tracking method through the cloud server 14.
In some embodiments, cloud server 14 may be accessed through smart terminal 15, smart terminal 15 may be a desktop, laptop, web server, palm top (PersonalDigital Assistant, PDA), mobile phone, smart bracelet, smart watch, tablet, wireless terminal device, communication device, or embedded device, etc. The cloud server 14 can be conveniently accessed through the intelligent terminal 15 to acquire the tracking track of the moving object.
The sports area 100 may be a football field, a basketball field, a volleyball field, a football field, a badminton field, a tennis field, or the like, and the sports object may be a sports object, a cluster robot, or the like. These exercises require a large number of running objects for collaborative competition by a single person or multiple persons.
The first imaging device 11 and the second imaging device 12 are mounted around the movement region 100, and the imaging regions of the first imaging device 11 and the second imaging device 12 mounted around the movement region 100 can completely cover the movement region 100. The first photographing device 11 and the second photographing device 12 may be uniformly distributed around the movement region 100 at intervals, for example, the first photographing device 11 is disposed at a first position, the second photographing device 12 is disposed at a second position, and the first photographing device 11 is disposed at a third position, thereby being uniformly distributed around the movement region 100, so that the photographing regions of the first photographing device 11 and the second photographing device 12 completely cover the movement region 100.
In some embodiments, the first photographing device 11 and the second photographing device 12 are arranged in groups, that is, at the same photographing position, and the first photographing device 11 and the second photographing device 12 are simultaneously provided, so that wiring can be reduced, and accurate matching of the first target information and the second target information of the same moving target is facilitated. As shown in fig. 2, a plurality of sets of first photographing devices 11 and second photographing devices 12 are installed around the movement region 100, and data collection is performed from a plurality of viewpoints. The installation position of each set of devices is not limited, and as the size of the movement area 100 increases, the number of sets of hardware devices needs to be increased, and the devices need only to be able to cover the target movement area 100.
After the equipment is installed, a motion area coordinate system is established by taking one corner point of the motion area as a coordinate source point. By measuring the intrinsic coordinate system of each first cameraTransformation matrix with motion region coordinate system C F And (3) completing calibration of the first shooting devices, wherein n is the number of the first shooting devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T. Calculating the picture plane coordinate system/>, of each second shooting device by using a Zhengyou calibration methodMapping parameters with motion region coordinate system C F And (3) finishing the calibration of the second shooting devices, wherein m is the number of the second shooting devices. By the method, the three-in-one calibration of the first shooting device, the second shooting device and the movement area coordinate system is completed.
In some embodiments, all of the first cameras and the second cameras are connected to the same edge server through a network cable and a switch. The edge server is internally provided with a time synchronization program for synchronizing the time stamps of all the first shooting devices and the second shooting devices in the movement area and is responsible for data acquisition of all the first shooting devices and the second shooting devices. The first shooting device point cloud data and the second shooting device picture data can be processed on the edge server in a deep processing mode, the processed data are uploaded to the cloud server, and finally the processed data are matched with database information in the cloud server and then pushed to all intelligent terminals.
And projecting the point cloud data of the multi-view first shooting device to a motion area coordinate system, and acquiring first target information by using a 3D neural network. The 3D neural network may be implemented based on any one or a combination of algorithms of PointNet algorithm, voxelNet algorithm, pointPillar algorithm, CENTERNET algorithm, and the like.
In some embodiments, the calibration information of the first camera will be calculated for the point clouds at the plurality of first camera perspectives according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to a motion area coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices after projection And then, directly fusing and adding the point cloud data projected from each view angle under a motion area coordinate system to form complete target point cloud data P. The ground height and the target movement area are then determined from the movement area coordinate system. And filtering the ground point cloud belonging to the ground and the off-site point cloud not belonging to the target movement area to obtain target point cloud data.
And inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises 3D detection frames of all moving targets in a target movement area and the centroid position of the moving targets.
In some embodiments, based on image data from a plurality of second photographing devices, each view angle is processed using a 2D neural network, and the detection result is projected to a motion region coordinate system to acquire second target information. The 2D neural network may be implemented based on any one or a combination of algorithms of FairMOT algorithm, YOLO algorithm, fastReid algorithm, SVHN algorithm, and the like.
In some embodiments, the second photographing device is configured to perform multi-view acquisition on the picture data in the target area. And the multi-view second shooting device synchronously shoots the target area to obtain picture data. And sequentially identifying and extracting the characteristics of the pictures at each view angle by using a 2D neural network to obtain the information such as a 2D detection frame, appearance characteristics, clothes numbers and the like of the moving target under each second shooting device picture.
And after the detection results of all the visual angles are obtained, projecting the bottom center coordinates of the 2D detection frame in the picture of the second shooting device to the coordinate system of the movement region according to the calibration information of the second shooting device. And obtaining second target information comprising projection coordinates of the 2D detection frames in a motion area coordinate system under the view angles of the second shooting devices, and information such as appearance characteristics and clothes numbers corresponding to each detection frame.
In some embodiments, the first target information and the second target information of the multi-view second camera are subjected to hungarian algorithm matching from view to view based on the position in the motion region coordinate system. And obtaining third target information.
Specifically, the first target information includes a 3D detection frame of a moving target generated after point cloud data fusion from the multi-view first photographing device. The second target information comprises projection coordinates of the 2D detection frames in a motion area coordinate system under the view angles of the plurality of second shooting devices, and information such as appearance characteristics and clothes numbers corresponding to each detection frame.
In the process of fusing the first target information and the second target information, the first target information and the information of each view angle in the second target information are sequentially matched. Specifically, in a motion area coordinate system, a 3D detection frame in the first target information and a 2D detection frame obtained from a single visual angle in the second target information are subjected to Hungary matching. Based on the matching result, the appearance feature in the second target information is added to the corresponding 3D detection frame in the first target information. And matching and adding the three visual angles one by one to obtain third target information, wherein the third target information comprises a 3D detection frame from the first target information and appearance characteristics from each visual angle in the second target information, and the 3D detection frame and the appearance characteristics have a corresponding relationship.
In some embodiments, multi-target tracking is performed using a 3D detection box in the third target information to obtain multi-segment track segments. Specifically, only the position information of the 3D detection frame in the third target information is used, and a multi-target algorithm based on the euclidean distance criterion and the Kalman filter algorithm is used to generate the track segment with identity consistency. Wherein the multi-objective tracking algorithm may be implemented based on any one or a combination of algorithms of Trackor ++ algorithm, CENTERTRACK algorithm, jointMOT algorithm, and the like.
In some embodiments, the second target information in each track segment is fused to obtain feature detection information of each track segment. Specifically, all second target information contained in the track segment is extracted based on the obtained track segment, including the moving target appearance feature vector and the clothing number of the moving target for each view angle of each frame in the track segment. All the appearance feature vectors are processed by using a mean algorithm or a clustering algorithm or a neural network to obtain a representative appearance feature vector. And counting all the numbers appearing to obtain the representative number. And fusing the representative appearance feature vector and the representative number as feature detection information of the corresponding track segment.
In some embodiments, as shown in fig. 3. Based on the position and time of the beginning and ending of each track segment, a plurality of track areas (Zoom) for track segment connection are created, and the method can stably and accurately complete the connection between track segments. In fig. 3, t1-t27 respectively represent track segments of different numbers, and circles indicate that this detection information belongs to track segments of different numbers. The marked circle indicates that this frame of information for this track segment is within Zoom.
Specifically, for each track segment t, the target detection information of the first frame is set as the start nodeThe target detection information of the last frame is the end node/>
The nodes that are temporally adjacent and spatially adjacent are drawn into the same track area (Zoom, Z) according to equation (3).
Wherein,For starting or ending nodes, e.g./>To start node,/>In order to end the node,For calculating the time difference between two different nodes. /(I)For computing a re-identification feature (reid Feature) similarity between two different nodes (start node or end node).
In some embodiments, the feature detection information is used to match track segments within each track area (Zoom) to obtain a complete and accurate moving object tracking track.
Specifically, for each track area, if only the end node of a track segment is within the track area, then that track segment is the input track segment for that track area. If only the start node is within the track area, it is the output track segment. An internal track segment if both the start node and the end node are within the track region. The track segment connection is performed in the track area, and only the input track segment and the output track segment are required to be matched.
And (3) comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing the matching of the input track segment and the output track segment based on the similarity matrix. Therefore, connection of all track sections of each moving target can be completed, and a complete and accurate tracking track of the moving target is obtained.
Simi=θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Phi (t i,tj) is a similarity model for extracting representative re-identification features (REPRESENTATIVE REID features) of two track segments and calculating the re-identification Feature similarity between the two. Sigma (t i) is the information for counting the number of clothes contained in the track segment, and calculates the representative number of clothes, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which it ends. Fig. 4 shows the similarity matrix between the track segments calculated by equations (4) - (6) for each track segment in Zoom 4 in fig. 3. It can be shown by fig. 4 that by matching the Zoom and formulas (4) - (6), similar track segments can be easily matched, and the larger the numerical value, the easier the matching is. Excluding dissimilar track segments.
In some embodiments, the edge server uploads the tracking track to the cloud server in real time, and the tracking track is cleaned in the cloud server for one round and is cross-validated with the database data, and then the data is sent to each intelligent terminal for data display.
From the aspect of the existing system, when in actual use, the application utilizes the first shooting device and the second shooting device which are distributed around the field to synchronously collect the motion data of the moving object in the motion area without dead angle. The first photographing device may effectively measure depth information of the moving object, i.e., a position in a three-dimensional space. The second photographing device can effectively acquire the appearance information of the moving object, such as clothes features, clothes numbers and the like. And finally, merging depth information and appearance information obtained from a plurality of view angles through an artificial intelligence algorithm deployed on an edge server, so that the position of each moving target and the corresponding appearance characteristics of each moving target can be accurately obtained. On the basis, tracking of all moving targets is completed by using a tracking algorithm by combining the acquired position information and the corresponding appearance characteristic information. The problem of need wearing equipment in the wearing scheme is solved, the problem that automatic tracking can not be handled in the traditional no-wearing scheme is solved, and high accuracy tracking is realized. The high-precision, purely automatic and noninductive moving target tracking is realized.
From the algorithm level, the application uses the sensing information from multiple view angles in multiple directions, and solves the problem that the traditional algorithm can not effectively solve the shielding problem in the tracking process. Meanwhile, the fusion of the first target information and the second target information of the conventional algorithm is basically limited to single-frame data. For example, in the context of a football game, when players gather, the fusion of single frames is prone to errors and leads to tracking errors due to the presence of calibration errors. The algorithm of the application stretches the matching of the first target information and the second target information to the track segment level. Therefore, even if a matching error of a single frame occurs, the tracking effect of a moving target is not affected.
The application also provides a motion trail tracking system, which comprises a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a motion target in the motion area, the second shooting device is used for acquiring second target information of the motion target in the motion area, and the edge server is used for executing the motion trail tracking method.
In some embodiments, the edge server is connected to a cloud server, the edge server receives the first target information and the second target information shot by the first shooting device and the second shooting device, and sends the received first target information and second target information to the cloud server, and the cloud server is used for executing the motion trail tracking method.
The foregoing is only illustrative of the present application and is not to be construed as limiting the scope of the application, and all equivalent structural changes made by the present application and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the present application.

Claims (10)

1. The motion track tracking method is characterized by comprising the following steps:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of the moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; based on the position information in the motion area, matching the first target information and the second target information from one view to another view to obtain third target information;
Step S2: performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for connecting the track segments according to the starting and ending positions and the ending time of the track segments, and matching the track segments by using the characteristic detection information to obtain the complete tracking track of the moving object.
2. The motion trajectory tracking method according to claim 1, wherein the first photographing device is a laser radar or a millimeter wave radar capable of measuring a distance characteristic of the moving object; the second shooting device is a camera or a camera capable of shooting the external display characteristics of the moving target; the first shooting device and the second shooting device are installed around the movement area in groups, and the shooting areas of the first shooting device and the second shooting device installed around the movement area can completely cover the movement area.
3. The method of claim 2, wherein a motion area coordinate system is established by using one corner point of the motion area as a coordinate source point, and the intrinsic coordinate system of each first photographing device is measuredTransformation matrix/>, with the motion region coordinate system C F Wherein n is the number of the first photographing devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T; calculating the picture plane coordinate system of each second shooting device by using a Zhengyou calibration methodMapping parameters/>, between the motion region coordinate system C F M is the number of second photographing devices; and calibrating the first shooting device, the second shooting device and the moving area coordinate system in a three-in-one way.
4. The method according to claim 3, wherein the point clouds at the plurality of first camera angles are obtained according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to the motion region coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices, after projectionDirectly fusing and adding the point cloud data projected from each view angle under the motion area coordinate system to form complete target point cloud data P; determining the ground height and a target movement area according to the movement area coordinate system; filtering ground point clouds belonging to the ground and off-site point clouds not belonging to the target motion area to obtain target point cloud data;
Inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises all 3D detection frames of the moving targets in the moving area and centroid positions of the moving targets.
5. The motion trail tracking method according to claim 4, wherein the 2D neural network is used for sequentially identifying and extracting features of pictures of each view angle to obtain 2D detection frames, appearance features and/or clothes number information of the moving object under each second shooting device picture; after detection results of all visual angles are obtained, projecting the center coordinates of the bottom of the 2D detection frame in the second shooting device picture to the position below the motion area coordinate system; and obtaining second target information, wherein the second target information comprises projection coordinates of the 2D detection frames in the motion area coordinate system under the view angles of the second shooting devices, and appearance characteristics and clothes number information corresponding to each 2D detection frame.
6. The motion trajectory tracking method according to claim 5, wherein the trajectory segment having identity consistency is generated using a multi-target tracking algorithm based on a euclidean distance criterion and a Kalman filter algorithm using only position information of a 3D detection frame in the third target information.
7. The motion trajectory tracking method according to claim 6, wherein a plurality of trajectory areas for connection of the trajectory segments are created according to the start and end positions and times of the respective trajectory segments, and for each of the trajectory segments t, target detection information of a first frame thereof is set as a start node and target detection information of a last frame thereof is set as an end node; scribing the temporally adjacent and spatially adjacent nodes into the same track area according to equation (3);
Wherein, For start node or end node,/>To start node,/>To end node,/>For calculating a time difference between two different nodes; /(I)For computing the re-recognition feature similarity between two different nodes.
8. The method according to claim 7, wherein for each of the track areas, if only an end node of one of the track sections is within the track area, the track section is an input track section of the track area; if only the starting node is in the track area, the track section is an output track section; if the start node and the end node are both in the track area, the track area is an internal track section; track segment connection is carried out in the track area, and only the input track segment is matched with the output track segment; comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing matching of the input track segment and the output track segment based on the similarity matrix; completing the connection of all the track segments of each moving target to obtain the complete tracking track;
Simi=θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Wherein: phi (t i,tj) is a similarity model, and is used for extracting representative re-identification features of two track segments and calculating the similarity of the re-identification features between the two; sigma (t i) is used to count the number of clothes contained in the track segment and calculate representative number of clothes information, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which the track segment ends.
9. The motion trail tracking system is characterized by comprising a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a moving target in the motion area, the second shooting device is used for acquiring second target information of the moving target in the motion area, and the edge server is used for executing the motion trail tracking method according to any one of claims 1-8.
10. The motion trajectory tracking system of claim 9, wherein the edge server is connected to a cloud server, the edge server receives the first target information and the second target information captured by the first capturing device and the second capturing device, and transmits the received first target information and second target information to the cloud server, and the cloud server is configured to perform the motion trajectory tracking method of any one of claims 1 to 8.
CN202410070839.6A 2024-01-18 2024-01-18 Motion trail tracking method and system Pending CN117953010A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410070839.6A CN117953010A (en) 2024-01-18 2024-01-18 Motion trail tracking method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410070839.6A CN117953010A (en) 2024-01-18 2024-01-18 Motion trail tracking method and system

Publications (1)

Publication Number Publication Date
CN117953010A true CN117953010A (en) 2024-04-30

Family

ID=90804282

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410070839.6A Pending CN117953010A (en) 2024-01-18 2024-01-18 Motion trail tracking method and system

Country Status (1)

Country Link
CN (1) CN117953010A (en)

Similar Documents

Publication Publication Date Title
CN107871120B (en) Sports event understanding system and method based on machine learning
US11995846B2 (en) Three-dimensional object tracking using unverified detections registered by one or more sensors
CN109345568A (en) Sports ground intelligent implementing method and system based on computer vision algorithms make
CN104700408B (en) A kind of indoor single goal localization method based on camera network
CN113239797B (en) Human body action recognition method, device and system
JP2016001447A (en) Image recognition system, image recognition device, image recognition method and computer program
CN103337094A (en) Method for realizing three-dimensional reconstruction of movement by using binocular camera
US11657578B2 (en) Registration for augmented reality system for viewing an event
Park et al. Accurate and efficient 3d human pose estimation algorithm using single depth images for pose analysis in golf
CN114120168A (en) Target running distance measuring and calculating method, system, equipment and storage medium
CN115376034A (en) Motion video acquisition and editing method and device based on human body three-dimensional posture space-time correlation action recognition
CN111899279A (en) Method and device for detecting motion speed of target object
CN114037923A (en) Target activity hotspot graph drawing method, system, equipment and storage medium
US20230237748A1 (en) Augmented reality system for viewing an event with mode based on crowd sourced images
US11880953B2 (en) Augmented reality system for viewing an event with distributed computing
CN117953010A (en) Motion trail tracking method and system
CN113421286B (en) Motion capturing system and method
CN111784749A (en) Space positioning and motion analysis system based on binocular vision
Lu Research on Football object Recognition based on Mixed Gaussian model
CN110415302A (en) Mine positioning system based on image recognition
US20230306682A1 (en) 3d reference point detection for survey for venue model construction
US12003806B2 (en) Augmented reality system for viewing an event with multiple coordinate systems and automatically generated model
Wang Method for Capturing Tennis Training Actions under Machine Vision Images
US20220295139A1 (en) Augmented reality system for viewing an event with multiple coordinate systems and automatically generated model
Pansiot et al. RACKET: Real-time autonomous computation of kinematic elements in tennis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination