CN117953010A - Motion trail tracking method and system - Google Patents
Motion trail tracking method and system Download PDFInfo
- Publication number
- CN117953010A CN117953010A CN202410070839.6A CN202410070839A CN117953010A CN 117953010 A CN117953010 A CN 117953010A CN 202410070839 A CN202410070839 A CN 202410070839A CN 117953010 A CN117953010 A CN 117953010A
- Authority
- CN
- China
- Prior art keywords
- track
- target
- information
- target information
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 112
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 68
- 239000011159 matrix material Substances 0.000 claims description 19
- 238000013528 artificial neural network Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000007547 defect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000000386 athletic effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The application discloses a motion trail tracking method and a motion trail tracking system; the method comprises the steps of obtaining first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of a moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; matching the first target information and the second target information one by one in view to obtain third target information; carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. The high-precision, purely automatic and noninductive moving target tracking is realized.
Description
Technical Field
The present application relates to the field of motion trajectory tracking technologies, and in particular, to a motion trajectory tracking method and system.
Background
Analysis of athletic data provides important support for, and greatly facilitates the development of, many team sports such as football, basketball, rugby, and the like. The motion trail of the moving object is the most basic and key item in motion data analysis. To gain a competitive advantage and reduce the risk of injury, a large number of teams are studying how to design a set of tracking systems that can quantify training and competition characteristics. Currently, tracking systems of moving targets are mainly divided into two types, namely a wearable scheme based on GPS or UWB and a non-wearable scheme based on a camera.
The wearable technical scheme based on GPS is as follows: before a race, each moving object needs to pick up a wearing device, or is bound on an arm, or is fixed on the back. The device comprises a GPS positioning chip, a Bluetooth transmission module or a wifi transmission module. During the competition, the position of the moving object is acquired in real time through a positioning chip on the wearable equipment, and the position information is transmitted back to the server through communication between the transmission module and the field side server. Thereby acquiring the positions of all moving targets in real time. The technical scheme has the following defects: each match is preceded by a wearable device distributed for each moving object, and each wearable device needs to be charged and debugged in advance before being worn. The match is easy to cause the loss of the corresponding moving object information due to the hardware failure of the wearable device. The scheme is a wearing scheme, and can generate burden to the movement of the moving object, so that the movement experience is poor. The acquisition of the location information is based on a GPS chip. However, the positioning accuracy of the GPS is not high, and thus the accuracy of the running information obtained is also not high. And cannot be used for calculating intensity information such as speed and acceleration.
The wearable technical scheme based on UWB is as follows: UWB base stations are required to be built around a movement area, and the UWB base stations can be fixed or movable, and each movement target is required to pick up a wearing device, be bound on an arm or be fixed on the back. The device incorporates UWB locating tags. During the competition, the position of the moving target is acquired through communication between the UWB positioning tags and the surrounding UWB positioning base stations, and the UWB base stations communicate with the field side server through the transmission module to transmit the position information back to the server. Thereby acquiring the positions of all moving targets in real time. The technical scheme has the following defects: each match is preceded by a wearable device distributed for each moving object, and each wearable device needs to be charged and debugged in advance before being worn. The match is easy to cause the loss of the corresponding moving object information due to the hardware failure of the wearable device. The scheme is a wearing scheme, and can generate burden to the motion of the moving object, so that the motion experience is poor. The acquisition of the location information is based on UWB positioning. Because of the principle, when the shielding occurs between the tag and the base station, the positioning information can fluctuate and even jump, and the obtained positioning information is not particularly accurate and cannot be used for calculating the intensity information such as speed, acceleration and the like.
The non-wearing scheme based on the camera is as follows: and a plurality of high-definition cameras are arranged around the movement area to pointedly shoot each area of the movement area. During the competition, the AI algorithm is used for semi-automatic information extraction of the video, and then the data correction and connection are carried out by operators in cooperation with corresponding software, so that the complete tracking of the moving target track is completed. The technical scheme has the following defects: the existing algorithm cannot be completely automated, and a large amount of manual operations are needed to complete tracking of the motion trail of the moving target. Because the camera cannot accurately acquire depth information of a moving object, the position information acquired based on the scheme is the least accurate, and the more distant from the camera, the more inaccurate. And therefore cannot be used to calculate intensity information such as speed and acceleration.
Both the above wearing and non-wearing solutions are currently imperfect. The cumbersome pre-match debugging and unavoidable hardware wear of wearable solutions gives the moving object a poor moving experience. However, the vision-based scheme cannot achieve complete automation because the camera cannot obtain accurate depth information, and a large amount of manpower is often required for access. In addition, the two traditional methods are poor in performance in terms of accuracy and consistency, and tracking tracks of moving targets cannot be accurately acquired.
Disclosure of Invention
The application mainly solves the technical problem of providing a motion trail tracking method and solving the problem that the tracking trail of a moving target cannot be accurately acquired.
In order to solve the technical problems, the application provides a motion trail tracking method, which comprises the following steps:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of the moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; and matching the first target information and the second target information from view to view based on the position information in the motion area to obtain third target information.
Step S2: performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for connecting the track segments according to the starting and ending positions and the ending time of the track segments, and matching the track segments by using the characteristic detection information to obtain the complete tracking track of the moving object.
In some embodiments, the first camera is a lidar or millimeter wave radar capable of determining a distance characteristic of the moving object; the second shooting device is a camera or a camera capable of shooting the external display characteristics of the moving target; the first shooting device and the second shooting device are installed around the movement area in groups, and the shooting areas of the first shooting device and the second shooting device installed around the movement area can completely cover the movement area.
In some embodiments, a motion region coordinate system is established by using one corner point of the motion region as a coordinate source point, and the intrinsic coordinate system of each first photographing device is measuredTransformation matrix/>, with the motion region coordinate system C F Wherein n is the number of the first photographing devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T; calculating the picture plane coordinate system/>, of each second shooting device by using a Zhengyou calibration methodMapping parameters with the motion region coordinate system C F M is the number of second photographing devices; and calibrating the first shooting device, the second shooting device and the moving area coordinate system in a three-in-one way.
In some embodiments, the point clouds at the view angles of a plurality of the first photographing devices will be calculated according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to the motion region coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices, after projectionDirectly fusing and adding the point cloud data projected from each view angle under the motion area coordinate system to form complete target point cloud data P; determining the ground height and a target movement area according to the movement area coordinate system; filtering ground point clouds belonging to the ground and off-site point clouds not belonging to the target motion area to obtain target point cloud data;
Inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises all 3D detection frames of the moving targets in the moving area and centroid positions of the moving targets.
In some embodiments, the 2D neural network is used to sequentially identify and extract the features of the pictures of each view angle, so as to obtain 2D detection frames, appearance features and/or clothes number information of the moving target under each second shooting device picture; after detection results of all visual angles are obtained, projecting the center coordinates of the bottom of the 2D detection frame in the second shooting device picture to the position below the motion area coordinate system; and obtaining second target information, wherein the second target information comprises projection coordinates of the 2D detection frames in the motion area coordinate system under the view angles of the second shooting devices, and appearance characteristics and clothes number information corresponding to each 2D detection frame.
In some embodiments, the track segment with identity consistency is generated using only the position information of the 3D detection box in the third target information using a multi-target tracking algorithm based on euclidean distance criteria and a Kalman filter algorithm.
In some embodiments, creating a plurality of track areas for connecting the track segments according to the starting and ending positions and time of each track segment, and setting the target detection information of the first frame as a starting node and the target detection information of the last frame as an ending node for each track segment t; scribing the temporally adjacent and spatially adjacent nodes into the same track area according to equation (3);
Wherein, For start node or end node,/>To start node,/>To end node,/>For calculating a time difference between two different nodes; /(I)For computing the re-recognition feature similarity between two different nodes.
In some embodiments, for each of the track areas, if only an end node of one of the track segments is within the track area, then that track segment is the input track segment for that track area; if only the starting node is in the track area, the track section is an output track section; if the start node and the end node are both in the track area, the track area is an internal track section; track segment connection is carried out in the track area, and only the input track segment is matched with the output track segment; comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing matching of the input track segment and the output track segment based on the similarity matrix; completing the connection of all the track segments of each moving target to obtain the complete tracking track;
Simi＝θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Wherein: phi (t i,tj) is a similarity model, and is used for extracting representative re-identification features of two track segments and calculating the similarity of the re-identification features between the two; sigma (t i) is used to count the number of clothes contained in the track segment and calculate representative number of clothes information, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which the track segment ends.
The application also provides a motion trail tracking system, which comprises a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a motion target in the motion area, the second shooting device is used for acquiring second target information of the motion target in the motion area, and the edge server is used for executing the motion trail tracking method.
In some embodiments, the edge server is connected to a cloud server, the edge server receives the first target information and the second target information shot by the first shooting device and the second shooting device, and sends the received first target information and second target information to the cloud server, and the cloud server is used for executing the motion trail tracking method.
The beneficial effects of the application are as follows: according to the application, the first target information corresponding to the first shooting device and the second target information corresponding to the second shooting device are matched and fused, so that the final third target information of running perception of the moving target is obtained. Performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. Therefore, the problem that the wearing equipment is required in the wearing scheme is solved, and the problem that automatic tracking and high-precision tracking cannot be processed in the traditional wearable scheme is solved. The high-precision, purely automatic and noninductive moving target tracking is realized.
Drawings
FIG. 1 is a flow chart according to an embodiment of the application;
FIG. 2 is a schematic diagram of a structure according to an embodiment of the application;
FIG. 3 is a schematic diagram of a track area according to an embodiment of the application;
Fig. 4 is a schematic diagram of feature detection information between an input track segment and an output track segment according to an embodiment of the present application.
Detailed Description
In order that the application may be readily understood, a more particular description thereof will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Preferred embodiments of the present application are shown in the drawings. This application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and/or" as used in this specification includes any and all combinations of one or more of the associated listed items.
Fig. 1 shows an embodiment of a motion trajectory tracking method of the present application, including:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of a moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; based on the position information in the motion area, matching the first target information and the second target information from one view angle to another to obtain third target information;
Step S2: carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object.
According to the application, the first target information corresponding to the first shooting device and the second target information corresponding to the second shooting device are matched and fused, so that the final third target information of running perception of the moving target is obtained. Carrying out multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for track segment connection according to the starting and ending positions and time of each track segment, and matching the track segments by using the feature detection information to obtain the complete tracking track of the moving object. Therefore, the problem that the wearing equipment is required in the wearing scheme is solved, and the problem that automatic tracking and high-precision tracking cannot be processed in the traditional wearable scheme is solved. The high-precision, purely automatic and noninductive moving target tracking is realized.
The first imaging device 11 is a device such as a laser radar or a millimeter wave radar capable of measuring characteristics such as a distance from a moving object. The first camera 11 is used for optical detection and ranging, has extremely high accuracy, and is the most complex and reliable way to measure the speed, moving object and position of an object. The first camera 11 may be used indoors, outdoors, in any stadium and in any training sport area 100 and provides accurate, consistent and reliable results. The second photographing device 12 is a video camera, a camera, or the like capable of photographing an external feature of a moving object.
In some embodiments, as shown in fig. 2, an edge server 13 is further disposed around the movement area 100, where the edge server 13 is communicatively connected to the first camera 11 and the second camera 12, and may be connected through wired communication through a network cable, or may be connected through wireless communication such as WiFi, bluetooth, or the like. The above-described motion trajectory tracking method may be performed in the edge server 13.
In some embodiments, the edge server 13 is connected to the cloud server 14, the edge server 13 receives only the first target information and the second target information shot by the first shooting device 11 and the second shooting device 12, sends the received first target information and second target information to the cloud server 14, and performs the above-mentioned motion trail tracking method through the cloud server 14.
In some embodiments, cloud server 14 may be accessed through smart terminal 15, smart terminal 15 may be a desktop, laptop, web server, palm top (PersonalDigital Assistant, PDA), mobile phone, smart bracelet, smart watch, tablet, wireless terminal device, communication device, or embedded device, etc. The cloud server 14 can be conveniently accessed through the intelligent terminal 15 to acquire the tracking track of the moving object.
The sports area 100 may be a football field, a basketball field, a volleyball field, a football field, a badminton field, a tennis field, or the like, and the sports object may be a sports object, a cluster robot, or the like. These exercises require a large number of running objects for collaborative competition by a single person or multiple persons.
The first imaging device 11 and the second imaging device 12 are mounted around the movement region 100, and the imaging regions of the first imaging device 11 and the second imaging device 12 mounted around the movement region 100 can completely cover the movement region 100. The first photographing device 11 and the second photographing device 12 may be uniformly distributed around the movement region 100 at intervals, for example, the first photographing device 11 is disposed at a first position, the second photographing device 12 is disposed at a second position, and the first photographing device 11 is disposed at a third position, thereby being uniformly distributed around the movement region 100, so that the photographing regions of the first photographing device 11 and the second photographing device 12 completely cover the movement region 100.
In some embodiments, the first photographing device 11 and the second photographing device 12 are arranged in groups, that is, at the same photographing position, and the first photographing device 11 and the second photographing device 12 are simultaneously provided, so that wiring can be reduced, and accurate matching of the first target information and the second target information of the same moving target is facilitated. As shown in fig. 2, a plurality of sets of first photographing devices 11 and second photographing devices 12 are installed around the movement region 100, and data collection is performed from a plurality of viewpoints. The installation position of each set of devices is not limited, and as the size of the movement area 100 increases, the number of sets of hardware devices needs to be increased, and the devices need only to be able to cover the target movement area 100.
After the equipment is installed, a motion area coordinate system is established by taking one corner point of the motion area as a coordinate source point. By measuring the intrinsic coordinate system of each first cameraTransformation matrix with motion region coordinate system C F And (3) completing calibration of the first shooting devices, wherein n is the number of the first shooting devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T. Calculating the picture plane coordinate system/>, of each second shooting device by using a Zhengyou calibration methodMapping parameters with motion region coordinate system C F And (3) finishing the calibration of the second shooting devices, wherein m is the number of the second shooting devices. By the method, the three-in-one calibration of the first shooting device, the second shooting device and the movement area coordinate system is completed.
In some embodiments, all of the first cameras and the second cameras are connected to the same edge server through a network cable and a switch. The edge server is internally provided with a time synchronization program for synchronizing the time stamps of all the first shooting devices and the second shooting devices in the movement area and is responsible for data acquisition of all the first shooting devices and the second shooting devices. The first shooting device point cloud data and the second shooting device picture data can be processed on the edge server in a deep processing mode, the processed data are uploaded to the cloud server, and finally the processed data are matched with database information in the cloud server and then pushed to all intelligent terminals.
And projecting the point cloud data of the multi-view first shooting device to a motion area coordinate system, and acquiring first target information by using a 3D neural network. The 3D neural network may be implemented based on any one or a combination of algorithms of PointNet algorithm, voxelNet algorithm, pointPillar algorithm, CENTERNET algorithm, and the like.
In some embodiments, the calibration information of the first camera will be calculated for the point clouds at the plurality of first camera perspectives according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to a motion area coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices after projection And then, directly fusing and adding the point cloud data projected from each view angle under a motion area coordinate system to form complete target point cloud data P. The ground height and the target movement area are then determined from the movement area coordinate system. And filtering the ground point cloud belonging to the ground and the off-site point cloud not belonging to the target movement area to obtain target point cloud data.
And inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises 3D detection frames of all moving targets in a target movement area and the centroid position of the moving targets.
In some embodiments, based on image data from a plurality of second photographing devices, each view angle is processed using a 2D neural network, and the detection result is projected to a motion region coordinate system to acquire second target information. The 2D neural network may be implemented based on any one or a combination of algorithms of FairMOT algorithm, YOLO algorithm, fastReid algorithm, SVHN algorithm, and the like.
In some embodiments, the second photographing device is configured to perform multi-view acquisition on the picture data in the target area. And the multi-view second shooting device synchronously shoots the target area to obtain picture data. And sequentially identifying and extracting the characteristics of the pictures at each view angle by using a 2D neural network to obtain the information such as a 2D detection frame, appearance characteristics, clothes numbers and the like of the moving target under each second shooting device picture.
And after the detection results of all the visual angles are obtained, projecting the bottom center coordinates of the 2D detection frame in the picture of the second shooting device to the coordinate system of the movement region according to the calibration information of the second shooting device. And obtaining second target information comprising projection coordinates of the 2D detection frames in a motion area coordinate system under the view angles of the second shooting devices, and information such as appearance characteristics and clothes numbers corresponding to each detection frame.
In some embodiments, the first target information and the second target information of the multi-view second camera are subjected to hungarian algorithm matching from view to view based on the position in the motion region coordinate system. And obtaining third target information.
Specifically, the first target information includes a 3D detection frame of a moving target generated after point cloud data fusion from the multi-view first photographing device. The second target information comprises projection coordinates of the 2D detection frames in a motion area coordinate system under the view angles of the plurality of second shooting devices, and information such as appearance characteristics and clothes numbers corresponding to each detection frame.
In the process of fusing the first target information and the second target information, the first target information and the information of each view angle in the second target information are sequentially matched. Specifically, in a motion area coordinate system, a 3D detection frame in the first target information and a 2D detection frame obtained from a single visual angle in the second target information are subjected to Hungary matching. Based on the matching result, the appearance feature in the second target information is added to the corresponding 3D detection frame in the first target information. And matching and adding the three visual angles one by one to obtain third target information, wherein the third target information comprises a 3D detection frame from the first target information and appearance characteristics from each visual angle in the second target information, and the 3D detection frame and the appearance characteristics have a corresponding relationship.
In some embodiments, multi-target tracking is performed using a 3D detection box in the third target information to obtain multi-segment track segments. Specifically, only the position information of the 3D detection frame in the third target information is used, and a multi-target algorithm based on the euclidean distance criterion and the Kalman filter algorithm is used to generate the track segment with identity consistency. Wherein the multi-objective tracking algorithm may be implemented based on any one or a combination of algorithms of Trackor ++ algorithm, CENTERTRACK algorithm, jointMOT algorithm, and the like.
In some embodiments, the second target information in each track segment is fused to obtain feature detection information of each track segment. Specifically, all second target information contained in the track segment is extracted based on the obtained track segment, including the moving target appearance feature vector and the clothing number of the moving target for each view angle of each frame in the track segment. All the appearance feature vectors are processed by using a mean algorithm or a clustering algorithm or a neural network to obtain a representative appearance feature vector. And counting all the numbers appearing to obtain the representative number. And fusing the representative appearance feature vector and the representative number as feature detection information of the corresponding track segment.
In some embodiments, as shown in fig. 3. Based on the position and time of the beginning and ending of each track segment, a plurality of track areas (Zoom) for track segment connection are created, and the method can stably and accurately complete the connection between track segments. In fig. 3, t1-t27 respectively represent track segments of different numbers, and circles indicate that this detection information belongs to track segments of different numbers. The marked circle indicates that this frame of information for this track segment is within Zoom.
Specifically, for each track segment t, the target detection information of the first frame is set as the start nodeThe target detection information of the last frame is the end node/>
The nodes that are temporally adjacent and spatially adjacent are drawn into the same track area (Zoom, Z) according to equation (3).
Wherein,For starting or ending nodes, e.g./>To start node,/>In order to end the node,For calculating the time difference between two different nodes. /(I)For computing a re-identification feature (reid Feature) similarity between two different nodes (start node or end node).
In some embodiments, the feature detection information is used to match track segments within each track area (Zoom) to obtain a complete and accurate moving object tracking track.
Specifically, for each track area, if only the end node of a track segment is within the track area, then that track segment is the input track segment for that track area. If only the start node is within the track area, it is the output track segment. An internal track segment if both the start node and the end node are within the track region. The track segment connection is performed in the track area, and only the input track segment and the output track segment are required to be matched.
And (3) comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing the matching of the input track segment and the output track segment based on the similarity matrix. Therefore, connection of all track sections of each moving target can be completed, and a complete and accurate tracking track of the moving target is obtained.
Simi＝θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Phi (t i,tj) is a similarity model for extracting representative re-identification features (REPRESENTATIVE REID features) of two track segments and calculating the re-identification Feature similarity between the two. Sigma (t i) is the information for counting the number of clothes contained in the track segment, and calculates the representative number of clothes, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which it ends. Fig. 4 shows the similarity matrix between the track segments calculated by equations (4) - (6) for each track segment in Zoom 4 in fig. 3. It can be shown by fig. 4 that by matching the Zoom and formulas (4) - (6), similar track segments can be easily matched, and the larger the numerical value, the easier the matching is. Excluding dissimilar track segments.
In some embodiments, the edge server uploads the tracking track to the cloud server in real time, and the tracking track is cleaned in the cloud server for one round and is cross-validated with the database data, and then the data is sent to each intelligent terminal for data display.
From the aspect of the existing system, when in actual use, the application utilizes the first shooting device and the second shooting device which are distributed around the field to synchronously collect the motion data of the moving object in the motion area without dead angle. The first photographing device may effectively measure depth information of the moving object, i.e., a position in a three-dimensional space. The second photographing device can effectively acquire the appearance information of the moving object, such as clothes features, clothes numbers and the like. And finally, merging depth information and appearance information obtained from a plurality of view angles through an artificial intelligence algorithm deployed on an edge server, so that the position of each moving target and the corresponding appearance characteristics of each moving target can be accurately obtained. On the basis, tracking of all moving targets is completed by using a tracking algorithm by combining the acquired position information and the corresponding appearance characteristic information. The problem of need wearing equipment in the wearing scheme is solved, the problem that automatic tracking can not be handled in the traditional no-wearing scheme is solved, and high accuracy tracking is realized. The high-precision, purely automatic and noninductive moving target tracking is realized.
From the algorithm level, the application uses the sensing information from multiple view angles in multiple directions, and solves the problem that the traditional algorithm can not effectively solve the shielding problem in the tracking process. Meanwhile, the fusion of the first target information and the second target information of the conventional algorithm is basically limited to single-frame data. For example, in the context of a football game, when players gather, the fusion of single frames is prone to errors and leads to tracking errors due to the presence of calibration errors. The algorithm of the application stretches the matching of the first target information and the second target information to the track segment level. Therefore, even if a matching error of a single frame occurs, the tracking effect of a moving target is not affected.
The application also provides a motion trail tracking system, which comprises a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a motion target in the motion area, the second shooting device is used for acquiring second target information of the motion target in the motion area, and the edge server is used for executing the motion trail tracking method.
In some embodiments, the edge server is connected to a cloud server, the edge server receives the first target information and the second target information shot by the first shooting device and the second shooting device, and sends the received first target information and second target information to the cloud server, and the cloud server is used for executing the motion trail tracking method.
The foregoing is only illustrative of the present application and is not to be construed as limiting the scope of the application, and all equivalent structural changes made by the present application and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the present application.
Claims (10)
1. The motion track tracking method is characterized by comprising the following steps:
Step S1: acquiring first target information of a moving target in a moving area through a first shooting device, wherein the first target information comprises a detection frame and a mass center of the moving target; acquiring second target information of the moving target in the moving area through a second shooting device, wherein the second target information comprises the apparent characteristics of the moving target; based on the position information in the motion area, matching the first target information and the second target information from one view to another view to obtain third target information;
Step S2: performing multi-target tracking through a detection frame in the third target information to obtain a plurality of track segments; fusing second target information in each track segment to obtain feature detection information of each track segment; and creating a plurality of track areas for connecting the track segments according to the starting and ending positions and the ending time of the track segments, and matching the track segments by using the characteristic detection information to obtain the complete tracking track of the moving object.
2. The motion trajectory tracking method according to claim 1, wherein the first photographing device is a laser radar or a millimeter wave radar capable of measuring a distance characteristic of the moving object; the second shooting device is a camera or a camera capable of shooting the external display characteristics of the moving target; the first shooting device and the second shooting device are installed around the movement area in groups, and the shooting areas of the first shooting device and the second shooting device installed around the movement area can completely cover the movement area.
3. The method of claim 2, wherein a motion area coordinate system is established by using one corner point of the motion area as a coordinate source point, and the intrinsic coordinate system of each first photographing device is measuredTransformation matrix/>, with the motion region coordinate system C F Wherein n is the number of the first photographing devices, and the transformation matrix T comprises a rotation matrix R and a translation matrix T; calculating the picture plane coordinate system of each second shooting device by using a Zhengyou calibration methodMapping parameters/>, between the motion region coordinate system C F M is the number of second photographing devices; and calibrating the first shooting device, the second shooting device and the moving area coordinate system in a three-in-one way.
4. The method according to claim 3, wherein the point clouds at the plurality of first camera angles are obtained according to formulas (1) and (2)Sequentially performing coordinate conversion, and projecting the coordinate conversion to the motion region coordinate system to obtain the point cloud/>, under the view angles of the first shooting devices, after projectionDirectly fusing and adding the point cloud data projected from each view angle under the motion area coordinate system to form complete target point cloud data P; determining the ground height and a target movement area according to the movement area coordinate system; filtering ground point clouds belonging to the ground and off-site point clouds not belonging to the target motion area to obtain target point cloud data;
Inputting the target point cloud data P into a 3D neural network to obtain first target information, wherein the first target information comprises all 3D detection frames of the moving targets in the moving area and centroid positions of the moving targets.
5. The motion trail tracking method according to claim 4, wherein the 2D neural network is used for sequentially identifying and extracting features of pictures of each view angle to obtain 2D detection frames, appearance features and/or clothes number information of the moving object under each second shooting device picture; after detection results of all visual angles are obtained, projecting the center coordinates of the bottom of the 2D detection frame in the second shooting device picture to the position below the motion area coordinate system; and obtaining second target information, wherein the second target information comprises projection coordinates of the 2D detection frames in the motion area coordinate system under the view angles of the second shooting devices, and appearance characteristics and clothes number information corresponding to each 2D detection frame.
6. The motion trajectory tracking method according to claim 5, wherein the trajectory segment having identity consistency is generated using a multi-target tracking algorithm based on a euclidean distance criterion and a Kalman filter algorithm using only position information of a 3D detection frame in the third target information.
7. The motion trajectory tracking method according to claim 6, wherein a plurality of trajectory areas for connection of the trajectory segments are created according to the start and end positions and times of the respective trajectory segments, and for each of the trajectory segments t, target detection information of a first frame thereof is set as a start node and target detection information of a last frame thereof is set as an end node; scribing the temporally adjacent and spatially adjacent nodes into the same track area according to equation (3);
Wherein, For start node or end node,/>To start node,/>To end node,/>For calculating a time difference between two different nodes; /(I)For computing the re-recognition feature similarity between two different nodes.
8. The method according to claim 7, wherein for each of the track areas, if only an end node of one of the track sections is within the track area, the track section is an input track section of the track area; if only the starting node is in the track area, the track section is an output track section; if the start node and the end node are both in the track area, the track area is an internal track section; track segment connection is carried out in the track area, and only the input track segment is matched with the output track segment; comparing the characteristic detection information between the input track segment and the output track segment, establishing a similarity matrix between the track segments through formulas (4) - (6), and completing matching of the input track segment and the output track segment based on the similarity matrix; completing the connection of all the track segments of each moving target to obtain the complete tracking track;
Simi＝θ·Φ(ti,tj)·Ω(ti,tj)ti∈Tstart,tj∈Tend (4)
Wherein: phi (t i,tj) is a similarity model, and is used for extracting representative re-identification features of two track segments and calculating the similarity of the re-identification features between the two; sigma (t i) is used to count the number of clothes contained in the track segment and calculate representative number of clothes information, t i { start_frame } and t i { end_frame } respectively represent the time at which the track segment starts and the time at which the track segment ends.
9. The motion trail tracking system is characterized by comprising a first shooting device, a second shooting device and an edge server, wherein the first shooting device, the second shooting device and the edge server are arranged around a motion area, the edge server is in communication connection with the first shooting device and the second shooting device, the first shooting device is used for acquiring first target information of a moving target in the motion area, the second shooting device is used for acquiring second target information of the moving target in the motion area, and the edge server is used for executing the motion trail tracking method according to any one of claims 1-8.
10. The motion trajectory tracking system of claim 9, wherein the edge server is connected to a cloud server, the edge server receives the first target information and the second target information captured by the first capturing device and the second capturing device, and transmits the received first target information and second target information to the cloud server, and the cloud server is configured to perform the motion trajectory tracking method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410070839.6A CN117953010A (en) | 2024-01-18 | 2024-01-18 | Motion trail tracking method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410070839.6A CN117953010A (en) | 2024-01-18 | 2024-01-18 | Motion trail tracking method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117953010A true CN117953010A (en) | 2024-04-30 |
Family
ID=90804282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410070839.6A Pending CN117953010A (en) | 2024-01-18 | 2024-01-18 | Motion trail tracking method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117953010A (en) |
-
2024
- 2024-01-18 CN CN202410070839.6A patent/CN117953010A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107871120B (en) | Sports event understanding system and method based on machine learning | |
| US11995846B2 (en) | Three-dimensional object tracking using unverified detections registered by one or more sensors | |
| US20180350084A1 (en) | Techniques for object tracking | |
| CN109345568A (en) | Sports ground intelligent implementing method and system based on computer vision algorithms make | |
| CN104700408B (en) | A kind of indoor single goal localization method based on camera network | |
| CN113239797B (en) | Human body action recognition method, device and system | |
| JP2016001447A (en) | Image recognition system, image recognition device, image recognition method and computer program | |
| US11657578B2 (en) | Registration for augmented reality system for viewing an event | |
| Park et al. | Accurate and efficient 3d human pose estimation algorithm using single depth images for pose analysis in golf | |
| CN114120168A (en) | Target running distance measuring and calculating method, system, equipment and storage medium | |
| CN115376034A (en) | Motion video acquisition and editing method and device based on human body three-dimensional posture space-time correlation action recognition | |
| CN111899279A (en) | Method and device for detecting motion speed of target object | |
| CN114037923A (en) | Target activity hotspot graph drawing method, system, equipment and storage medium | |
| US20220129669A1 (en) | System and Method for Providing Multi-Camera 3D Body Part Labeling and Performance Metrics | |
| US20230237748A1 (en) | Augmented reality system for viewing an event with mode based on crowd sourced images | |
| US11880953B2 (en) | Augmented reality system for viewing an event with distributed computing | |
| CN117953010A (en) | Motion trail tracking method and system | |
| CN113421286B (en) | Motion capturing system and method | |
| CN111784749A (en) | Space positioning and motion analysis system based on binocular vision | |
| Lu | Research on Football object Recognition based on Mixed Gaussian model | |
| CN110415302A (en) | Mine positioning system based on image recognition | |
| US20230306682A1 (en) | 3d reference point detection for survey for venue model construction | |
| US12003806B2 (en) | Augmented reality system for viewing an event with multiple coordinate systems and automatically generated model | |
| Wang | Method for Capturing Tennis Training Actions under Machine Vision Images | |
| US20220295139A1 (en) | Augmented reality system for viewing an event with multiple coordinate systems and automatically generated model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |