WO2023206904A1 - Pedestrian trajectory tracking method and system, and related apparatus - Google Patents

Abstract

The present application relates to the field of image processing, and provides a pedestrian trajectory tracking method, comprising: obtaining image data; performing feature extraction on the image data, and constructing a candidate box relationship mask; extracting a historical frame feature set of a pedestrian trajectory library, and performing feature calculation with a candidate box in the candidate box relationship mask to obtain a human-box feature distance matrix and a box-human feature distance matrix; calculating a feature distance between the target pedestrian and the candidate box, and classifying, under the trajectory of the target pedestrian, a target candidate box satisfying that the feature distances between the target candidate box and the target pedestrian are minimum with respect to each other, until no detection box satisfying the condition exists in a current frame detection box; and updating the pedestrian track library, and outputting a pedestrian index set of the target pedestrian and a corresponding position trajectory. According to the present application, the problem of insufficient feature richness during pedestrian tracking can be effectively solved, and the pedestrian tracking detection precision is improved. The present application further provides a pedestrian trajectory tracking system, a computer readable storage medium and an electronic device, which achieve the beneficial effects above.

Description

A pedestrian trajectory tracking method, system and related devices

Cross-references to related applications

This application claims priority to the Chinese patent application submitted to the China Patent Office on April 30, 2022, with the application number 202210469020.8, and the application title is "A pedestrian trajectory tracking method, system and related devices", the entire content of which is incorporated by reference in this application.

Technical field

The present application relates to the field of image processing, and in particular to a pedestrian trajectory tracking method, system and related devices.

Background technique

Pedestrian target tracking has always been one of the most important research directions in the field of computer vision. Due to its high implementation value and practicality, pedestrian target tracking has attracted the attention of researchers from all aspects.

Multiple Object Tracking (MOT, Multiple Object Tracking) is a difficult topic in the field of target tracking. At this stage, researchers in this field usually combine target detection and metric learning to achieve it. Usually, we use target detection algorithms to detect pedestrians. Positioning, using metric learning to extract features of the located pedestrian, and then realizing the calculation of the trajectory of the same pedestrian through the feature matching strategy. However, due to the large number of tracking targets, the existing strategy will cause more frame missing (False Negative) and ID drift (ID-Switch) phenomena.

Therefore, the inventor realized that how to improve pedestrian tracking accuracy is an urgent technical problem that needs to be solved by those skilled in the art.

Contents of the invention

According to various embodiments disclosed in this application, a pedestrian trajectory tracking method, a pedestrian trajectory tracking system, a computer-readable storage medium, and an electronic device are provided.

A pedestrian trajectory tracking method includes: acquiring image data; performing feature extraction on the image data, and constructing a candidate frame relationship mask based on the extracted features; the value in the candidate frame relationship mask indicates whether the detection frame of the current frame is related to the target pedestrian A reasonable trajectory relationship can be formed; the historical frame feature set of the pedestrian trajectory library is extracted, and the feature calculation is performed with the candidate frames in the candidate frame relationship mask to obtain the person-frame feature distance matrix and the frame-person feature distance matrix; according to the person-frame feature distance matrix The feature distance matrix of the sum frame person calculates the feature distance between the target pedestrian and the candidate frame. In response to the existence of the feature distance between the target candidate frame and the target pedestrian that satisfies the minimum distance between each other, the target candidate frame is classified into the trajectory of the target pedestrian until the current frame detection frame There is no detection frame that meets the conditions; and the pedestrian trajectory library is updated, and the pedestrian index set and the corresponding position trajectory of the target pedestrian are output.

A pedestrian trajectory tracking system includes: an image acquisition module for acquiring image data; a feature extraction module for extracting spatial features and appearance features from the image data, and constructing a candidate frame relationship mask based on the extracted features; features The calculation module is used to extract the historical frame feature set of the pedestrian trajectory library, and performs feature calculation with the candidate frames in the candidate frame relationship mask to obtain the human frame feature distance matrix and the frame human feature distance matrix; the detection module is used to calculate the feature distance matrix based on the human frame The characteristic distance matrix and the framer characteristic distance matrix calculate the characteristic distance between the target pedestrian and the candidate frame. In response to the existence of the characteristic distance between the target candidate frame and the target pedestrian that satisfies the mutual minimum distance, the target candidate frame is classified into the trajectory of the target pedestrian until the current There is no detection frame that meets the conditions in the frame detection frame; and a trajectory update module is used to update the pedestrian trajectory library and output the pedestrian index set and the corresponding position trajectory of the target pedestrian.

A non-volatile computer-readable storage medium has computer-readable instructions stored thereon. When the computer-readable instructions are executed by a processor, the steps of the above method are implemented.

An electronic device includes a memory and one or more processors. Computer-readable instructions are stored in the memory. When the processor calls the computer-readable instructions in the memory, the steps of the above method are implemented.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description, drawings, and claims.

Description of drawings

In order to explain the embodiments of the present application or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only This is an embodiment of the present application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts.

Figure 1 is a flow chart of a pedestrian trajectory tracking method provided in one or more embodiments;

Figure 2 is a flowchart of the steps for extracting features from image data and constructing candidate frame relationship masks based on features provided in one or more embodiments;

Figure 3 is an example diagram of candidate box relationship mask visualization according to one or more embodiments;

Figure 4 is a schematic structural diagram of a pedestrian trajectory tracking system according to one or more embodiments;

Figure 5 is a structural block diagram of an electronic device according to one or more embodiments.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments These are part of the embodiments of this application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

Please refer to FIG. 1 , which is a flow chart of a pedestrian trajectory tracking method provided in one or more embodiments. The method can be applied to electronic devices. The method includes:

S101: Obtain image data;

This step aims to obtain image data. There is no limitation on how to obtain the image data. Video data collected by roadside cameras can usually be used as the source of image data. In response to the fact that the source data is video data, image frame processing can be performed on it to obtain the image data required in this step.

S102: Extract features from the image data and construct a candidate frame relationship mask based on the extracted features;

This step aims to extract features from the image data to construct a candidate box relationship mask. The value in the candidate frame relationship mask indicates whether the detection frame of the current frame and the target pedestrian can form a reasonable trajectory relationship. The execution object of this step is the image data obtained in step S101, which can be processed frame by frame according to this step.

In some embodiments of the present application, as shown in Figure 2, this step of extracting features from the image data and constructing a candidate frame relationship mask based on the extracted features may include the following steps:

S1021: Use the first network model to perform target prediction on the image frames in the image data to obtain the first detection result; wherein the first detection result includes the coordinate frame position information of each pedestrian and the number of pedestrians;

S1022: Use the second network model to extract features in the coordinate frame and obtain a feature set;

S1023: Calculate the trajectory prediction coordinates of each pedestrian at the current moment according to the trajectory prediction formula;

S1024: Determine the spatial feasible range of the pedestrian at the current moment based on the trajectory prediction coordinates and coordinate frame position information of each pedestrian;

S1025: Generate a candidate frame relationship mask corresponding to each pedestrian based on the spatial feasible range; the value in the candidate frame relationship mask indicates whether the detection frame of the current frame and the pedestrian can form a reasonable trajectory relationship.

The pedestrian detection network can be trained using a dual-stage detector or a single-stage detector, and the pedestrian frame labels and images in the training data set are input to the pedestrian detection network, and the network parameters are adjusted to obtain the first network model. The first network model is mainly used to perform pedestrian detection. The pedestrian frame labels and pictures in the training data set can be input to the pedestrian detection network, and the pedestrian detection network can be trained using a dual-stage detector or a single-stage detector to obtain the first network model. .

For the second network model, model training can be performed based on the person re-identification mode (Person Re-Identification), and the pedestrian frames in the training data set can be cropped to obtain the second network model. The second network model is mainly a metric learning network training model.

Specifically, the above process is explained below with relevant formulas:

Perform target detection on the i-th frame image, use the trained first network model to predict the detection result, and obtain the detection result D _i = {p ₁ , p ₂ ,..., p _m }, p _j represents the coordinate box position of each pedestrian [x1, y1, x2, y2], and the coordinates of the upper left corner and lower right corner of the target frame, m represents the number of pedestrians detected in the current frame, and the detection result here corresponds to the first detection result in S1021 above.

Pedestrian target feature extraction is then performed. Use the trained second network model to extract features from the m predicted pedestrian detection frames. F _i ={f ₁ , f ₂ ,..., f _m } can be obtained.

Then perform spatial condition restriction calculation: predict the trajectory of r pedestrians in the pedestrian trajectory library T, where x and y respectively represent the coordinates of the target pedestrian on the two-dimensional image, k represents the index of the target pedestrian in T, and lqx(* ) represents the least squares formula used to fit the trajectory curve. According to the trajectory prediction formula, the trajectory prediction coordinates (x _{t+1, k} , y _{t+1, k} ) of each pedestrian at the current moment are calculated, that is, the trajectory coordinates of each pedestrian at the next moment at the current moment, where, y _{t+1, k} represents the longitudinal predicted coordinate value of the k-th target pedestrian, and x _{t+1, k} represents the lateral predicted coordinate value of the k-th target pedestrian. Here, the trajectory prediction formula can use the least squares formula, but it is not limited to this method. Among them, the least squares formula is as follows:

x _t+1,k =lqx({x _t,k |t∈T,k∈[1,...,r]})

y _t+1,k =lqx({y _t,k |t∈T,k∈[1,...,r]})

In the formula, lqx(*) is the least squares formula used to fit the trajectory curve, (x _t,k ,y _t,k ) represents the trajectory coordinates of the k-th target pedestrian at the current moment, where, x _t,k represents the abscissa value of the k-th target pedestrian at the current moment, and y _t,k represents the ordinate value of the k-th target pedestrian at the current moment.

After that, for each pedestrian k, according to its trajectory prediction coordinates (x _{t+1, k} , y _{t+1, k} ) and its final moment target box size [w _{t, k} , h _{t, k} ], calculate its The feasible range of space at the current moment

S _{t, k} =

{(x, y)|x∈[x _t+1,k -λ _w ×w _t,k ,x _t+1,k +λ _w ×w _t,k ],

y∈[y _{t+1, k} -λ _h ×h _{t, k} , y _{t+1, k} +λ _h ×h _{t, k} ]}

Among them, λ is the expansion coefficient of the target frame. Correspondingly, λ _h represents the vertical expansion coefficient of the target frame, λ _w represents the horizontal expansion coefficient of the target frame, and thresh is a threshold, which is a parameter that can be set to limit the "feasible"Range", the degree of overlap between a certain frame (i.e. one of the frames) s and the target frame S _{t, k} must be greater than this value. S is the set of all s that meet the conditions, that is, the "feasible range".

Afterwards, the candidate frame relationship mask can be generated, and the detection results of the current frame are combined with the spatial condition constraints of r pedestrians in T to calculate the candidate frame relationship mask M for each pedestrian in T. There is no limitation on the representation method of the candidate frame relationship mask. A feasible method is that the candidate frame relationship mask is an M*N matrix containing 0 and 1, where M is the number of pedestrian detection frames and N is the pedestrian. quantity.

We can obtain the feasible range of all pedestrians (assuming N people) in the current frame (a total of N S), in addition to all pedestrian detection frames detected in the current frame (assuming M b), by comparing whether each b belongs to each S, an M×N matrix can be obtained. The elements in the i-th row and j-th column of the matrix represent the situation where the i-th detection frame may be used as the j-th pedestrian candidate frame in the pedestrian library. Referring to Figure 3, Figure 3 is an example diagram of candidate box relationship mask visualization according to one or more embodiments. In Figure 2, 1 represents true, 0 represents false, and 1 represents the existence of potential candidate relationships between related pedestrians. Of course, other methods can also be used to identify the candidate frame relationship mask, which are not limited to examples here. For example, 1 represents true, -1 represents false, and so on.

S103: Extract the historical frame feature set from the pedestrian trajectory database, perform feature calculations with the candidate frames in the candidate frame relationship mask, and obtain the person frame feature distance matrix and the frame person feature distance matrix;

This step is aimed at feature extraction and feature distance calculation. Specifically, the historical frame feature set of each pedestrian in the pedestrian trajectory library can be extracted, and the cosine distance operation is performed on the candidate frame of the current frame to obtain the human frame feature distance matrix Disttd. In addition, the candidate frame of the current frame has a candidate relationship with it. Perform feature distance calculation on the pedestrians to obtain the frame-person distance matrix Distdt. Disttd represents the distance matrix between t pedestrians in the library and d candidate boxes in the current frame. Conversely, Distdt represents the distance matrix between the candidate boxes and pedestrians. In the "matching" process, for the i-th pedestrian and j-th box, If and only if Disttd[i,j]==min(Disttd[i,*]) and Distdt[j,i]==min(Distdt[j,*]), candidate box j is assigned to pedestrian i. The function min(array) represents the minimum value of the array array; matric[j,*] represents the j-th row array of the matrix matric.

S104: In response to the feature distance between the target pedestrian and the target candidate frame being the minimum distance from each other, classify the target candidate frame into the trajectory of the target pedestrian until there is no detection frame that meets the conditions in the current frame detection frame;

In response to the characteristic distance between pedestrian k and the candidate frame p of the current frame satisfying the recall condition of being the minimum distance to each other, the candidate frame p is classified into the trajectory of pedestrian k. Repeat this operation until there is no target that meets the conditions in the current frame detection frame Di. This part of the frame is set as a new pedestrian and stored in the pedestrian trajectory library.

S105: Update the pedestrian trajectory database and output the pedestrian index set and the corresponding position trajectory of the target pedestrian.

The matched pedestrian frame, its position and characteristics are stored in the relevant pedestrian, its position information and feature queue are updated, and finally the pedestrian index set and the corresponding position trajectory of the target pedestrian can be output.

There is no specific limit on when to build the pedestrian trajectory database. It only requires that a corresponding database or data queue exists during the application process. It is also possible to construct a pedestrian trajectory database before executing this embodiment. The pedestrian trajectory database includes the historical location of each pedestrian and the characteristic information of each historical location.

The embodiment of the present application constructs a candidate frame relationship mask by extracting features from the image data, which is used to judge the rationality of the trajectory relationship between pedestrians and detection frames identified in the image data, and then compares the history in the pedestrian trajectory database. The frame feature set is used to calculate the feature distance to determine the target pedestrian and the target candidate frame, thereby determining the pedestrian trajectory in the image data and realizing pedestrian trajectory tracking. This application can effectively solve the problem of pedestrian index replacement caused by spatial distance weighting in extremely large scenes, as well as the problem of insufficient feature richness in the pedestrian tracking process, which leads to excessive dependence on the metric learning model, and improves the accuracy of pedestrian tracking detection.

It should be understood that although the steps in the flowcharts of Figures 1 and 2 are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 1 and 2 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps or stages The order of execution is not necessarily sequential, but may be performed in turn or alternately with other steps or sub-steps of other steps or at least part of the stages.

The pedestrian trajectory tracking system provided by the embodiment of the present application is introduced below. The pedestrian trajectory tracking system described below and the pedestrian trajectory tracking method described above can be mutually referenced.

Referring to Figure 4, Figure 4 is a schematic structural diagram of a pedestrian trajectory tracking system according to one or more embodiments. This application also provides a pedestrian trajectory tracking system, including an image acquisition module, a feature extraction module, a feature calculation module, a detection module and Track update module, including:

Image acquisition module, used to acquire image data;

The feature extraction module is used to extract spatial features and appearance features from image data, and construct a candidate frame relationship mask based on the extracted features;

The feature calculation module is used to extract the historical frame feature set of the pedestrian trajectory library, perform feature calculation with the candidate frames in the candidate frame relationship mask, and obtain the person frame feature distance matrix and the frame person feature distance matrix;

The detection module is used to calculate the characteristic distance between the target pedestrian and the candidate frame based on the human frame characteristic distance matrix and the frame human characteristic distance matrix. In response to the existence of the characteristic distance between the target candidate frame and the target pedestrian that satisfies the mutual minimum distance, the target candidate frame is classified into Enter the trajectory of the target pedestrian until there is no detection frame that meets the conditions in the current frame detection frame;

The trajectory update module is used to update the pedestrian trajectory library and output the pedestrian index set and the corresponding position trajectory of the target pedestrian.

In some embodiments of the present application, the human trajectory tracking system may also include a trajectory library building module, which is used to construct a pedestrian trajectory library; the pedestrian trajectory library includes the historical location of each pedestrian and the time the pedestrian is at each historical location. characteristic information.

In some embodiments of the present application, the feature extraction module includes a first feature extraction unit, a second feature extraction unit, a trajectory calculation unit, a spatial prediction unit, and a candidate frame relationship mask generation unit, wherein:

The first feature extraction unit is used to use the first network model to perform target prediction on the image frames in the image data to obtain the first detection result; wherein the first detection result includes the coordinate frame position information of each pedestrian and the number of pedestrians;

The second feature extraction unit is used to extract features in the coordinate frame using the second network model to obtain a feature set;

The trajectory calculation unit is used to calculate the trajectory prediction coordinates of each pedestrian at the current moment according to the trajectory prediction formula;

The spatial prediction unit is used to determine the spatial feasible range of the pedestrian at the current moment based on the trajectory prediction coordinates and coordinate frame position information of each pedestrian;

The candidate frame relationship mask generation unit is used to generate the candidate frame relationship mask corresponding to each pedestrian based on the spatial feasible range; the value in the candidate frame relationship mask indicates whether the detection frame of the current frame and the pedestrian can form a reasonable trajectory relationship.

In some embodiments of the present application, the pedestrian trajectory tracking system also includes a first network model generation module. The first network model generation module is used to input pedestrian frame labels and pictures in the training data set into the pedestrian detection network, and uses dual The stage detector or single stage detector trains the pedestrian detection network to obtain the first network model.

In some embodiments of the present application, the pedestrian trajectory tracking system also includes a second network model generation module, which is used to perform model training based on the pedestrian re-identification mode and crop the pedestrian frame in the training data set, Obtain the second network model.

For specific limitations on the pedestrian trajectory tracking system, please refer to the limitations on the pedestrian trajectory tracking method mentioned above, which will not be described again here. Each module in the above-mentioned pedestrian trajectory tracking system can be implemented in whole or in part through software, hardware, and combinations thereof. Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

This application also provides a non-volatile computer-readable storage medium on which computer-readable instructions are stored. When executed, the computer-readable instructions can implement the steps provided in any of the above embodiments. The non-volatile computer-readable storage medium can include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc. A medium on which program code can be stored.

This application also provides an electronic device, as shown in Figure 5, which may include a memory and one or more processors. Computer-readable instructions are stored in the memory. When the processor calls the computer-readable instructions in the memory, The steps provided in any of the above embodiments can be implemented.

Among them, the processor can be implemented in at least one hardware form among digital signal processing DSP (Digital Signal Processing), field programmable gate array FPGA (Field-Programmable Gate Array), and programmable logic array PLA (Programmable Logic Array). The processor can also include a main processor and a co-processor. The main processor is used to process data in the wake-up state, also called the central processing unit (CPU); the co-processor is used to process data in the wake-up state. A low-power processor that processes data in standby mode. In some embodiments, the processor may be integrated with a graphics processor GPU (Graphics Processing Unit), which is responsible for rendering and drawing content to be displayed on the display screen. In some embodiments of the present application, the processor may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

Memory may include one or more computer-readable storage media, which may be non-volatile. Memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is at least used to store the following computer-readable instructions. After the computer-readable instructions are loaded and executed by the processor, the relevant steps in the pedestrian trajectory tracking method disclosed in any of the foregoing embodiments can be implemented. In addition, the resources stored in the memory may also include operating systems and data, and the storage method may be short-term storage or permanent storage. The data may include but is not limited to the data involved in the above methods.

In some embodiments of the present application, of course the electronic device may also include various network interfaces, some or all of the components such as power supplies, display screens, power supplies, input and output interfaces, sensors, and communication buses.

Each embodiment in the specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. As for the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple. For relevant details, please refer to the description in the method section.

This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art, several improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or sequence between operations. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element qualified by the statement "comprises a..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

Claims (15)

1. A pedestrian trajectory tracking method, characterized by including:

   Get image data;

   Perform feature extraction on the image data, and construct a candidate frame relationship mask based on the extracted features; the value in the candidate frame relationship mask indicates whether the detection frame of the current frame and the target pedestrian can form a reasonable trajectory relationship;

   Extract the historical frame feature set from the pedestrian trajectory library, perform feature calculations with the candidate frames in the candidate frame relationship mask, and obtain the person frame feature distance matrix and the frame person feature distance matrix; and

   Calculate the characteristic distance between the target pedestrian and the candidate frame according to the person frame characteristic distance matrix and the frame person characteristic distance matrix, in response to the existence of the characteristic distance between the target candidate frame and the target pedestrian that satisfies the mutual minimum distance, Classify the target candidate frame into the trajectory of the target pedestrian until there is no detection frame that meets the conditions in the current frame detection frame;

   The pedestrian trajectory database is updated, and the pedestrian index set and the corresponding position trajectory of the target pedestrian are output.
2. The pedestrian trajectory tracking method according to claim 1, further comprising:

   The pedestrian trajectory database is constructed; the pedestrian trajectory database includes the historical location of each pedestrian and the characteristic information of the pedestrian at each historical location.
3. The pedestrian trajectory tracking method according to claim 1 or 2, characterized in that performing feature extraction on the image data and constructing a candidate frame relationship mask based on the extracted features includes:

   Using the first network model to perform target prediction on the image frames in the image data, the first detection result is obtained; wherein the first detection result includes the coordinate frame position information of each pedestrian and the number of pedestrians;

   Use the second network model to extract features in the coordinate frame to obtain a feature set;

   Calculate the trajectory prediction coordinates of each pedestrian at the current moment according to the trajectory prediction formula;

   Determine the spatial feasible range of the pedestrian at the current moment based on the trajectory prediction coordinates of each pedestrian and the coordinate frame position information; and

   A candidate frame relationship mask corresponding to each pedestrian is generated according to the spatial feasible range.
4. The pedestrian trajectory tracking method according to claim 3, characterized in that calculating the trajectory prediction coordinates of each pedestrian at the current moment according to the trajectory prediction formula includes:

   The predicted trajectory coordinates of each pedestrian at the current moment are calculated according to the least squares formula.
5. The pedestrian trajectory tracking method according to claim 3 or 4, characterized in that the spatial feasible range of the pedestrian at the current moment is determined based on the trajectory prediction coordinates of each pedestrian and the coordinate frame position information, include:

   Calculate the spatial feasible range of pedestrian k at the current moment according to the following formula;

   

   S _t,k ={(x,y)|x∈[x _t+1,k -λ _w ×w _t,k ,x _t+1,k +λ _w ×w _t,k ]

   y∈[y _{t+1, k} -λ _h ×h _{t, k} , y _{t+1, k} +λ _h ×h _{t, k} ]}

   Among them, thresh is the set threshold parameter, s represents one of the boxes, S _{t, k} represents the target frame, (x, y) represents the coordinates of the target pedestrian, (x _t+1,k ,y _t+1,k ) represents The trajectory prediction coordinates of the k-th target pedestrian, [w _t+1,k ,h _t+1,k ] represents the size of the target frame, λ _h represents the longitudinal expansion coefficient of the target frame, λ _w represents the horizontal expansion coefficient of the target frame, y _{t+1, k} represents the longitudinal predicted coordinate value of the k-th target pedestrian, x _{t+1, k} represents the lateral predicted coordinate value of the k-th target pedestrian, and S is a set of boxes that meet the conditions.
6. The pedestrian trajectory tracking method according to any one of claims 3 to 5, characterized in that, before using the first network model to perform target prediction on the image frames in the image data and obtaining the first detection result, it also includes:

   Input the pedestrian frame labels and pictures in the training data set to the pedestrian detection network, and use a dual-stage detector or a single-stage detector to train the pedestrian detection network to obtain the first network model.
7. The pedestrian trajectory tracking method according to any one of claims 3 to 6, characterized in that before using the second network model to extract features in the coordinate frame and obtaining the feature set, it also includes:

   Model training is performed based on the pedestrian re-identification mode, and pedestrian frames in the training data set are cropped to obtain the second network model.
8. The pedestrian trajectory tracking method according to any one of claims 1 to 7, characterized in that the candidate frame relationship mask is an M*N matrix containing 0 and 1; where M is the number of pedestrian detection frames, N is the number of pedestrians.
9. The pedestrian trajectory tracking method according to any one of claims 1 to 8, characterized in that the feature set of historical frames extracted from the pedestrian trajectory library is calculated with the candidate frames in the candidate frame relationship mask to obtain The person-frame feature distance matrix and the frame-person feature distance matrix include:

   Extract the historical frame feature set of each pedestrian in the pedestrian trajectory database, perform cosine distance operations on its candidate frames of the current frame, and obtain the human frame feature distance matrix; and

   Perform feature distance operations on the candidate frames of the current frame and the pedestrians with candidate relationships to obtain the frame-person distance matrix.
10. The pedestrian trajectory tracking method according to any one of claims 1 to 9, characterized in that, in response to the existence of a target candidate frame and the characteristic distance of the target pedestrian satisfying each other's minimum distance, the target candidate frame is classified into Enter the trajectory of the target pedestrian, including:

    In response to Disttd[i,j]==min(Disttd[i,*]) and Distdt[j,i]==min(Distdt[j,*]), classify the j-th candidate box into the i-th pedestrian traces of;

    Among them, Disttd[i, j] represents the person frame feature distance matrix between the i-th pedestrian and the j-th candidate frame, and Distdt[j, i] represents the frame-person feature distance matrix between the i-th pedestrian and the j-th candidate frame.
11. The pedestrian trajectory tracking method according to any one of claims 1 to 10, characterized in that the obtaining image data includes:

    Obtain video data, perform image frame processing on the video data, and obtain the image data.
12. A pedestrian trajectory tracking system is characterized by including:

    Image acquisition module, used to acquire image data;

    A feature extraction module, used to extract spatial features and appearance features from the image data, and construct a candidate frame relationship mask based on the extracted features; the numerical value in the candidate frame relationship mask represents the detection frame and target of the current frame. Whether pedestrians can form a reasonable trajectory relationship;

    The feature calculation module is used to extract the historical frame feature set of the pedestrian trajectory library, perform feature calculation with the candidate frames in the candidate frame relationship mask, and obtain the person frame feature distance matrix and the frame person feature distance matrix;

    A detection module configured to calculate the characteristic distance between the target pedestrian and the candidate frame according to the human frame characteristic distance matrix and the frame human characteristic distance matrix, in response to the existence of the characteristic distance between the target candidate frame and the target pedestrian satisfying being the minimum distance from each other, the target candidate frame is classified into the trajectory of the target pedestrian until there is no detection frame that meets the conditions in the current frame detection frame; and

    A trajectory update module is used to update the pedestrian trajectory library and output the pedestrian index set and the corresponding position trajectory of the target pedestrian.
13. The pedestrian trajectory tracking system according to claim 12, further comprising:

    A trajectory library construction module is used to construct a pedestrian trajectory library; the pedestrian trajectory library includes the historical location of each pedestrian and the characteristic information of the pedestrian at each historical location.
14. A non-volatile computer-readable storage medium on which computer-readable instructions are stored, characterized in that when the computer-readable instructions are executed by a processor, the pedestrian movement as described in any one of claims 1 to 11 is realized. The steps of the trajectory tracking method.
15. An electronic device, characterized in that it includes a memory and one or more processors. Computer-readable instructions are stored in the memory. When the processor calls the computer-readable instructions in the memory, it implements the rights as claimed. The steps of the pedestrian trajectory tracking method according to any one of claims 1 to 11.

Images